A chaos engineering parameter adjustment method and device, an electronic device, and a program product

Abstract

The application discloses a chaos engineering parameter adjustment method and device, electronic equipment and program product, and relates to the technical field of computers. The method comprises the following steps: acquiring a real-time state vector of a distributed system; inputting the real-time state vector into a reinforcement learning intelligent agent, outputting a fault injection parameter instruction of a continuous action space through the reinforcement learning intelligent agent; performing safety constraint verification on the fault injection parameter instruction; performing a fault injection operation on the distributed system based on the fault injection parameter instruction, and acquiring a fault injection execution result; calculating a reward value corresponding to the current fault injection parameter instruction according to the fault injection execution result; and updating the policy network parameters of the reinforcement learning intelligent agent based on the reward value. The technical scheme of the application realizes adaptive adjustment of the fault injection parameter in chaos engineering testing of the distributed system, and effectively improves the safety and efficiency of the chaos engineering testing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device and program product for adjusting chaotic engineering parameters. Background Technology

[0002] In the field of chaotic engineering testing of distributed systems, adaptive adjustment of fault injection parameters is one of the key technologies to ensure the effectiveness of distributed system stability testing. Traditional fault injection schemes often adopt manual fixed parameter or static threshold configuration strategies, and rely on operation and maintenance personnel to manually adjust parameters such as fault intensity and injection interval based on business experience. This makes it difficult to adapt to the complex testing requirements of different business loads, fluctuations in system operating indicators, and dynamic changes in multiple types of fault scenarios.

[0003] Therefore, there is an urgent need for a chaotic engineering parameter adjustment method that can achieve adaptive adjustment of fault injection parameters and ensure the safe operation of distributed systems. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and program product for adjusting chaotic engineering parameters.

[0005] In a first aspect, embodiments of this application provide a method for adjusting chaotic engineering parameters, including: Obtain the real-time state vector of the distributed system; The real-time state vector is input into the reinforcement learning agent, which then outputs fault injection parameter instructions for the continuous action space. Perform safety constraint verification on the fault injection parameter command; determine whether the fault parameters in the fault injection parameter command exceed the range limited by the preset constraint rules; if they do, adjust the fault injection parameter command to the range limited by the constraint rules. The system performs fault injection operations based on fault injection parameter instructions and obtains the fault injection execution results. Calculate the reward value corresponding to the current fault injection parameter instruction based on the fault injection execution result; The policy network parameters of the reinforcement learning agent are updated based on the reward value.

[0006] In some embodiments, the preset constraint rules include upper limits for the node impact ratio and lower limits for the continuous injection interval, which are set separately for different fault types.

[0007] In some embodiments, determining whether the fault parameters in the fault injection parameter instruction exceed the range defined by preset constraint rules includes: Determine the fault type corresponding to the fault injection parameter command; Obtain the upper limit of the node impact ratio and the lower limit of the continuous injection interval corresponding to the fault type; Determine whether the node influence ratio in the fault injection parameter command is greater than the upper limit of the node influence ratio, and determine whether the time interval between two adjacent fault injections corresponding to the fault injection parameter command is less than the lower limit of the continuous injection interval.

[0008] In some embodiments, during the execution of a fault injection operation, the operating metrics of the distributed system are monitored. When the operating metrics of the distributed system exceed the preset safety threshold, the current fault injection operation is terminated, and a snapshot rollback operation is performed to restore the distributed system to its operating state before the fault injection operation.

[0009] In some embodiments, the reward value corresponding to the current fault injection parameter instruction is calculated based on the fault injection execution result, including: Based on the results of the fault injection execution, determine the fault coverage metric and the business impact metric; Obtain the real-time load status of a distributed system; Based on the real-time load status, adjust the first weight corresponding to the fault coverage indicator and the second weight corresponding to the business impact indicator; wherein, the first weight is negatively correlated with the real-time load status, the second weight is positively correlated with the real-time load status, and the sum of the first weight and the second weight is a preset constant. The reward value is calculated based on the first weight, the second weight, the fault coverage rate indicator, and the business impact indicator.

[0010] In some embodiments, the real-time load status includes the CPU utilization and memory usage of the distributed system; the fault coverage metric is calculated based on the ratio of the number of injected fault types to the total number of preset injectable fault types. Business impact metrics are calculated based on the request failure rate and response latency of the distributed system.

[0011] In some embodiments, updating the policy network parameters of the reinforcement learning agent based on the reward value includes: The real-time state vector, fault injection parameter instructions, reward value, and real-time state vector at the next moment after each fault injection operation are stored as a set of experience samples in the experience replay pool. A batch of experience samples is drawn from the experience replay pool, and the policy network parameters of the reinforcement learning agent are updated using the batch of experience samples.

[0012] Secondly, embodiments of this application provide a chaotic engineering parameter adjustment device, including: The status acquisition module is configured to acquire the real-time status vector of the distributed system. The instruction generation module is configured to input the real-time state vector into the reinforcement learning agent and generate fault injection parameter instructions in the continuous action space through the reinforcement learning agent. The safety constraint module is configured to perform safety constraint verification on the fault injection parameter command; determine whether the fault parameters in the fault injection parameter command exceed the range limited by the preset constraint rules; if they exceed the range, adjust the fault injection parameter command to the range limited by the constraint rules. The fault injection module is configured to perform fault injection operations on the distributed system according to fault injection parameter instructions and obtain the fault injection execution results. The reward calculation module is configured to calculate the reward value corresponding to the current fault injection parameter instruction based on the fault injection execution result. The policy update module is configured to update the policy network parameters of the reinforcement learning agent based on the reward value.

[0013] In some embodiments, the reward calculation module includes: The metrics determination submodule is configured to determine fault coverage metrics and business impact metrics based on the results of fault injection execution. The load acquisition submodule is configured to acquire the real-time load status of the distributed system. The weight adjustment submodule is configured to adjust the first weight corresponding to the fault coverage indicator and the second weight corresponding to the business impact indicator according to the real-time load status. The first weight is negatively correlated with the real-time load status, the second weight is positively correlated with the real-time load status, and the sum of the first weight and the second weight is a preset constant. The reward calculation submodule is configured to calculate the reward value based on the first weight, the second weight, the fault coverage metric, and the business impact metric.

[0014] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement any one of the chaotic engineering parameter adjustment methods.

[0015] Fourthly, embodiments of this application provide a computer program product, including a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any one of the chaotic engineering parameter adjustment methods.

[0016] The at least one technical solution adopted in this application can achieve the following beneficial effects: by outputting fault injection parameter instructions based on the real-time state vector of the system through a reinforcement learning agent, the parameters are adaptively adjusted under different loads and fault scenarios, improving the targeting and coverage efficiency of fault injection; by limiting the proportion of nodes injected in a single injection and the time interval of continuous injection through security constraint verification, the scope or frequency of fault injection is avoided to prevent the business continuity of the distributed system during the testing process; by continuously monitoring the operating indicators of the distributed system during the injection process, and automatically terminating the fault injection and performing snapshot rollback when the indicators deviate from the preset safety threshold, the additional risks introduced by fault injection are reduced; by calculating the reward value based on the fault injection execution result and updating the policy network parameters of the reinforcement learning agent using the constructed experience samples, the update and iteration of fault injection parameter instructions are realized. Through the technical solutions of this application, the security and efficiency of chaotic engineering testing of distributed systems are effectively improved.

[0017] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0018] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 This illustration schematically shows an overall process diagram of a chaos engineering parameter adjustment method according to an embodiment of this application; Figure 2 This illustration schematically shows a safety constraint verification process for a chaos engineering parameter adjustment method according to an embodiment of this application. Figure 3 This illustration schematically shows a reward value calculation process for a chaos engineering parameter adjustment method according to an embodiment of this application. Figure 4 This illustration schematically shows a fault injection termination and rollback process of a chaos engineering parameter adjustment method according to an embodiment of this application. Figure 5 This illustration schematically shows a strategy network parameter update process of a chaos engineering parameter adjustment method according to an embodiment of this application; Figure 6 This illustration schematically shows an overall structural diagram of a chaos engineering parameter adjustment device according to an embodiment of this application; Figure 7 This illustration schematically shows a reward calculation module structure of a chaos engineering parameter adjustment device according to an embodiment of this application; Figure 8 The illustration shows a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application; Figure 9 An exemplary block diagram of a computer program product according to an embodiment of this application is illustrated schematically.

[0020] In the diagram: 601, Status Acquisition Module; 602, Instruction Generation Module; 603, Security Constraint Module; 604, Fault Injection Module; 605, Reward Calculation Module; 606, Policy Update Module; 607, Indicator Determination Submodule; 608, Load Acquisition Submodule; 609, Weight Adjustment Submodule; 610, Reward Calculation Submodule; 800, Electronic Equipment; 801, Processor; 802, Memory; 901, Computer Program. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0022] The chaos engineering parameter adjustment method provided in this application can be applied to scenarios such as chaos engineering testing of distributed systems, stability verification of distributed systems, or chaos engineering experimental platforms. For example, this method can be applied to chaos testing scenarios of public cloud distributed systems, fault drill scenarios of private cloud microservice architectures, or system resilience assessment scenarios of hybrid cloud environments, for adaptive adjustment and security control of fault injection parameters in distributed systems. The execution subject of the chaos engineering parameter adjustment method provided in this application is a chaos engineering management server or an electronic device integrating this method. The following is combined with... Figure 1-9 This application provides a detailed description of a method for adjusting chaotic engineering parameters according to embodiments.

[0023] Figure 1 The illustration shows an overall flowchart of a method for adjusting chaotic engineering parameters according to an embodiment of this application.

[0024] like Figure 1 As shown, the steps include: S101, obtain the real-time state vector of the distributed system.

[0025] Specifically, various operational metrics during the operation of the distributed system are collected, including but not limited to CPU utilization, memory usage, request failure rate, and response latency. These metrics are then integrated into a real-time state vector. The real-time state vector represents the current operational state of the distributed system and serves as input to the reinforcement learning agent. By uniformly representing the multi-dimensional operational state of the system as a state vector, the reinforcement learning agent can comprehensively perceive the current operational state of the distributed system, providing a data foundation for subsequent decision-making.

[0026] S102, input the real-time state vector to the reinforcement learning agent, and output the fault injection parameter instruction in the continuous action space through the reinforcement learning agent.

[0027] Specifically, the acquired real-time state vector is input into the reinforcement learning agent. The reinforcement learning agent processes the input real-time state vector based on the current policy network, outputting fault injection parameter instructions in the continuous action space. These instructions include fault type, fault intensity, fault duration, and fault impact range. The fault injection parameter instructions are continuously selected values ​​in the continuous action space to adapt to the differentiated requirements of fault injection granularity in different testing scenarios.

[0028] S103 performs safety constraint verification on the fault injection parameter command.

[0029] Specifically, the system determines whether the fault parameters in the fault injection parameter command exceed the safe range based on preset constraint rules. If the fault parameters exceed the range defined by the preset constraint rules, the fault parameters in the fault injection parameter command are adjusted to the range defined by the constraint rules. Through safety constraint verification, the fault injection experiment is restricted to execution within the safe boundary, avoiding risky actions by the reinforcement learning agent that may seriously affect the distributed system due to exploration or decision-making biases, thus ensuring the business continuity of the distributed system during chaos engineering testing.

[0030] S104: Perform fault injection operation on the distributed system based on fault injection parameter instructions, and obtain the fault injection execution result.

[0031] Specifically, based on the fault type, fault strength, fault duration, and fault impact range specified in the fault injection parameter instructions, the corresponding fault is injected into the target node of the distributed system, and the fault injection execution result is obtained. The fault injection execution result includes whether the fault was successfully injected, the actual impact range of the fault injection, and the duration of the fault injection. By obtaining the fault injection execution result, a data foundation is provided for the subsequent calculation of the reward value.

[0032] S105, Calculate the reward value corresponding to the current fault injection parameter instruction based on the fault injection execution result.

[0033] Specifically, the fault coverage metric and business impact metric are determined based on the fault injection execution results, and the real-time load status of the distributed system is obtained. The first weight corresponding to the fault coverage metric and the second weight corresponding to the business impact metric are adjusted according to the real-time load status. The first weight is negatively correlated with the real-time load status, and the second weight is positively correlated with the real-time load status. The sum of the first weight and the second weight is a preset constant. The reward value is calculated based on the first weight, the second weight, the fault coverage metric, and the business impact metric.

[0034] S106, update the policy network parameters of the reinforcement learning agent based on the reward value.

[0035] Specifically, the real-time state vector, fault injection parameter command, reward value, and the real-time state vector at the next moment after each fault injection operation are stored as a set of experience samples in the experience replay pool. A batch of experience samples is extracted from the experience replay pool, and these batch experience samples are used to update the policy network parameters of the reinforcement learning agent. Storing historical experience samples in the experience replay pool reduces the correlation between samples, improves the stability and convergence efficiency of the policy network training, and enables the reinforcement learning agent to continuously optimize the generation strategy of fault injection parameter commands during the iteration process.

[0036] The following combination Figure 2 right Figure 1 The security constraint verification process in the method shown will be further explained.

[0037] Figure 2 The illustration shows a schematic diagram of the safety constraint verification process of a chaotic engineering parameter adjustment method according to an embodiment of this application.

[0038] like Figure 2 As shown, the steps include: S201, Obtain Fault Injection Parameter Command.

[0039] Specifically, the fault injection parameter command is a continuous action space parameter output by the reinforcement learning agent based on the real-time state vector, including fault type, fault intensity, fault duration, and fault impact range. The fault impact range is represented by the node impact ratio, i.e., the proportion of nodes affected by the fault injection to the total number of nodes in the distributed system. After obtaining the fault injection parameter command, it is used as input for security constraint verification, checking each parameter in the command to see if it meets the preset constraint rules.

[0040] S202, determine the fault type corresponding to the fault injection parameter command.

[0041] Specifically, the obtained fault injection parameter instructions are parsed to extract the fault type carried within, thus determining the fault type corresponding to this fault injection operation. Fault types include network latency faults, node termination faults, and CPU overload faults. Different types of faults have different degrees and ways of impacting the distributed system; subsequent steps require obtaining the corresponding security constraint parameters for each fault type.

[0042] S203, obtain the upper limit of the node impact ratio and the lower limit of the continuous injection interval corresponding to the fault type.

[0043] Specifically, after determining the fault type, the corresponding upper limit of node impact ratio and lower limit of continuous injection interval are retrieved from the preset constraint rules based on the fault type. The upper limit of node impact ratio is the maximum node proportion that a single fault injection is allowed to affect, and the lower limit of continuous injection interval is the minimum time interval allowed between two adjacent fault injections.

[0044] For example, the upper limit of the node impact ratio corresponding to a network latency failure can be configured to 20%, and the lower limit of the continuous injection interval can be configured to 15 minutes; the upper limit of the node impact ratio corresponding to a node termination failure can be configured to 10%, and the lower limit of the continuous injection interval can be configured to 30 minutes; the upper limit of the node impact ratio corresponding to a CPU full load failure can be configured to 15%, and the lower limit of the continuous injection interval can be configured to 20 minutes. By setting differentiated constraint rules for different failure types, precise security control over the injection of different types of failures can be achieved.

[0045] S204, determine whether the node influence ratio in the fault injection parameter instruction is greater than the upper limit of the node influence ratio, and whether the time interval between two adjacent fault injections corresponding to the fault injection parameter instruction is less than the lower limit of the continuous injection interval.

[0046] Specifically, the node influence ratio in the fault injection parameter command is compared with the obtained upper limit of the node influence ratio. If the node influence ratio is greater than the upper limit, it is determined that the node influence ratio exceeds the limit, and the node influence ratio in the fault injection parameter command is adjusted down to the upper limit. Simultaneously, the time difference between the current time and the time of the last fault injection operation is obtained as the time interval between two adjacent fault injections corresponding to the fault injection parameter command. This fault injection time interval is compared with the obtained lower limit of the continuous injection interval. If the time interval is less than the lower limit of the continuous injection interval, it is determined that the injection time interval exceeds the limit, and the execution time of this fault injection is changed to meet the lower limit of the continuous injection interval before execution. If both the node influence ratio and the fault injection time interval exceed the limits, both of the above adjustment operations are performed simultaneously. Through the above judgment and adjustment, it is ensured that each execution of the fault injection operation meets the preset constraint rules.

[0047] S205 outputs the fault injection parameter command after safety constraint verification.

[0048] Specifically, after completing the above constraint verification and parameter adjustment, a fault injection parameter instruction within the safety constraints is obtained. All fault parameters in this instruction have been confirmed to be within the safety range defined by the constraint rules. Based on this instruction, the distributed system is driven to perform a safe and controllable fault injection operation.

[0049] The following combination Figure 3 right Figure 1 The reward value calculation process in the method shown will be further explained.

[0050] Figure 3 The illustration shows a schematic diagram of the reward value calculation process of a chaotic engineering parameter adjustment method according to an embodiment of this application.

[0051] like Figure 3 As shown, the steps include: S301, based on the results of the fault injection execution, determine the fault coverage rate and business impact indicators.

[0052] Specifically, the fault injection execution result includes information such as whether the fault was successfully injected and the actual scope of its impact. Based on the fault injection execution result, fault coverage and business impact metrics are determined. The fault coverage metric measures the extent to which the fault injection operation covers potential fault scenarios in the distributed system, while the business impact metric measures the degree of interference caused by the fault injection operation to the normal operation of the distributed system.

[0053] S302, obtain the real-time load status of the distributed system.

[0054] Specifically, real-time load status includes the CPU utilization and memory usage of the distributed system, which characterize the load pressure level of the distributed system at the current moment. During the reward value calculation process, the current CPU utilization and memory usage of the distributed system are obtained as the basis for subsequent dynamic weight adjustment, enabling the reward value to be calculated based on the current load pressure level of the distributed system, thereby achieving differentiated reward value calculation under different load conditions.

[0055] S303 adjusts the first weight of the fault coverage metric and the second weight of the business impact metric based on the real-time load status.

[0056] Specifically, based on the obtained CPU utilization and memory usage of the distributed system, the first weight corresponding to the fault coverage metric and the second weight corresponding to the business impact metric are adjusted. The first weight is negatively correlated with the real-time load state; that is, when the real-time load state increases, the first weight decreases, thus reducing the contribution of the fault coverage metric to the reward value. The second weight is positively correlated with the real-time load state; that is, when the real-time load state increases, the second weight increases, thus increasing the contribution of the business impact metric to the reward value. Simultaneously, the sum of the first and second weights is a preset constant to ensure that the two weights remain balanced, avoiding an overall shift in the reward value due to weight adjustments. Through this dynamic adjustment of weights, the calculation of the reward value can adaptively balance the conflict between fault coverage and business impact based on the current load pressure level of the distributed system.

[0057] S304 calculates the reward value based on the first weight, the second weight, the fault coverage rate indicator, and the business impact indicator.

[0058] Specifically, after determining the first weight, second weight, fault coverage metric, and business impact metric, a weighted fusion calculation is performed on the fault coverage metric and business impact metric based on the first and second weights to obtain the reward value corresponding to the current fault injection parameter command. The reward value comprehensively reflects the overall performance of this fault injection operation in both fault coverage and business impact dimensions. A higher reward value indicates that good fault coverage has been achieved while ensuring business stability. The reward value is used for subsequent updates to the policy network parameters of the reinforcement learning agent to guide it in continuously optimizing the generation strategy of fault injection parameter commands.

[0059] The following combination Figure 4 right Figure 1 The fault injection termination and rollback process in the method shown will be further explained.

[0060] Figure 4 The illustration shows a fault injection termination and rollback process of a chaos engineering parameter adjustment method according to an embodiment of this application.

[0061] like Figure 4 As shown, the steps include: S401 monitors the operating metrics of the distributed system during the fault injection operation.

[0062] Specifically, while injecting faults into the nodes of the distributed system using fault injection parameter commands, the system continuously collects operational metrics, including but not limited to request failure rate, response latency, CPU utilization, and memory usage. By continuously monitoring the operational status of the distributed system during fault injection, changes in these metrics can be detected in real time.

[0063] S402, determine whether the operating indicators exceed the preset safety threshold.

[0064] Specifically, the collected operational metrics of the distributed system are compared with their corresponding preset safety thresholds to determine whether any metric exceeds its preset safety threshold. The preset safety thresholds define the safety boundary for the normal operation of the distributed system. When the operational metrics are within the safety threshold range, it indicates that the distributed system is currently operating normally, and the fault injection operation has not seriously affected the stability of the distributed system; the fault injection operation continues. When any operational metric exceeds the preset safety threshold, it indicates that the operating state of the distributed system has deviated from the safety boundary, and the fault injection operation may have had an unacceptable impact on the stability or business continuity of the distributed system. In this case, a rollback is immediately triggered, terminating the currently executing fault injection operation, stopping the injection of faults into the nodes of the distributed system, preventing the scope of the fault's impact from further expanding; and a snapshot rollback is performed to restore the state of the distributed system to the state before the execution of this fault injection operation, eliminating all kinds of impacts caused by this fault injection on the distributed system.

[0065] By using the aforementioned termination fault injection and snapshot-based distributed system recovery, the distributed system can be restored to a normal state in a timely manner when various operating indicators deviate from the preset safety threshold, effectively reducing the risks caused by uncontrolled fault injection operations.

[0066] The following combination Figure 5 right Figure 1 The process of updating the policy network parameters in the method shown will be further explained.

[0067] Figure 5 The illustration shows a schematic diagram of the strategy network parameter update process of a chaos engineering parameter adjustment method according to an embodiment of this application.

[0068] like Figure 5 As shown, the steps include: S501 stores the real-time state vector, fault injection parameter instructions, reward value, and the real-time state vector at the next moment after each fault injection operation as a set of experience samples into the experience replay pool.

[0069] Specifically, after each fault injection operation, the reinforcement learning agent obtains the real-time state vector, fault injection parameter instructions, reward value, and the real-time state vector at the next moment after the fault injection. This data is combined to construct a set of experience samples, which are then stored in the experience replay pool. The historical experience samples stored in the experience replay pool can be sampled and reused multiple times. This batch sampling method updates the policy network, thereby reducing the impact of temporal correlation between consecutive experience samples on training and improving the stability and convergence efficiency of the policy network training.

[0070] S502, extracts a batch of experience samples from the experience replay pool, and uses the batch of experience samples to update the policy network parameters of the reinforcement learning agent.

[0071] Specifically, when the number of experience samples stored in the experience replay pool reaches the preset training condition, a batch of experience samples is randomly drawn from the experience replay pool, and the policy network parameters of the reinforcement learning agent are updated using the drawn batch of experience samples. By randomly sampling historical data from the experience replay pool for training, the temporal correlation between consecutive experience samples is reduced, thereby improving the stability and convergence efficiency of the policy network training.

[0072] For example, the policy network of a reinforcement learning agent can adopt an architecture of action network and evaluation network. During training, based on the extracted batch of experience samples, the temporal difference error between the current Q-value output by the evaluation network and the target Q-value is calculated, and this temporal difference error is used to update the parameters of the evaluation network. The parameter update of the action network is based on the action value gradient output by the evaluation network, and iterative optimization is performed in the manner of policy gradient. Through the above training process, the parameters of the action network and the evaluation network are updated and optimized.

[0073] The specific implementation of the action network and evaluation network mentioned above is not limited to the DDPG algorithm. Other reinforcement learning algorithms suitable for continuous action spaces, such as the SAC algorithm, TD3 algorithm, or PPO algorithm, can also be used.

[0074] Corresponding to the method embodiments of this application, this application also provides a chaotic engineering parameter adjustment device.

[0075] Figure 6 The schematic diagram illustrates the overall structure of a chaotic engineering parameter adjustment device according to an embodiment of this application.

[0076] like Figure 6 As shown, the device includes a status acquisition module 601, an instruction generation module 602, a security constraint module 603, a fault injection module 604, a reward calculation module 605, and a strategy update module 606.

[0077] Among them, the state acquisition module 601 is configured to acquire the real-time state vector of the distributed system.

[0078] Specifically, the status acquisition module 601 is used to collect various operating indicators during the operation of the distributed system, including but not limited to CPU utilization, memory usage, request failure rate, response latency, etc., integrate the above operating indicators into a real-time status vector, and output the real-time status vector.

[0079] The instruction generation module 602 is configured to input the real-time state vector to the reinforcement learning agent and generate fault injection parameter instructions for the continuous action space through the reinforcement learning agent.

[0080] Specifically, the reinforcement learning agent processes the input real-time state vector based on the current policy network, outputs fault injection parameter instructions in the continuous action space, and then outputs these instructions. The fault injection parameter instructions include fault type, fault strength, fault duration, and fault impact range. These instructions are continuously selected values ​​in the continuous action space to adapt to the varying granularity of fault injection required by different testing scenarios.

[0081] The safety constraint module 603 is configured to perform safety constraint verification on the fault injection parameter command; determine whether the fault parameters in the fault injection parameter command exceed the range limited by the preset constraint rules; if they exceed the range, adjust the fault injection parameter command to the range limited by the constraint rules.

[0082] Specifically, the security constraint module 603, based on the fault injection parameter command, determines whether the fault parameters in the command exceed the safe range according to preset constraint rules. If the fault parameters exceed the range defined by the preset constraint rules, the security constraint module 603 adjusts the fault parameters to the range defined by the constraint rules. By restricting the fault injection operation to execution within the safe boundary through the security constraint module 603, the stability and business continuity of the distributed system during chaos engineering testing are ensured.

[0083] The fault injection module 604 is configured to perform fault injection operations on the distributed system according to fault injection parameter instructions and obtain the fault injection execution results.

[0084] Specifically, the fault injection module 604 injects the corresponding fault into the nodes of the distributed system according to the fault type, fault strength, fault duration, and fault impact range in the fault injection parameter instruction, and obtains the fault injection execution result. The fault injection execution result includes whether the fault was successfully injected, the actual impact range of the fault injection, and the duration of the fault injection.

[0085] The reward calculation module 605 is configured to calculate the reward value corresponding to the current fault injection parameter instruction based on the fault injection execution result.

[0086] Specifically, the reward calculation module 605 determines the fault coverage rate indicator and the business impact indicator based on the fault injection execution result, and obtains the real-time load status of the distributed system. Based on the real-time load status, it adjusts the first weight corresponding to the fault coverage rate indicator and the second weight corresponding to the business impact indicator. The first weight is negatively correlated with the real-time load status, and the second weight is positively correlated with the real-time load status. The sum of the first weight and the second weight is a preset constant. The reward value is calculated based on the first weight, the second weight, the fault coverage rate indicator, and the business impact indicator, thereby realizing the quantitative evaluation of fault coverage and business impact.

[0087] The policy update module 606 is configured to update the policy network parameters of the reinforcement learning agent based on the reward value.

[0088] Specifically, the policy update module 606 stores the real-time state vector, fault injection parameter command, reward value, and the real-time state vector at the next moment after each fault injection operation as a set of experience samples in the experience replay pool. It then extracts a batch of experience samples from the experience replay pool and uses these batch experience samples to update the policy network parameters of the reinforcement learning agent in the command generation module 602. By storing and reusing historical experience samples, the policy update module 606 reduces the correlation between samples, improves the stability and convergence efficiency of the policy network training, and enables the reinforcement learning agent to continuously optimize the generation strategy of fault injection parameter commands during the iteration process.

[0089] The following combination Figure 7 right Figure 6 The structure of the reward calculation module 605 in the device shown will be further explained.

[0090] Figure 7 The diagram illustrates the structure of the reward calculation module of a chaos engineering parameter adjustment device according to an embodiment of this application.

[0091] like Figure 7 As shown, the reward calculation module 605 includes an indicator determination submodule 607, a load acquisition submodule 608, a weight adjustment submodule 609, and a reward calculation submodule 610.

[0092] Among them, the indicator determination submodule 607 is configured to determine the fault coverage indicator and the business impact indicator based on the fault injection execution result.

[0093] Specifically, the metric determination submodule 607 determines the fault coverage metric and the business impact metric based on the fault injection execution results. The fault coverage metric measures the extent to which the fault injection operation covers potential fault scenarios in the distributed system, while the business impact metric measures the degree of interference caused by the fault injection operation to the normal operation of the distributed system.

[0094] The load acquisition submodule 608 is configured to acquire the real-time load status of the distributed system.

[0095] Specifically, the load acquisition submodule 608 acquires the current CPU utilization and memory usage of the distributed system as the real-time load status.

[0096] The weight adjustment submodule 609 is configured to adjust the first weight corresponding to the fault coverage indicator and the second weight corresponding to the business impact indicator according to the real-time load status. The first weight is negatively correlated with the real-time load status, the second weight is positively correlated with the real-time load status, and the sum of the first weight and the second weight is a preset constant.

[0097] The reward calculation submodule 610 is configured to calculate the reward value based on a weighted fusion of the first weight, the second weight, the fault coverage rate indicator, and the business impact indicator.

[0098] This application also provides an electronic device for implementing the above-described method embodiments.

[0099] Figure 8 The illustration shows a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application.

[0100] like Figure 8 As shown, the electronic device 800 includes a processor 801 and a memory 802. The memory 802 stores a computer program 901. When the processor 801 executes the computer program 901, it implements the chaotic engineering parameter adjustment method provided in any embodiment of this application.

[0101] This application also provides a computer program product for implementing the above-described method embodiments.

[0102] Figure 9 An exemplary block diagram of a computer program product according to an embodiment of this application is illustrated schematically.

[0103] like Figure 9 As shown, the computer program product stores a computer program 901. When the computer program 901 is executed by the processor 801, it implements the chaotic engineering parameter adjustment method provided in any embodiment of this application.

[0104] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0105] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0106] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0107] It should also be noted that in the system and method of this application, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of this application.

[0108] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this application is not limited to the specific aspects of the processes, machines, manufacturing, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufacturing, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described above can be utilized. Therefore, the appended claims include such processes, machines, manufacturing, events, means, methods, or actions within their scope.

[0109] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0110] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for adjusting chaotic engineering parameters, characterized in that, include: Obtain the real-time state vector of the distributed system; The real-time state vector is input to the reinforcement learning agent, and the reinforcement learning agent outputs fault injection parameter instructions for the continuous action space. The fault injection parameter instructions are subject to safety constraint verification. Determine whether the fault parameters in the fault injection parameter command exceed the range limited by the preset constraint rules; If the limit is exceeded, the fault injection parameter command will be adjusted to the range defined by the constraint rule; Based on the fault injection parameter instructions, a fault injection operation is performed on the distributed system, and the fault injection execution result is obtained; Calculate the reward value corresponding to the current fault injection parameter instruction based on the fault injection execution result; The policy network parameters of the reinforcement learning agent are updated based on the reward value.

2. The method for adjusting chaotic engineering parameters according to claim 1, characterized in that, The preset constraint rules include upper limits for the node impact ratio and lower limits for the continuous injection interval, which are set separately for different fault types.

3. The method for adjusting chaotic engineering parameters according to claim 2, characterized in that, Determining whether the fault parameters in the fault injection parameter instruction exceed the range defined by the preset constraint rules includes: Determine the fault type corresponding to the fault injection parameter command; Obtain the upper limit of the node impact ratio and the lower limit of the continuous injection interval corresponding to the fault type; Determine whether the node influence ratio in the fault injection parameter instruction is greater than the upper limit of the node influence ratio, and determine whether the time interval between two adjacent fault injections corresponding to the fault injection parameter instruction is less than the lower limit of the continuous injection interval.

4. The method for adjusting chaotic engineering parameters according to claim 1, characterized in that, During the fault injection operation, the operating metrics of the distributed system are monitored. When the operating metrics of the distributed system exceed a preset security threshold, the current fault injection operation is terminated, and a snapshot rollback operation is performed to restore the distributed system to its operating state before the fault injection operation.

5. The method for adjusting chaotic engineering parameters according to claim 1, characterized in that, The reward value corresponding to the current fault injection parameter instruction is calculated based on the fault injection execution result, including: Based on the results of the fault injection execution, determine the fault coverage rate and business impact metrics; Obtain the real-time load status of the distributed system; Based on the real-time load status, adjust the first weight corresponding to the fault coverage index and the second weight corresponding to the business impact index; wherein, the first weight is negatively correlated with the real-time load status, the second weight is positively correlated with the real-time load status, and the sum of the first weight and the second weight is a preset constant. The reward value is calculated based on the first weight, the second weight, the fault coverage index, and the business impact index.

6. The method for adjusting chaotic engineering parameters according to claim 5, characterized in that, The real-time load status includes the CPU utilization and memory usage of the distributed system; the fault coverage metric is calculated based on the ratio of the number of injected fault types to the total number of preset injectable fault types. The business impact metrics are calculated based on the request failure rate and response latency of the distributed system.

7. The method for adjusting chaotic engineering parameters according to claim 1, characterized in that, Updating the policy network parameters of the reinforcement learning agent based on the reward value includes: The real-time state vector, the fault injection parameter instruction, the reward value, and the real-time state vector at the next moment after the fault injection are corresponding to each fault injection operation and stored in the experience replay pool as a set of experience samples. A batch of experience samples is extracted from the experience replay pool, and the policy network parameters of the reinforcement learning agent are updated using the batch of experience samples.

8. A chaotic engineering parameter adjustment device, characterized in that, include: The status acquisition module is configured to acquire the real-time status vector of the distributed system. The instruction generation module is configured to input the real-time state vector into the reinforcement learning agent and generate fault injection parameter instructions for the continuous action space through the reinforcement learning agent. The safety constraint module is configured to perform safety constraint verification on the fault injection parameter instructions; Determine whether the fault parameters in the fault injection parameter command exceed the range limited by the preset constraint rules; If the limit is exceeded, the fault injection parameter command will be adjusted to the range defined by the constraint rule; The fault injection module is configured to perform a fault injection operation on the distributed system according to the fault injection parameter instructions, and obtain the fault injection execution result; The reward calculation module is configured to calculate the reward value corresponding to the current fault injection parameter instruction based on the fault injection execution result. The policy update module is configured to update the policy network parameters of the reinforcement learning agent based on the reward value.

9. The chaotic engineering parameter adjustment device according to claim 8, characterized in that, The reward calculation module includes: The indicator determination submodule is configured to determine the fault coverage indicator and the business impact indicator based on the fault injection execution result. The load acquisition submodule is configured to acquire the real-time load status of the distributed system. The weight adjustment submodule is configured to adjust the first weight corresponding to the fault coverage index and the second weight corresponding to the business impact index according to the real-time load status; wherein, the first weight is negatively correlated with the real-time load status, the second weight is positively correlated with the real-time load status, and the sum of the first weight and the second weight is a preset constant. The reward calculation submodule is configured to calculate the reward value based on the first weight, the second weight, the fault coverage index, and the business impact index.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the chaotic engineering parameter adjustment method according to any one of claims 1 to 7.

11. A computer program product comprising a computer-readable storage medium on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements the chaotic engineering parameter adjustment method according to any one of claims 1 to 7.

Images