Implementation method of chaos engineering application scenario experiment based on ant colony algorithm

Abstract

The present invention relates to the field of software engineering, and specifically to an experimental implementation method for chaos engineering application scenarios based on an ant colony algorithm; performing structured processing and time series modeling on the collected complex raw data; importing the processed information into the ant colony algorithm, determining parameters such as the number of ant colonies, pheromone concentration, evaporation rate, and initializing the pheromone matrix during initialization, and utilizing resource usage and relevant business information to assist in decision-making. During the execution of the algorithm, ants select paths based on a probability formula, determine the weights of pheromones and heuristic indicators based on the fault history and business information, and adjust the pheromone matrix according to the volatilization, addition, and update rules after traversal, and repeat until the conditions are met. The algorithm outputs an adjacency matrix containing pheromone, node, and edge information, which is used to analyze fault paths, impacts, propagation, and generate visual charts. This method can improve experimental efficiency, accuracy, and dynamic adaptability, and enhance system reliability and resilience.

Description

Technical Field

[0001] This invention belongs to the field of software engineering and focuses on the technical direction of implementing chaos engineering experiments. Specifically, it relates to an experimental implementation method for chaos engineering application scenarios based on an ant colony algorithm. This method is mainly applied to various software systems with high requirements for stability and reliability, especially distributed systems in cloud computing environments, application systems with microservice architectures, etc. It plays a key role in the development, testing, and operation and maintenance stages of software systems. It aims to improve the software system's ability to cope with failures in complex environments through innovative technical means, and enhance the system's reliability and resilience. Background Art

[0002] In software engineering, the widespread use of cloud computing technology has made resource deployment and management in software systems more flexible and efficient, significantly improving resource utilization and significantly enhancing system scalability. The rise of microservices architecture has further transformed the way software systems are built, breaking large systems into numerous small, independent services that can be independently developed, deployed, and upgraded, greatly enhancing system flexibility and maintainability. Distributed systems leverage networks to connect multiple nodes for coordinated operation, effectively improving system processing capabilities and reliability. For example, many internet companies have rapidly expanded their businesses and achieved rapid iteration using microservices architectures. However, this architectural shift has also brought numerous problems. With the rapid increase in the number of services and components, the interactions and dependencies between them have become extremely complex, making system behavior unpredictable. Once a failure occurs in a single link, the propagation path is complex and unpredictable, easily triggering a chain reaction that could potentially lead to the collapse of the entire system. For example, in the microservices architecture of a large e-commerce platform, a failure in the order service can quickly spread to multiple related services such as inventory management, payment, and logistics, severely impacting user experience and business operations.

[0003] Chaos engineering, an emerging practice, has been adopted by some leading software companies. It proactively introduces various faults into production environments, such as simulating network latency, service outages, and hardware failures, and observes system responses to assess system resilience and fault tolerance. Some companies use open-source chaos engineering tools, such as Netflix and ChaosMonkey, to conduct fault injection experiments during non-critical business hours to identify potential system weaknesses. However, traditional chaos engineering scenario experiments have significant shortcomings when applied to complex systems. Experimental scenario design is extremely challenging, as the high complexity of the system makes it difficult to fully consider all possible fault scenarios and combinations. The implementation process is cumbersome and requires extensive manual intervention to configure fault injection parameters, monitor the experiment, and analyze the results. For example, when conducting network fault injection experiments, technicians must manually set network latency parameters, determine the affected service scope, and deploy monitoring tools to collect data. This is not only time-consuming and labor-intensive, but also prone to inaccurate results due to human error. Furthermore, traditional methods struggle to dynamically adjust fault injection strategies based on real-time system status, making them ineffective in adapting to evolving system environments and business needs.

[0004] The ant colony algorithm (ACO) mimics the foraging behavior of ants in nature. During their foraging journey, ants release pheromones to mark their paths and communicate with other ants. Paths with higher pheromone concentrations are more likely to be chosen by other ants, ultimately allowing the ant colony to find the optimal path from their nest to the food source. This algorithm excels in solving complex combinatorial optimization problems, such as the traveling salesman problem and vehicle routing. Incorporating the ACO into chaos engineering offers unique advantages. Leveraging the accumulation and volatilization of pheromones, it efficiently searches for optimal fault injection paths within complex application topologies. Compared to other algorithms, the ACO is highly adaptable and robust. Even without full global system information, it can gradually approach the optimal solution through individual ants' local search and global pheromone updates. Furthermore, the ACO can be easily integrated with other chaos engineering operations, such as flexibly adjusting ant movement strategies based on application performance and resource usage, thereby better adapting to diverse testing environments and business needs.

[0005] Given the many problems that existing technologies have in dealing with the stability and reliability of complex software systems, especially the limitations of traditional chaos engineering experiments, and the significant advantages of ant colony algorithms in solving combinatorial optimization problems, it is particularly urgent to introduce ant colony algorithms into the field of chaos engineering to improve the implementation of chaos engineering application scenario experiments. Summary of the Invention

[0006] In light of the shortcomings of current related technologies, this paper provides a method for implementing chaos engineering application scenario experiments based on an ant colony algorithm. This method utilizes a variety of dynamic and static methods to comprehensively collect detailed information about managed applications. After processing, it is fed into the ant colony algorithm. During the algorithm's execution, fault injection parameters and strategies are dynamically adjusted to automatically generate the optimal fault injection path. This reduces manual intervention and configuration time, improves the coverage and accuracy of chaos engineering experiments, ensures efficient and precise implementation of chaos experiments, effectively adapts to diverse testing environments and business needs, and ultimately enhances the reliability and resilience of software systems, providing a strong guarantee for their stable operation in complex environments.

[0007] To achieve the above objectives, the present invention provides a chaos engineering application scenario experiment optimization method based on an ant colony algorithm, comprising the following steps:

[0008] S1: Information collection step: Application-related information is obtained and stored through various static and dynamic means. The application-related information includes application architecture information, application operation status information, exception handling mechanism information, and automated operation and maintenance capability information. Application architecture information covers module and service structure, data flow, and network topology. Application operation status information includes service health status, log information, and fault injection history. Exception handling mechanism information includes retry mechanism, circuit breaker strategy, and degradation strategy. Automated operation and maintenance capability information covers monitoring, alarm system, and self-recovery mechanism.

[0009] S2: Data preprocessing step, which performs structured processing and time series processing on the collected raw data. The structured processing includes using an adjacency matrix to represent the topological relationship between applications and adding descriptive information to nodes and edges. The time series processing includes grouping information into specific intervals and then aggregating, cleaning, smoothing, and performing minimum-maximum normalization on the data within the group.

[0010] S3: Algorithm initialization step: importing preprocessed information into the ant colony algorithm and setting initial parameters, including determining one or more parameters such as ant colony size, pheromone concentration, and evaporation rate. The pheromone matrix is ​​initialized so that the pheromone values ​​on each edge are equal. Resource usage is used to provide weight information for path decision making, and critical paths are evaluated using high-frequency services, service monitoring information, and fault history.

[0011] S4: Algorithm execution step, starting the algorithm to perform path traversal and pheromone matrix update. During path traversal, the ant selects the next node according to a specific probability formula at the current node. After completing the path traversal, the pheromone matrix is ​​updated according to the three steps of pheromone volatilization, addition, and update. This is repeated until the maximum number of iterations or convergence conditions are met;

[0012] S5: Algorithm output and analysis step, outputs the adjacency matrix containing pheromones, node information and edge information, and analyzes it to obtain fault path selection, fault impact analysis, fault propagation analysis and visualization chart results.

[0013] Furthermore, in step S1, information collection is achieved specifically by the following methods:

[0014] S11: Use code decompilation to obtain information related to the application's code structure, including module and service structure and referenced third-party dependencies;

[0015] S12: Collect abnormal behaviors, error patterns, and business logic-related information recorded in the logs through log auditing, including the response frequency of business logic and the behavior patterns of user requests;

[0016] S13: Using stack analysis to obtain stack information during system runtime to assist in analyzing application architecture and running status;

[0017] S14: Use behavioral analysis to monitor application behavior patterns, including calling behaviors between services.

[0018] Furthermore, in step S2:

[0019] In the structured processing step, an adjacency matrix is ​​used to represent the topological relationship between applications, and descriptive information including service name, resource usage, call frequency, retry mechanism, degradation strategy, automated operation and maintenance tools, and fault injection history is attached to nodes and edges. Resource usage includes CPU usage, memory usage, network bandwidth usage, request response time, and number of concurrent requests. High-frequency services include interface information and corresponding retry and degradation mechanisms. Automated operation and maintenance tools include service monitoring tools. Edge description information includes call relationships, communication protocols, and data transmission volume.

[0020] In the time series processing step, the original information is grouped according to the set time interval, and aggregation, cleaning, smoothing and minimum-maximum normalization are performed. The aggregation operation includes finding the average, maximum and minimum values. Data cleaning adopts the method of filling missing values ​​and correcting outliers. The smoothing adopts the moving average method or exponential smoothing method. The minimum-maximum normalization scales the data to the range of [0, 1].

[0021] Furthermore, in step S3:

[0022] The number of ant colonies is set to a multiple of the number n of application topology nodes;

[0023] The initial value of the pheromone concentration is;

[0024] The evaporation rate is set to a decimal between 0.1 and 0.5. The pheromone matrix is ​​set based on the application service adjacency matrix, and the pheromone value τ of the node is set by traversing. ij =τ0.

[0025] Furthermore, in step S4, each ant gradually builds a path. When the current node i probabilistically selects the next node j, the probability P ij The basic calculation formula is:

[0026] in

[0027] α is the pheromone weight, β is the heuristic indicator weight, τ ij is the pheromone concentration on edge (i,j), η ij is a heuristic indicator, and allowed is the set of next nodes that can be selected.

[0028] α is the pheromone weight. It determines the ant's reliance on the successful experience of existing barrier injection paths based on the barrier injection history collected in the information collection step on the service node and the barrier injection results during the traversal. The larger the value, the more dependent the algorithm is on the existing pheromone concentration, making the ants more likely to choose paths with significant barrier injection history.

[0029] β is the heuristic indicator weight. Based on the high-frequency business and service monitoring information collected in the information collection step on the service node, it guides the ants to prioritize the exploration of high-impact paths. In scenarios with many critical paths in the system, increasing the value can make the algorithm more inclined to select these paths to cover the critical paths.

[0030] τ ij is the pheromone concentration on the (i, j) edge;

[0031] η ij It is a heuristic indicator based on the information collected in the information collection step on the service node, such as resource usage, business call frequency, service dependencies, degradation and retry mechanism, and the sensitivity of the service node to the atomicity of failure. The specific calculation method is to comprehensively evaluate the importance and risk level of nodes and edges based on the above information, and express it in numerical form to guide ants to choose paths.

[0032] Furthermore, in step S4, the pheromone matrix update includes the following steps:

[0033] The pheromone volatilization formula is τ ij (new)=(1-ρ)τ ij (old), where ρ is the volatility of pheromone, ranging from 0.1 to 0.5, to prevent the infinite accumulation of pheromone and maintain the diversity of path exploration;

[0034] The formula for increasing pheromones is: Where Q is the total amount of pheromone, L k is the total length or cost of the path taken by the kth ant. The calculation requires accurate statistics of the ant path lengths or costs and reasonable distribution based on the total pheromone amount to strengthen high-quality paths and make effective paths more likely to be selected by subsequent ants.

[0035] The pheromone update formula is τ ij (new) = τ ij (old)+Δτ ij , ensuring that the pheromone matrix is ​​updated timely and accurately to reflect changes in the quality of the paths.

[0036] Furthermore, in step S5:

[0037] The fault path selection is performed by traversing the pheromone matrix, sorting the pheromone concentration from high to low, selecting the path with higher concentration as the fault path that may affect system stability, and analyzing the dependency relationship between services on the path to determine the potential fault point;

[0038] The fault impact analysis simulates fault injection into services on the fault path, uses testing tools to accurately measure service latency, availability degradation, and fault recovery time, and compares changes before and after the fault injection to assess the fault impact.

[0039] The fault propagation analysis sets monitoring points in the system to track the propagation of faults between services, records the fault propagation path and the affected services, and analyzes the blocking or mitigation effects of existing retry and degradation mechanisms on fault propagation when a fault occurs.

[0040] The visualization chart is generated by data collection and graphics drawing tools, showing one or more of the CPU usage, memory usage changes, response time changes, and request frequency changes of the service node under the influence of the failure, providing developers with intuitive data display and assisting in system optimization and improvement.

[0041] Furthermore, the method also includes an environment migration step. When migrating the experimental implementation plan from the development or test environment to the production environment, configuration file mapping or script conversion is used to address the encryption obfuscation and log level adjustment that may exist in the production environment, thereby ensuring the smooth transfer and effective implementation of the experiment between different environments.

[0042] Specifically, it includes: identifying the differences between configuration files in the development or test environment and the production environment, establishing a mapping relationship, and accurately converting the configuration parameters in the development or test environment into the parameters required for the production environment; for encrypted and obfuscated code, parsing the encryption algorithm or using an adapted decryption tool to enable the experimental tool to correctly identify and process it; adjusting the log level according to the requirements of the production environment, while ensuring that sufficient log information can be obtained for analysis and evaluation during the experiment, and that the system performance will not be affected by excessive log volume.

[0043] The present invention adopts the above technical solution, which has at least the following beneficial effects:

[0044] 1. Improve experimental efficiency: Comprehensive information collection and targeted processing: Through various analytical methods (such as code decompilation, log auditing, stack analysis, and behavioral analysis), comprehensive application-related information is collected, covering application architecture, operating status, exception handling mechanisms, automated operation and maintenance capabilities, and other aspects, making information collection more targeted. After classification and organization, this information provides a rich and organized data foundation for subsequent experiments, reducing the time spent on information screening and organization.

[0045] Optimize data preprocessing: Perform structured processing (such as using adjacency matrices to represent topological relationships) and time series modeling (including grouping by specific intervals, aggregation, cleaning, smoothing, and normalization) on the collected complex raw data, and convert unstructured data into a format suitable for algorithm processing, effectively reducing data processing time, improving data quality, and enabling algorithms to process data and make decisions faster.

[0046] Intelligent Algorithm Decision-Making and Execution: Based on the ant colony algorithm, fault injection strategies are automatically generated, eliminating the need for extensive manual intervention and tedious configuration. The algorithm automatically traverses paths and updates the pheromone matrix based on pre-set rules and collected information. This significantly shortens experiment preparation and implementation time, improves the overall efficiency of chaos engineering experiments, reduces the need for manual parameter tuning, and reduces the difficulty of implementing chaos engineering in complex systems.

[0047] 2. Enhanced Experimental Accuracy: Accurate Information Acquisition and Analysis: The multi-category detailed information obtained during the information collection step (covering application architecture, operating status, exception handling mechanisms, automated operation and maintenance capabilities, etc.) provides a comprehensive basis for accurate system analysis. For example, code structure information obtained through code decompilation helps to deeply understand the internal logic of the system, and log auditing provides abnormal behavior and business logic information to help identify potential problems.

[0048] Effective data preprocessing and feature mining: Structuring and time series processing in data preprocessing can better uncover data features and patterns, providing more accurate input for the algorithm. For example, aggregation operations can reveal the central tendency and dispersion of data, cleaning and smoothing can remove noise and outliers, and normalization can make different types of data comparable, thereby improving the algorithm's accuracy in determining system status.

[0049] Intelligent Algorithm Path Selection and Optimization: During algorithm execution, paths are selected based on probability calculation formulas and combined with heuristic indicators (calculated based on node and edge resource usage and historical fault data) to more accurately select fault injection points and parameters. The Ant Colony Algorithm (ACA) guides ants to select optimal paths based on pheromone concentration and heuristic indicators, thereby more realistically simulating potential system failures. This improves the accuracy of assessments of system resilience and fault tolerance, and can discover previously unconsidered fault injection paths.

[0050] 3. Dynamic Adaptability: Real-Time Information Feedback and Algorithm Adjustment: During the algorithm execution step, based on real-time system monitoring data and experimental feedback, the pheromone matrix is ​​updated (including evaporation, addition, and update operations) to dynamically adjust fault injection parameters and strategies. For example, if a path is found to perform poorly in an experiment, pheromone evaporation will reduce the probability of selecting that path, while increasing pheromone on a successful path will make it more favored, thereby guiding the algorithm to explore a more optimal path.

[0051] Flexible response to system changes and business needs: This paper introduces the ant colony algorithm into chaos engineering, leveraging its unique characteristics to balance multiple indicators to achieve a more optimal chaos engineering implementation strategy. This algorithm dynamically adjusts search strategies based on system feedback and automatically generates application scenario experiments. Whether influenced by changes in system architecture, fluctuations in business traffic, or other external factors, this method can respond promptly, ensuring that experiments always align with the actual system operating environment and business needs. This effectively improves the effectiveness and practicality of experiments and enables chaos engineering experiments to better adapt to complex and changing real-world situations.

[0052] 4. Improving System Reliability and Resilience: Comprehensive Risk Assessment and Discovery: Leveraging the aforementioned advantages of improved experimental efficiency, accuracy, and dynamic adaptability, this invention enables a more comprehensive and in-depth identification of potential risks and weaknesses in the system. By simulating various complex failure scenarios and conducting effective assessments during experiments, development teams can optimize and improve the system in a targeted manner, taking proactive measures to enhance the system's fault tolerance and resilience.

[0053] Enhance system stability and business assurance: This improves the reliability and resilience of software systems in actual operation, reduces the probability of system failures, ensures the continuous and stable operation of businesses, and mitigates the economic losses and business risks caused by system failures. For example, in financial systems, chaos engineering experiments can identify and fix potential failure points, effectively avoiding serious problems such as transaction interruptions and data loss, and ensuring the normal operation of financial services.

[0054] 5. Promote the application and promotion of chaos engineering in complex systems: Reduce implementation difficulty and barriers to entry: Traditional chaos engineering experiments face numerous challenges in complex systems, such as high implementation difficulty and low efficiency, which limit their widespread application. This invention optimizes experimental implementation methods, reduces implementation difficulty, improves efficiency and effectiveness, and makes the application of chaos engineering in complex software systems more feasible and easier to operate.

[0055] Promoting technology adoption and industry development: This will help more companies and developers adopt chaos engineering techniques to ensure software system quality, promote the popularization and development of chaos engineering techniques in the software engineering field, and drive technological progress in system reliability and resilience across the industry. For example, small and medium-sized enterprises can use this invention to more easily incorporate chaos engineering into their software development and operations processes, improving overall software quality and competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 This is a flow chart of the chaos engineering application scenario experimental optimization method based on the ant colony algorithm of the present invention;

[0058] Figure 2 It is a schematic diagram of the classification and detailed description of application-related information obtained in the information collection step of the present invention;

[0059] Figure 3 This is a schematic diagram of node description information for the xxx-biz service of the present invention;

[0060] Figure 4 This is a schematic diagram of the edge description information of the xxx-biz service and the xxx-upms service of the present invention. DETAILED DESCRIPTION

[0061] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0062] like Figure 1 As shown, this embodiment provides a chaos engineering application scenario experiment optimization method based on an ant colony algorithm, comprising the following steps:

[0063] S1: Information collection step, which uses a variety of static and dynamic methods to obtain and store application-related information;

[0064] Figure 2 This is a diagram of the classification and detailed description of the application-related information obtained in the information collection step, which intuitively shows the specific content and mutual relationship of various types of information such as application architecture information, application operation status information, exception handling mechanism information and automated operation and maintenance capability information. Figure 2 The application-related information includes application architecture information, application running status information, exception handling mechanism information, and automated operation and maintenance capability information. The application architecture information includes module and service structure (collecting the modular design of the application, the dependencies between services, and the referenced third-party dependencies), data flow (identifying the path of data flow and understanding the transmission path of data from input to output), network topology (network communication path between system components), and the application running status includes service health status (collecting health indicators of each service (CPU, memory, network bandwidth, etc.)), log information (collecting logs to obtain records of abnormal behavior and possible Error patterns, response frequency of business logic, behavioral patterns of user requests, etc.), fault injection history (whether chaos experiments have been performed before collection, and their impact); the exception handling mechanism includes the retry mechanism (when the system fails, whether there is a retry mechanism, the frequency and conditions of the retry), the circuit breaker strategy (when the service is unavailable, whether there is a circuit breaker mechanism to protect the system), and the degradation strategy (whether some functions can be degraded when the service fails); the automated operation and maintenance capabilities include monitoring and alarm systems (analyzing whether there are monitoring tools to capture abnormal behavior) and self-recovery capabilities (when the system encounters problems, whether there is a self-recovery mechanism).

[0065] S2: Data preprocessing step, which performs structured processing and time series processing on the collected raw data. The structured processing includes using an adjacency matrix to represent the topological relationship between applications and adding descriptive information to nodes and edges. The time series processing includes grouping information into specific intervals and then aggregating, cleaning, smoothing, and performing minimum-maximum normalization on the data within the group.

[0066] S3: Algorithm initialization step: importing preprocessed information into the ant colony algorithm and setting initial parameters, including determining one or more parameters such as ant colony size, pheromone concentration, and evaporation rate. The pheromone matrix is ​​initialized so that the pheromone values ​​on each edge are equal. Resource usage is used to provide weight information for path decision making, and critical paths are evaluated using high-frequency services, service monitoring information, and fault history.

[0067] S4: Algorithm execution step, starting the algorithm to perform path traversal and pheromone matrix update. During path traversal, the ant selects the next node according to a specific probability formula at the current node. After completing the path traversal, the pheromone matrix is ​​updated according to the three steps of pheromone volatilization, addition, and update. This is repeated until the maximum number of iterations or convergence conditions are met;

[0068] S5: Algorithm output and analysis step, outputs the adjacency matrix containing pheromones, node information and edge information, and analyzes it to obtain fault path selection, fault impact analysis, fault propagation analysis and visualization chart results.

[0069] As an implementation method, in step S1 of this embodiment, information collection is specifically achieved in the following manner:

[0070] S11: Use code decompilation to obtain information related to the application's code structure, including module and service structure and referenced third-party dependencies;

[0071] S12: Collect abnormal behaviors, error patterns, and business logic-related information recorded in the logs through log auditing, including the response frequency of business logic and the behavior patterns of user requests;

[0072] S13: Using stack analysis to obtain stack information during system runtime to assist in analyzing application architecture and running status;

[0073] S14: Use behavioral analysis to monitor application behavior patterns, including calling behaviors between services.

[0074] As an implementation manner, in step S2 of this embodiment:

[0075] In the structured processing step, an adjacency matrix is ​​used to represent the topological relationship between applications, and descriptive information including service name, resource usage, call frequency, retry mechanism, degradation strategy, automated operation and maintenance tools, and fault injection history is attached to nodes and edges. Resource usage includes CPU usage, memory usage, network bandwidth usage, request response time, and number of concurrent requests. High-frequency services include interface information and corresponding retry and degradation mechanisms. Automated operation and maintenance tools include service monitoring tools. Edge description information includes call relationships, communication protocols, and data transmission volume.

[0076] Use the adjacency matrix to represent the topological relationship between applications. Each service and module is considered a node in the graph. Descriptive information is attached to each node and edge. Suppose there is the following service scenario:

[0077] xxx-auth: authorization service

[0078] xxx-upms: general user rights management service, which will call the xxx-auth service

[0079] xxx-biz: application business service, which will call xxx-auth and xxx-upms services

[0080] xxx-gateway: Application gateway service, which will call xxx-auth, xxx-upms, and xxx-biz services

[0081] It can be represented by the adjacency matrix shown in Table 1:

[0082] Table 1

[0083] xxx-gateway xxx-auth xxx-upms xxx-biz xxx-gateway 0 1 1 1 xxx-auth 0 0 0 0 xxx-upms 0 1 0 0 xxx-biz 0 1 1 0

[0084] The node description information of xxx-biz service can be expressed as follows Figure 3 As shown:

[0085] The side description information of xxx-biz service and xxx-upms service is as follows Figure 4 As shown:

[0086] In the time series processing step, the original information is grouped according to the set time interval, and aggregation, cleaning, smoothing and minimum-maximum normalization are performed. The aggregation operation includes finding the average, maximum and minimum values. Data cleaning adopts the method of filling missing values ​​and correcting outliers. The smoothing adopts the moving average method or exponential smoothing method. The minimum-maximum normalization scales the data to the range of [0, 1].

[0087] After determining the time interval (e.g., 5 seconds), the collected raw information (application running status, exception handling mechanism execution, monitoring alarm status, etc.) is grouped by time interval. The data within each time period group is then aggregated (average, maximum, and minimum values ​​are calculated). Data cleaning is performed to fill in missing data within the time period and correct outliers. The aggregated data is smoothed to reduce noise. Finally, minimum-maximum normalization is used to scale the data to [0, 1] for easy use by the ant colony algorithm.

[0088] As an implementation manner, in this embodiment, in step S3:

[0089] The number of ant colonies is set to a multiple of the number n of application topology nodes;

[0090] Generally, it is 2 to 5 times the number of application topology nodes n;

[0091] The initial value of the pheromone concentration is;

[0092] The evaporation rate is set to a decimal between 0.1 and 0.5. The pheromone matrix is ​​set based on the application service adjacency matrix, and the pheromone value τ of the node is set by traversing. ij =τ0.

[0093] Furthermore, in step S4, each ant gradually builds a path. When the current node i probabilistically selects the next node j, the probability P ij The basic calculation formula is:

[0094] in

[0095] α is the pheromone weight, β is the heuristic indicator weight, τ ij is the pheromone concentration on edge (i,j), η ij is a heuristic indicator, and allowed is the set of next nodes that can be selected.

[0096] α is the pheromone weight. It determines the ant's reliance on the successful experience of existing barrier injection paths based on the barrier injection history collected in the information collection step on the service node and the barrier injection results during the traversal. The larger the value, the more dependent the algorithm is on the existing pheromone concentration, making the ants more likely to choose paths with significant barrier injection history.

[0097] β is the heuristic indicator weight. Based on the high-frequency business and service monitoring information collected in the information collection step on the service node, it guides the ants to prioritize the exploration of high-impact paths. In scenarios with many critical paths in the system, increasing the value can make the algorithm more inclined to select these paths to cover the critical paths.

[0098] τ ij is the pheromone concentration on the (i, j) edge;

[0099] η ij It is a heuristic indicator based on the information collected in the information collection step on the service node, such as resource usage, business call frequency, service dependencies, degradation and retry mechanism, and the sensitivity of the service node to the atomicity of failure. The specific calculation method is to comprehensively evaluate the importance and risk level of nodes and edges based on the above information, and express it in numerical form to guide ants to choose paths.

[0100] As an implementation manner, in this embodiment, in step S4, updating the pheromone matrix includes the following steps:

[0101] The pheromone volatilization formula is τ ij (new)=(1-ρ)τ ij (old), where ρ is the volatility of pheromone, ranging from 0.1 to 0.5, to prevent the infinite accumulation of pheromone and maintain the diversity of path exploration;

[0102] The formula for increasing pheromones is: Where Q is the total amount of pheromone, L k is the total length or cost of the path taken by the kth ant. The calculation requires accurate statistics of the ant path lengths or costs and reasonable distribution based on the total pheromone amount to strengthen high-quality paths and make effective paths more likely to be selected by subsequent ants.

[0103] The pheromone update formula is τ ij (new) = τ ij (old)+Δτ ij , ensuring that the pheromone matrix is ​​updated timely and accurately to reflect changes in the quality of the paths.

[0104] As an implementation manner, in this embodiment, in step S5:

[0105] The fault path selection is performed by traversing the pheromone matrix, sorting the pheromone concentration from high to low, selecting the path with higher concentration as the fault path that may affect system stability, and analyzing the dependency relationship between services on the path to determine the potential fault point;

[0106] The fault impact analysis simulates fault injection into services on the fault path, uses testing tools to accurately measure service latency, availability degradation, and fault recovery time, and compares changes before and after the fault injection to assess the fault impact.

[0107] The fault propagation analysis sets monitoring points in the system to track the propagation of faults between services, records the fault propagation path and the affected services, and analyzes the blocking or mitigation effects of existing retry and degradation mechanisms on fault propagation when a fault occurs.

[0108] The visualization chart is generated by data collection and graphics drawing tools, showing one or more of the CPU usage, memory usage changes, response time changes, and request frequency changes of the service node under the influence of the failure, providing developers with intuitive data display and assisting in system optimization and improvement.

[0109] As an implementation method, the method described in this embodiment also includes an environment migration step. When migrating the experimental implementation plan from the development or test environment to the production environment, configuration file mapping or script conversion is used to handle the encryption obfuscation and log level adjustment that may exist in the production environment, thereby ensuring the smooth transfer and effective implementation of the experiment between different environments.

[0110] Specifically, it includes: identifying the differences between configuration files in the development or test environment and the production environment, establishing a mapping relationship, and accurately converting the configuration parameters in the development or test environment into the parameters required for the production environment; for encrypted and obfuscated code, parsing the encryption algorithm or using an adapted decryption tool to enable the experimental tool to correctly identify and process it; adjusting the log level according to the requirements of the production environment, while ensuring that sufficient log information can be obtained for analysis and evaluation during the experiment, and that the system performance will not be affected by excessive log volume.

[0111] This example designs a method for implementing chaos engineering application scenario experiments based on an ant colony algorithm. This method uses dynamic and static methods to collect information about managed applications. This information is then processed and fed into a heuristic algorithm, such as the ant colony algorithm. The algorithm then dynamically adjusts fault injection parameters and strategies, ultimately generating a fault injection path. This reduces manual intervention and configuration time, improves the coverage of chaos engineering experiments, and enables more efficient and accurate implementation of chaos experiments, adapting to diverse testing environments and business needs.

[0112] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A chaos engineering application scenario experiment optimization method based on ant colony algorithm, characterized by: The following steps are involved: S1: Information collection step: Application-related information is obtained and stored through various static and dynamic means. The application-related information includes application architecture information, application operation status information, exception handling mechanism information, and automated operation and maintenance capability information. Application architecture information covers module and service structure, data flow, and network topology. Application operation status information includes service health status, log information, and fault injection history. Exception handling mechanism information includes retry mechanism, circuit breaker strategy, and degradation strategy. Automated operation and maintenance capability information covers monitoring, alarm system, and self-recovery mechanism. S2: Data preprocessing step, which performs structured processing and time series processing on the collected raw data. The structured processing includes using an adjacency matrix to represent the topological relationship between applications and adding descriptive information to nodes and edges. The time series processing includes grouping the information according to set time intervals, and then aggregating, cleaning, smoothing, and min-max normalizing the data within the group; S3: Algorithm initialization step: importing preprocessed information into the ant colony algorithm and setting initial parameters, including determining one or more parameters such as ant colony size, pheromone concentration, and evaporation rate. The pheromone matrix is ​​initialized so that the pheromone values ​​on each edge are equal. Resource usage is used to provide weight information for path decision making, and critical paths are evaluated using high-frequency services, service monitoring information, and fault history. S4: Algorithm execution step, starting the algorithm to perform path traversal and pheromone matrix update. During path traversal, the ant selects the next node according to a specific probability formula at the current node. After completing the path traversal, the pheromone matrix is ​​updated according to the three steps of pheromone volatilization, addition, and update. This is repeated until the maximum number of iterations or convergence conditions are met; In step S4, each ant gradually builds a path. When the current node i probabilistically selects the next node j, the probability P ij The basic calculation formula is: in α is the pheromone weight, β is the heuristic indicator weight, τ ij is the pheromone concentration on edge (i,j), η ij is a heuristic indicator, and allowed is the set of next nodes that can be selected; S5: Algorithm output and analysis step, outputs the adjacency matrix containing pheromones, node information and edge information, and analyzes it to obtain fault path selection, fault impact analysis, fault propagation analysis and visualization chart results.

2. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to claim 1 is characterized by: In step S1, information collection is achieved specifically through the following methods: S11: Use code decompilation to obtain information related to the application's code structure, including module and service structure and referenced third-party dependencies; S12: Collect abnormal behaviors, error patterns, and business logic-related information recorded in the logs through log auditing, including the response frequency of business logic and the behavior patterns of user requests; S13: Using stack analysis to obtain stack information during system runtime to assist in analyzing application architecture and running status; S14: Use behavioral analysis to monitor application behavior patterns, including calling behaviors between services.

3. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to claim 1 is characterized by: In step S2: In the structured processing step, an adjacency matrix is ​​used to represent the topological relationship between applications, and descriptive information including service name, resource usage, call frequency, retry mechanism, degradation strategy, automated operation and maintenance tools, and fault injection history is attached to nodes and edges. Resource usage includes CPU usage, memory usage, network bandwidth usage, request response time, and number of concurrent requests. High-frequency services include interface information and corresponding retry and degradation mechanisms. Automated operation and maintenance tools include service monitoring tools. Edge description information includes call relationships, communication protocols, and data transmission volume. In the time series processing step, the original information is grouped according to the set time interval, and aggregation, cleaning, smoothing and minimum-maximum normalization are performed. The aggregation operation includes finding the average, maximum and minimum values. Data cleaning adopts the method of filling missing values ​​and correcting outliers. The smoothing adopts the moving average method or exponential smoothing method. The minimum-maximum normalization scales the data to the range of [0, 1].

4. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to claim 1 is characterized by: In step S3: The number of ant colonies is set to a multiple of the number n of application topology nodes; The initial value of the pheromone concentration is: The evaporation rate is set to a decimal between 0.1 and 0.

5. The pheromone matrix is ​​set based on the application service adjacency matrix, and the pheromone value τ of the node is set by traversing. ij =τ0.

5. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to claim 1 is characterized by: In step S4, the pheromone matrix update includes the following steps: The pheromone volatilization formula is τ ij (new)=(1-ρ)τ ij (old), where ρ is the volatility of pheromone, ranging from 0.1 to 0.5, to prevent the infinite accumulation of pheromone and maintain the diversity of path exploration; The formula for increasing pheromones is: Where Q is the total amount of pheromone, L k is the total length or cost of the path taken by the kth ant. The calculation requires accurate statistics of the ant path lengths or costs and reasonable distribution based on the total pheromone amount to strengthen high-quality paths and make effective paths more likely to be selected by subsequent ants. The pheromone update formula is τ ij (new) = τ ij (old)+Δτ ij , ensuring that the pheromone matrix is ​​updated timely and accurately to reflect changes in the quality of the paths.

6. The method for optimizing chaos engineering application scenarios based on ant colony algorithm according to claim 1, characterized in that: In step S5: The fault path selection is performed by traversing the pheromone matrix, sorting the pheromone concentration from high to low, selecting the path with higher concentration as the fault path that may affect system stability, and analyzing the dependency relationship between services on the path to determine the potential fault point; The fault impact analysis simulates fault injection into services on the fault path, uses testing tools to accurately measure service latency, availability degradation, and fault recovery time, and compares changes before and after the fault injection to assess the fault impact. The fault propagation analysis sets monitoring points in the system to track the propagation of faults between services, records the fault propagation path and the affected services, and analyzes the blocking or mitigation effects of existing retry and degradation mechanisms on fault propagation when a fault occurs. The visualization chart is generated by data collection and graphics drawing tools, showing one or more of the CPU usage, memory usage changes, response time changes, and request frequency changes of the service node under the influence of the failure, providing developers with intuitive data display and assisting in system optimization and improvement.

7. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to any one of claims 1 to 6, characterized in that: The method also includes an environment migration step. When migrating the experimental implementation plan from the development or test environment to the production environment, configuration file mapping or script conversion is used to address the encryption obfuscation and log level adjustment that may exist in the production environment, thereby ensuring the smooth transfer and effective implementation of the experiment between different environments. Specifically, it includes: identifying the differences between configuration files in the development or test environment and the production environment, establishing a mapping relationship, and accurately converting the configuration parameters in the development or test environment into the parameters required for the production environment; for encrypted and obfuscated code, parsing the encryption algorithm or using an adapted decryption tool to enable the experimental tool to correctly identify and process it; adjusting the log level according to the requirements of the production environment, while ensuring that sufficient log information can be obtained for analysis and evaluation during the experiment, and that the system performance will not be affected by excessive log volume.

Images