Chaotic engineering application scene experiment implementation method based on ant colony algorithm

Abstract

The invention relates to the field of software engineering, in particular to a chaotic engineering application scene experiment implementation method based on an ant colony algorithm. Carrying out structured processing and time sequence modeling on the collected complex original data; and importing the processed information into an ant colony algorithm, determining parameters such as the number of ant colonies, pheromone concentration and evaporation rate during initialization, initializing a pheromone matrix, and assisting decision making by using resource use conditions and related service information. In algorithm execution, ants select paths according to a probability formula, pheromone and heuristic index weights are determined according to fault injection history and service information, a pheromone matrix is adjusted according to volatilization, addition and update rules after traversal, and the steps are repeated until conditions are met. The algorithm outputs an adjacent matrix containing information of pheromones, nodes and sides, and is used for analyzing a fault path, influence, propagation and generation of a visual chart. The method can improve the experiment efficiency, accuracy and dynamic adaptability, and improve the reliability and toughness of the system.

Description

Technical Field

[0001] The present invention belongs to the field of software engineering and focuses on the technical direction of chaos engineering experiment implementation, and specifically relates to a chaos engineering application scenario experiment implementation method based on ant colony algorithm; the method is mainly used in various software systems with high requirements on stability and reliability, especially distributed systems in cloud computing environments, application systems with microservice architectures, etc., and plays a key role in the development, testing and operation and maintenance stages of software systems. It aims to improve the software system's ability to cope with failures in complex environments through innovative technical means, and enhance the reliability and resilience of the system. Background Art

[0002] In the field of software engineering, the widespread application of cloud computing technology has made the resource deployment and management of software systems more flexible and efficient, greatly improved resource utilization, and significantly enhanced system scalability. The rise of microservice architecture has further changed the construction mode of software systems, splitting large systems into many small, independent services, each of which can be developed, deployed and upgraded separately, greatly enhancing the flexibility and maintainability of the system. Distributed systems use the network to connect multiple nodes for collaborative operation, effectively improving the processing power and reliability of the system. For example, many Internet companies use microservice architecture to rapidly expand their businesses and achieve rapid iteration. However, this architectural transformation has also brought many problems. With the rapid increase in the number of services and components, the interactions and dependencies between them have become extremely complex, and the system behavior is difficult to predict. Once a link fails, the fault propagation path is complex and changeable, which can easily trigger a chain reaction and may cause the entire system to crash. Taking a large e-commerce platform as an example, in its microservice architecture, the failure of the order service may quickly spread to multiple related services such as inventory management, payment, and logistics, seriously affecting user experience and business operations.

[0003] Chaos engineering, as an emerging practice, has been applied in some advanced software companies. It evaluates the resilience and fault tolerance of the system by actively introducing various types of faults in the production environment, such as simulating network delays, service interruptions, hardware failures, etc., observing the system response, etc. Some companies use open source chaos engineering tools, such as Netflix and ChaosMonkey, to conduct fault injection experiments during non-critical business hours to explore potential weaknesses in the system. However, traditional chaos engineering application scenario experiments have exposed obvious shortcomings when facing complex systems. The design of experimental scenarios is extremely difficult. Due to the high complexity of the system, it is difficult to fully consider various possible failure situations and combinations. The implementation process is cumbersome and requires a lot of manual intervention to configure fault injection parameters, monitor the experimental process, and analyze the results. For example, when conducting network fault injection experiments, technicians need to manually set network delay parameters, determine the scope of affected services, deploy monitoring tools to collect data, etc., which is not only time-consuming and labor-intensive, but also easy to cause inaccurate experimental results due to human negligence. Moreover, it is difficult for traditional methods to dynamically adjust fault injection strategies based on the real-time status of the system, and cannot effectively adapt to the changing system environment and business needs.

[0004] The ant colony algorithm simulates the foraging behavior of ants in nature. During the foraging process, ants release pheromones to mark the path and communicate with other ants. The higher the pheromone concentration, the greater the probability of being selected by other ants. Eventually, the ant colony can find the optimal path from the nest to the food source. This algorithm performs well in solving complex combinatorial optimization problems, such as the traveling salesman problem and vehicle path planning. The introduction of the ant colony algorithm in chaos engineering has unique advantages. It can efficiently search for a better fault injection path in a complex application topology by relying on the accumulation and volatilization mechanism of pheromones. Compared with other algorithms, the ant colony algorithm has strong adaptability and robustness. Even if the global information of the system is not fully mastered, it can gradually approach the optimal solution through the local search of individual ants and the global update of pheromones. At the same time, the ant colony algorithm is easy to combine with other operations in chaos engineering, such as flexibly adjusting the ant movement strategy according to the application running status and resource usage, so as to better adapt to different test environments and business needs.

[0005] In view of the many problems existing in existing technologies in dealing with the stability and reliability of complex software systems, especially the limitations of traditional chaos engineering experiments, and the significant advantages of ant colony algorithms in solving combinatorial optimization problems, it is particularly urgent to introduce ant colony algorithms into the field of chaos engineering to improve the implementation of chaos engineering application scenario experiments. Summary of the invention

[0006] In view of the shortcomings of the current related existing technologies, the present invention provides a method for implementing chaos engineering application scenario experiments based on ant colony algorithm, which uses a variety of dynamic and static means to comprehensively collect detailed information of managed applications, inputs the processed information into the ant colony algorithm, dynamically adjusts the parameters and strategies of fault injection with the help of the algorithm execution process, and automatically generates the optimal fault injection path. In this way, manual intervention is reduced, configuration time is reduced, the coverage and accuracy of chaos engineering experiments are improved, and chaos experiments can be implemented efficiently and accurately, effectively adapting to different test environments and business needs, thereby improving the reliability and resilience of software systems, and providing strong guarantees for the stable operation of software systems in complex environments.

[0007] To achieve the above object, the present invention provides a chaos engineering application scenario experiment optimization method based on ant colony algorithm, comprising the following steps:

[0008] S1: Information collection step, obtaining and storing application-related information through a variety of static and dynamic means, wherein the application-related information includes application architecture information, application running status information, exception handling mechanism information, and automated operation and maintenance capability information, wherein the application architecture information covers modules and service structures, data flows, and network topologies, the application running status information includes service health status, log information, and fault injection history, the exception handling mechanism information includes retry mechanisms, fuse strategies, and downgrade strategies, and the automated operation and maintenance capability information involves monitoring, alarm systems, and self-recovery mechanisms;

[0009] S2: Data preprocessing step, performing structured processing and time series processing on the collected raw data, wherein the structured processing includes using an adjacency matrix to represent the topological relationship between applications and adding description information to nodes and edges, and the time series processing includes grouping the information at specific intervals, and then aggregating, cleaning, smoothing, and minimum-maximum normalization of the data within the group;

[0010] S3: Algorithm initialization step, importing the preprocessed information into the ant colony algorithm and setting the initial parameters, including determining one or more parameters of the ant colony number, pheromone concentration and evaporation rate, and initializing the pheromone matrix to make the pheromone value on each edge equal. At the same time, the resource usage is used to provide weight information for path decision-making, and the key path is evaluated with the help of high-frequency business, service monitoring information and fault history.

[0011] S4: Algorithm execution step, start the algorithm to perform path traversal and pheromone matrix update. During the path traversal, the ant selects the next node at the current node according to a specific probability formula. After completing the path traversal, the pheromone matrix is ​​updated according to the three steps of pheromone volatilization, addition and update. Repeat the execution until the maximum number of iterations or convergence conditions are met;

[0012] S5: Algorithm output and analysis step, outputting the adjacency matrix containing pheromones, node information and edge information, and analyzing it to obtain fault path selection, fault impact analysis, fault propagation analysis and visualization chart results.

[0013] Furthermore, in step S1, information collection is implemented in the following manner:

[0014] S11: Use code decompilation to obtain information related to the code structure of the application, including the module and service structure and referenced third-party dependencies;

[0015] S12: Collect abnormal behaviors, error patterns, and business logic related information recorded in the logs through log auditing, including the response frequency of business logic and the behavior patterns of user requests;

[0016] S13: Using stack analysis to obtain stack information when the system is running, so as to assist in analyzing the application architecture and running status;

[0017] S14: Use behavioral analysis to monitor the behavior patterns of applications, including calling behaviors between services.

[0018] Furthermore, in step S2:

[0019] In the structured processing step, an adjacency matrix is ​​used to represent the topological relationship between applications, and descriptive information including service name, resource usage, call frequency, retry mechanism, degradation strategy, automated operation and maintenance tool, and fault injection history is added to nodes and edges, wherein resource usage includes CPU usage, memory usage, network bandwidth usage, request response time, and number of concurrent requests; high-frequency services include interface information and corresponding retry and degradation mechanisms; and automated operation and maintenance tools include service monitoring tools; edge description information includes call relationship, communication protocol, and data transmission volume;

[0020] In the time series processing step, the original information is grouped according to the set time interval, and aggregation, cleaning, smoothing and minimum-maximum normalization are performed. The aggregation operation includes finding the average value, maximum value and minimum value. Data cleaning adopts the method of filling missing values ​​and correcting outliers. The smoothing adopts the moving average method or exponential smoothing method. The minimum-maximum normalization scales the data to between [0,1].

[0021] Furthermore, in step S3:

[0022] The number of ant colonies is set to be a multiple of the number of application topology nodes n;

[0023] The initial value of the pheromone concentration is;

[0024] The evaporation rate is set to a decimal between 0.1 and 0.5. The pheromone matrix is ​​set based on the application service adjacency matrix. The pheromone value τ of the node is set by traversing ij =τ 0 .

[0025] Furthermore, in step S4, each ant gradually builds a path, and when the current node i probabilistically selects the next node j, the probability P ij The basic calculation formula is:

[0026]

[0027] α is the pheromone weight, β is the heuristic indicator weight, τ ij is the pheromone concentration on edge (i,j), η ij is a heuristic indicator, and allowed is the set of next nodes that can be selected.

[0028] α is the pheromone weight, which determines the ant's reliance on the successful experience of existing barrier injection paths based on the barrier injection history collected in the information collection step on the service node and the barrier injection results during the traversal. The larger the value, the more the algorithm relies on the existing pheromone concentration, making the ants more inclined to choose paths with significant barrier injection history effects;

[0029] β is the heuristic indicator weight. Based on the high-frequency business and service monitoring information collected in the information collection step on the service node, it guides the ants to prioritize the exploration of high-impact paths. In scenarios with many critical paths in the system, increasing the value can make the algorithm more inclined to select these paths to cover the critical paths.

[0030] τ ij is the pheromone concentration on the (i,j) edge;

[0031] η ij It is a heuristic indicator based on the resource usage, business call frequency, service dependency, downgrade retry mechanism and other information collected in the information collection step on the service node and the sensitivity of the service node to the atomic capability of failure. The specific calculation method is to comprehensively evaluate the importance and risk level of nodes and edges based on the above information, and express it in numerical form to guide ants to choose paths.

[0032] Furthermore, in step S4, the pheromone matrix update includes the following steps:

[0033] The pheromone volatilization formula is τ ij (new)=(1-ρ)τ ij (old), where ρ is the volatility of pheromone, ranging from 0.1 to 0.5, to prevent infinite accumulation of pheromone and maintain the diversity of path exploration;

[0034] The formula for increasing pheromones is: Where Q is the total amount of pheromone, L k is the total length or cost of the path taken by the kth ant. When calculating, it is necessary to accurately count the length or cost of the ant path and reasonably distribute it according to the total amount of pheromone to strengthen the high-quality path and make the effective path easier to be selected by subsequent ants;

[0035] The pheromone update formula is τ ij (new) = τ ij (old)+Δτ ij , ensuring that the pheromone matrix is ​​updated timely and accurately to reflect changes in the quality of the paths.

[0036] Further, in step S5:

[0037] The fault path selection is performed by traversing the pheromone matrix, sorting the pheromone concentration from high to low, selecting the path with higher concentration as the fault path that may affect the stability of the system, and analyzing the dependency relationship between services on the path to determine the potential fault point;

[0038] The fault impact analysis simulates fault injection into the service on the fault path, uses a test tool to accurately measure the service delay, availability degradation and fault recovery time, and compares the changes before and after the fault injection to evaluate the fault impact;

[0039] The fault propagation analysis sets monitoring points in the system, tracks the propagation process of faults between services, records the fault propagation path and the affected services, and analyzes the blocking or mitigation effect of existing retry and degradation mechanisms on fault propagation when a fault occurs;

[0040] The visualization chart is generated by data collection and graphics drawing tools, showing one or more of the CPU usage, memory usage changes, response time changes, and request frequency changes of the service node under the influence of a failure, providing developers with intuitive data display and assisting in system optimization and improvement.

[0041] Furthermore, the method also includes an environment migration step. When migrating the experimental implementation plan from the development or test environment to the production environment, a configuration file mapping or script conversion method is used to handle the encryption obfuscation and log level adjustment that may exist in the application in the production environment, so as to ensure the smooth transfer and effective implementation of the experiment between different environments;

[0042] Specifically, it includes: identifying the differences between configuration files in the development or test environment and the production environment, establishing a mapping relationship, and accurately converting the configuration parameters in the development or test environment into the parameters required for the production environment; for encrypted and obfuscated code, enabling the experimental tool to correctly identify and process it by parsing the encryption algorithm or using an adapted decryption tool; adjusting the log level according to the requirements of the production environment, while ensuring that sufficient log information can be obtained for analysis and evaluation during the experiment, and that the system performance is not affected by excessive log volume.

[0043] The present invention adopts the above technical solution, which has at least the following beneficial effects:

[0044] 1. Improve experimental efficiency: Comprehensive information collection and targeted processing: Comprehensively collect application-related information through a variety of analysis methods (such as code decompilation, log auditing, stack analysis, behavior analysis, etc.), covering application architecture, operating status, exception handling mechanism, automated operation and maintenance capabilities, etc., making information collection more targeted. After classification and organization, this information provides a rich and orderly data foundation for subsequent experiments, reducing the time for information screening and organization.

[0045] Optimize data preprocessing: Perform structured processing (such as using adjacency matrices to represent topological relationships) and time series modeling (including grouping at specific intervals, aggregation, cleaning, smoothing, and normalization) on the collected complex raw data, and convert unstructured data into a format suitable for algorithm processing, effectively reducing data processing time, improving data quality, and enabling the algorithm to process data faster and make decisions.

[0046] Intelligent algorithm decision-making and execution: Based on the ant colony algorithm, fault injection strategies are automatically generated without a lot of manual intervention and cumbersome configuration. The algorithm automatically traverses the path and updates the pheromone matrix according to the set rules and collected information, which greatly shortens the experimental preparation and implementation time, improves the overall efficiency of chaos engineering experiments, reduces the need for manual parameter adjustment, and reduces the difficulty of implementing chaos engineering in complex systems.

[0047] 2. Enhanced experimental accuracy: Accurate information acquisition and analysis: The multi-category detailed information obtained in the information collection step (covering application architecture, operating status, exception handling mechanism, automated operation and maintenance capabilities, etc.) provides a comprehensive basis for accurate analysis of the system. For example, the code structure information obtained through code decompilation helps to deeply understand the internal logic of the system, and the abnormal behavior and business logic information provided by log auditing can help discover potential problems.

[0048] Effective data preprocessing and feature mining: Structural and time series processing in the data preprocessing step can better mine data features and patterns, providing more accurate input for the algorithm. For example, aggregation operations can obtain the central tendency and discreteness of data, cleaning and smoothing can remove the interference of noise and outliers, and normalization processing can make different types of data comparable, thereby improving the accuracy of the algorithm's judgment of the system status.

[0049] Intelligent algorithm path selection and optimization: During the algorithm execution process, the path is selected according to the probability calculation formula, and combined with heuristic indicators (calculated based on the resource usage of nodes and edges and historical fault data), the selection of fault injection points and parameters is more accurate. The ant colony algorithm can guide ants to choose a better path based on pheromone concentration and heuristic indicators, thereby more realistically simulating possible fault conditions in the system, improving the accuracy of the assessment of system resilience and fault tolerance, and discovering previously unconsidered fault injection paths.

[0050] 3. Realize dynamic adaptability: Real-time information feedback and algorithm adjustment: In the algorithm execution step, according to the system real-time monitoring data and experimental feedback, the parameters and strategies of fault injection can be dynamically adjusted through the update of the pheromone matrix (including volatilization, addition and update operations). For example, when a path is found to perform poorly in the experiment, the volatilization of pheromones will reduce the probability of selecting the path, while the increase of pheromones on the successful path will make it more popular, thereby guiding the algorithm to explore a better path.

[0051] Flexible response to system changes and business needs: This paper introduces the ant colony algorithm into chaos engineering, and uses its characteristics to balance multiple indicators to obtain a better chaos engineering implementation strategy, so that the algorithm can dynamically adjust the search strategy according to system feedback and automatically generate application scenario experiments. Whether it is changes in the system architecture, fluctuations in business traffic, or the influence of other external factors, this method can respond in a timely manner to ensure that the experiment always matches the actual operating environment and business needs of the system, effectively improving the effectiveness and practicality of the experiment, so that the chaos engineering experiment can better adapt to the complex and changeable actual situation.

[0052] 4. Improve system reliability and resilience: Comprehensive risk assessment and discovery: Based on the above advantages of improving experimental efficiency, accuracy and dynamic adaptability, the invention can more comprehensively and deeply discover potential risk points and weak links in the system. By simulating various complex fault conditions in the experiment and conducting effective evaluations, the development team can optimize and improve the system in a targeted manner and take measures in advance to enhance the system's fault tolerance and recovery capabilities.

[0053] Enhance system stability and business assurance: This improves the reliability and resilience of software systems in actual operation, reduces the probability of system failures, ensures the continuous and stable operation of businesses, and reduces economic losses and business risks caused by system failures. For example, in financial systems, discovering and repairing potential failure points through chaos engineering experiments can effectively avoid serious problems such as transaction interruptions and data loss, and ensure the normal development of financial business.

[0054] 5. Promote the application and promotion of chaos engineering in complex systems: reduce the difficulty and threshold of implementation: Traditional chaos engineering experiments face many difficulties in complex systems, such as difficulty in implementation and low efficiency, which limits their wide application. This invention reduces the difficulty of implementation, improves efficiency and effect by optimizing the experimental implementation method, making the application of chaos engineering in complex software systems more feasible and easy to operate.

[0055] Promote technology popularization and industry development: This will help encourage more companies and developers to adopt chaos engineering technology to ensure the quality of software systems, promote the popularization and development of chaos engineering technology in the field of software engineering, and promote technological progress in system reliability and resilience across the industry. For example, small and medium-sized enterprises can use this invention to more easily introduce chaos engineering into their software development and operation and maintenance processes, thereby improving overall software quality and competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 It is a flow chart of the chaos engineering application scenario experiment optimization method based on ant colony algorithm of the present invention;

[0058] Figure 2 It is a schematic diagram of classification and detailed description of application related information obtained in the information collection step of the present invention;

[0059] Figure 3 It is a schematic diagram of node description information of the xxx-biz service of the present invention;

[0060] Figure 4 It is a schematic diagram of the edge description information of the xxx-biz service and the xxx-upms service of the present invention. DETAILED DESCRIPTION

[0061] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0062] like Figure 1 As shown, this embodiment provides a chaos engineering application scenario experiment optimization method based on ant colony algorithm, comprising the following steps:

[0063] S1: Information collection step, obtaining and storing application-related information through a variety of static and dynamic means;

[0064] Figure 2 This is a schematic diagram of the classification and detailed description of the application-related information obtained in the information collection step, which intuitively shows the specific content and mutual relationship of various types of information such as application architecture information, application operation status information, exception handling mechanism information, and automated operation and maintenance capability information. Figure 2 The application-related information includes application architecture information, application running status information, exception handling mechanism information, and automated operation and maintenance capability information. The application architecture information includes module and service structure (collecting the modular design of the application, the dependencies between services, and the referenced third-party dependencies), data flow (identifying the path of data flow, and understanding the transmission path of data from input to output), network topology (network communication path between system components), and the application running status includes service health status (collecting the health indicators of each service (CPU, memory, network bandwidth, etc.)), log information (collecting logs to obtain records of abnormal behavior and possible error mode, response frequency of business logic, behavior pattern of user requests, etc.), fault injection history (collect whether chaos experiments have been performed before and their impact), the exception handling mechanism includes retry mechanism (when the system fails, whether there is a retry mechanism, the frequency and conditions of the retry), circuit breaker strategy (when the service is unavailable, whether there is a circuit breaker mechanism to protect the system), degradation strategy (whether some functions can be downgraded when the service fails), the automated operation and maintenance capabilities include monitoring and alarm systems (analyzing whether there are monitoring tools to capture abnormal behaviors) and self-recovery capabilities (when the system encounters problems, whether there is a self-recovery mechanism).

[0065] S2: Data preprocessing step, performing structured processing and time series processing on the collected raw data, wherein the structured processing includes using an adjacency matrix to represent the topological relationship between applications and adding description information to nodes and edges, and the time series processing includes grouping the information at specific intervals, and then aggregating, cleaning, smoothing, and minimum-maximum normalization of the data within the group;

[0066] S3: Algorithm initialization step, importing the preprocessed information into the ant colony algorithm and setting the initial parameters, including determining one or more parameters of the ant colony number, pheromone concentration and evaporation rate, and initializing the pheromone matrix to make the pheromone value on each edge equal. At the same time, the resource usage is used to provide weight information for path decision-making, and the key path is evaluated with the help of high-frequency business, service monitoring information and fault history.

[0067] S4: Algorithm execution step, start the algorithm to perform path traversal and pheromone matrix update. During the path traversal, the ant selects the next node at the current node according to a specific probability formula. After completing the path traversal, the pheromone matrix is ​​updated according to the three steps of pheromone volatilization, addition and update. Repeat the execution until the maximum number of iterations or convergence conditions are met;

[0068] S5: Algorithm output and analysis step, outputting the adjacency matrix containing pheromones, node information and edge information, and analyzing it to obtain fault path selection, fault impact analysis, fault propagation analysis and visualization chart results.

[0069] As an implementation mode, in step S1 of this embodiment, information collection is implemented in the following manner:

[0070] S11: Use code decompilation to obtain information related to the code structure of the application, including the module and service structure and referenced third-party dependencies;

[0071] S12: Collect abnormal behaviors, error patterns, and business logic related information recorded in the logs through log auditing, including the response frequency of business logic and the behavior patterns of user requests;

[0072] S13: Using stack analysis to obtain stack information when the system is running, so as to assist in analyzing the application architecture and running status;

[0073] S14: Use behavioral analysis to monitor the behavior patterns of applications, including calling behaviors between services.

[0074] As an implementation manner, in this embodiment, in step S2:

[0075] In the structured processing step, an adjacency matrix is ​​used to represent the topological relationship between applications, and descriptive information including service name, resource usage, call frequency, retry mechanism, degradation strategy, automated operation and maintenance tool, and fault injection history is added to nodes and edges, wherein resource usage includes CPU usage, memory usage, network bandwidth usage, request response time, and number of concurrent requests; high-frequency services include interface information and corresponding retry and degradation mechanisms; and automated operation and maintenance tools include service monitoring tools; edge description information includes call relationship, communication protocol, and data transmission volume;

[0076] The adjacency matrix is ​​used to represent the topological relationship between applications. Each service and module is regarded as a node in the graph. Descriptive information is added to each node and edge. Assume the following service scenario:

[0077] xxx-auth: Authorization service

[0078] xxx-upms: Universal user rights management service, which will call the xxx-auth service

[0079] xxx-biz: application business service, which will call xxx-auth and xxx-upms services

[0080] xxx-gateway: application gateway service, which will call xxx-auth, xxx-upms and xxx-biz services

[0081] It can be represented by the adjacency matrix shown in Table 1:

[0082] Table 1

[0083] xxx-gateway xxx-auth xxx-upms xxx-biz xxx-gateway 0 1 1 1 xxx-auth 0 0 0 0 xxx-upms 0 1 0 0 xxx-biz 0 1 1 0

[0084] The node description information of the xxx-biz service can be expressed as follows Figure 3 As shown:

[0085] The side description information of xxx-biz service and xxx-upms service is as follows Figure 4 As shown:

[0086] In the time series processing step, the original information is grouped according to the set time interval, and aggregation, cleaning, smoothing and minimum-maximum normalization are performed. The aggregation operation includes finding the average value, maximum value and minimum value. Data cleaning adopts the method of filling missing values ​​and correcting outliers. The smoothing adopts the moving average method or exponential smoothing method. The minimum-maximum normalization scales the data to between [0,1].

[0087] After determining the time interval (such as 5 seconds), the collected original information (application running status, exception handling mechanism execution, monitoring alarm status, etc.) is grouped by time interval, and then the data in each time period group is aggregated (average, maximum, minimum), and data cleaning is performed to fill in the missing data in the time period and correct the outliers. The aggregated data is smoothed to reduce noise, and finally the minimum-maximum normalization is used to scale the data to [0,1] for easy use by the ant colony algorithm.

[0088] As an implementation mode, in this embodiment, in step S3:

[0089] The number of ant colonies is set to be a multiple of the number of application topology nodes n;

[0090] Generally, it is 2 to 5 times the number of application topology nodes n;

[0091] The initial value of the pheromone concentration is;

[0092] The evaporation rate is set to a decimal between 0.1 and 0.5. The pheromone matrix is ​​set based on the application service adjacency matrix. The pheromone value τ of the node is set by traversing ij =τ 0 .

[0093] Furthermore, in step S4, each ant gradually builds a path, and when the current node i probabilistically selects the next node j, the probability P ij The basic calculation formula is:

[0094] in

[0095] α is the pheromone weight, β is the heuristic indicator weight, τ ij is the pheromone concentration on edge (i,j), η ij is a heuristic indicator, and allowed is the set of next nodes that can be selected.

[0096] α is the pheromone weight, which determines the ant's reliance on the successful experience of existing barrier injection paths based on the barrier injection history collected in the information collection step on the service node and the barrier injection results during the traversal. The larger the value, the more the algorithm relies on the existing pheromone concentration, making the ants more inclined to choose paths with significant barrier injection history effects;

[0097] β is the heuristic indicator weight. Based on the high-frequency business and service monitoring information collected in the information collection step on the service node, it guides the ants to prioritize the exploration of high-impact paths. In scenarios with many critical paths in the system, increasing the value can make the algorithm more inclined to select these paths to cover the critical paths.

[0098] τ ijis the pheromone concentration on the (i,j) edge;

[0099] η ij It is a heuristic indicator based on the resource usage, business call frequency, service dependency, downgrade retry mechanism and other information collected in the information collection step on the service node and the sensitivity of the service node to the atomic capability of failure. The specific calculation method is to comprehensively evaluate the importance and risk level of nodes and edges based on the above information, and express it in numerical form to guide ants to choose paths.

[0100] As an implementation mode, in this embodiment, in step S4, the pheromone matrix update includes the following steps:

[0101] The pheromone volatilization formula is τ ij (new)=(1-ρ)τ ij (old), where ρ is the volatility of pheromone, ranging from 0.1 to 0.5, to prevent infinite accumulation of pheromone and maintain the diversity of path exploration;

[0102] The formula for increasing pheromones is: Where Q is the total amount of pheromone, L k is the total length or cost of the path taken by the kth ant. When calculating, it is necessary to accurately count the length or cost of the ant path and reasonably distribute it according to the total amount of pheromone to strengthen the high-quality path and make the effective path easier to be selected by subsequent ants;

[0103] The pheromone update formula is τ ij (new) = τ ij (old)+Δτ ij , ensuring that the pheromone matrix is ​​updated timely and accurately to reflect changes in the quality of the paths.

[0104] As an implementation manner, in this embodiment, in step S5:

[0105] The fault path selection is performed by traversing the pheromone matrix, sorting the pheromone concentration from high to low, selecting the path with higher concentration as the fault path that may affect the stability of the system, and analyzing the dependency relationship between services on the path to determine the potential fault point;

[0106] The fault impact analysis simulates fault injection into the service on the fault path, uses a test tool to accurately measure the service delay, availability degradation and fault recovery time, and compares the changes before and after the fault injection to evaluate the fault impact;

[0107] The fault propagation analysis sets monitoring points in the system, tracks the propagation process of faults between services, records the fault propagation path and the affected services, and analyzes the blocking or mitigation effect of existing retry and degradation mechanisms on fault propagation when a fault occurs;

[0108] The visualization chart is generated by data collection and graphics drawing tools, showing one or more of the CPU usage, memory usage changes, response time changes, and request frequency changes of the service node under the influence of a failure, providing developers with intuitive data display and assisting in system optimization and improvement.

[0109] As an implementation mode, the method described in this embodiment further includes an environment migration step. When migrating the experimental implementation plan from the development or test environment to the production environment, a configuration file mapping or script conversion method is used to perform corresponding processing for the encryption obfuscation and log level adjustment that may exist in the application in the production environment, so as to ensure the smooth transfer and effective implementation of the experiment between different environments;

[0110] Specifically, it includes: identifying the differences between configuration files in the development or test environment and the production environment, establishing a mapping relationship, and accurately converting the configuration parameters in the development or test environment into the parameters required for the production environment; for encrypted and obfuscated code, enabling the experimental tool to correctly identify and process it by parsing the encryption algorithm or using an adapted decryption tool; adjusting the log level according to the requirements of the production environment, while ensuring that sufficient log information can be obtained for analysis and evaluation during the experiment, and that the system performance is not affected by excessive log volume.

[0111] In this embodiment, a chaos engineering application scenario experiment implementation method based on ant colony algorithm is designed. The relevant information of the managed applications is collected by dynamic and static means, and the information is processed and input into heuristic algorithms such as ant colony algorithm. The parameters and strategies of fault injection are dynamically adjusted through algorithm execution, and finally the fault injection path is generated. This can reduce manual intervention and configuration time, improve the coverage of chaos engineering experiments, make chaos experiments more efficient and accurate, and adapt to different test environments and business needs.

[0112] Although the embodiments of the present invention have been shown and described above, it will be appreciated that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, substitute and vary the above embodiments within the scope of the present invention.

Claims

1. A chaos engineering application scenario experiment optimization method based on ant colony algorithm, characterized by: The following steps are involved: S1: Information collection step, obtaining and storing application-related information through a variety of static and dynamic means, wherein the application-related information includes application architecture information, application running status information, exception handling mechanism information, and automated operation and maintenance capability information, wherein the application architecture information covers modules and service structures, data flows, and network topologies, the application running status information includes service health status, log information, and fault injection history, the exception handling mechanism information includes retry mechanisms, fuse strategies, and downgrade strategies, and the automated operation and maintenance capability information involves monitoring, alarm systems, and self-recovery mechanisms; S2: Data preprocessing step, performing structured processing and time series processing on the collected raw data, wherein the structured processing includes using an adjacency matrix to represent the topological relationship between applications and adding description information to nodes and edges, and the time series processing includes grouping the information at specific intervals, and then aggregating, cleaning, smoothing, and minimum-maximum normalization of the data within the group; S3: Algorithm initialization step, importing the preprocessed information into the ant colony algorithm and setting the initial parameters, including determining one or more parameters of the ant colony number, pheromone concentration and evaporation rate, and initializing the pheromone matrix to make the pheromone value on each edge equal. At the same time, the resource usage is used to provide weight information for path decision-making, and the key path is evaluated with the help of high-frequency business, service monitoring information and fault history. S4: Algorithm execution step, start the algorithm to perform path traversal and pheromone matrix update. During the path traversal, the ant selects the next node at the current node according to a specific probability formula. After completing the path traversal, the pheromone matrix is ​​updated according to the three steps of pheromone volatilization, addition and update. Repeat the execution until the maximum number of iterations or convergence conditions are met; S5: Algorithm output and analysis step, outputting the adjacency matrix containing pheromones, node information and edge information, and analyzing it to obtain fault path selection, fault impact analysis, fault propagation analysis and visualization chart results.

2. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to claim 1 is characterized by: In step S1, information collection is specifically achieved in the following manner: S11: Use code decompilation to obtain information related to the code structure of the application, including the module and service structure and referenced third-party dependencies; S12: Collect abnormal behaviors, error patterns, and business logic related information recorded in the logs through log auditing, including the response frequency of business logic and the behavior patterns of user requests; S13: Using stack analysis to obtain stack information when the system is running, so as to assist in analyzing the application architecture and running status; S14: Use behavioral analysis to monitor the behavior patterns of applications, including calling behaviors between services.

3. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to claim 1 is characterized by: In step S2: In the structured processing step, an adjacency matrix is ​​used to represent the topological relationship between applications, and descriptive information including service name, resource usage, call frequency, retry mechanism, degradation strategy, automated operation and maintenance tool, and fault injection history is added to nodes and edges, wherein resource usage includes CPU usage, memory usage, network bandwidth usage, request response time, and number of concurrent requests; high-frequency services include interface information and corresponding retry and degradation mechanisms; and automated operation and maintenance tools include service monitoring tools; edge description information includes call relationship, communication protocol, and data transmission volume; In the time series processing step, the original information is grouped according to the set time interval, and aggregation, cleaning, smoothing and minimum-maximum normalization are performed. The aggregation operation includes finding the average value, maximum value and minimum value. Data cleaning adopts the method of filling missing values ​​and correcting outliers. The smoothing adopts the moving average method or exponential smoothing method. The minimum-maximum normalization scales the data to between [0,1].

4. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to claim 1 is characterized by: In step S3: The number of ant colonies is set to be a multiple of the number of application topology nodes n; The initial value of the pheromone concentration is; The evaporation rate is set to a decimal between 0.1 and 0.

5. The pheromone matrix is ​​set based on the application service adjacency matrix. The pheromone value τ of the node is set by traversing ij =τ0.

5. The method for optimizing chaos engineering application scenario experiments based on ant colony algorithm according to claim 1, characterized in that: In step S4, each ant gradually builds a path. When the current node i probabilistically selects the next node j, the probability P ij The basic calculation formula is: in α is the pheromone weight, β is the heuristic indicator weight, τ ij is the pheromone concentration on edge (i,j), η ij is a heuristic indicator, and allowed is the set of next nodes that can be selected. α is the pheromone weight, which determines the ant's reliance on the successful experience of existing barrier injection paths based on the barrier injection history collected in the information collection step on the service node and the barrier injection results during the traversal. The larger the value, the more the algorithm relies on the existing pheromone concentration, making the ants more inclined to choose paths with significant barrier injection history effects; β is the heuristic indicator weight. Based on the high-frequency business and service monitoring information collected in the information collection step on the service node, it guides the ants to prioritize the exploration of high-impact paths. In scenarios with many critical paths in the system, increasing the value can make the algorithm more inclined to select these paths to cover the critical paths. τ ij is the pheromone concentration on the (i, j) edge; η ij It is a heuristic indicator based on the resource usage, business call frequency, service dependency, downgrade retry mechanism and other information collected in the information collection step on the service node and the sensitivity of the service node to the atomic capability of failure. The specific calculation method is to comprehensively evaluate the importance and risk level of nodes and edges based on the above information, and express it in numerical form to guide ants to choose paths.

6. The method for optimizing chaos engineering application scenario experiments based on ant colony algorithm according to claim 1, characterized in that: In step S4, the pheromone matrix update includes the following steps: The pheromone volatilization formula is τ ij (new)=(1-ρ)τ ij (old), where ρ is the volatility of pheromone, ranging from 0.1 to 0.5, to prevent infinite accumulation of pheromone and maintain the diversity of path exploration; The formula for increasing pheromones is: Where Q is the total amount of pheromone, L k is the total length or cost of the path taken by the kth ant. When calculating, it is necessary to accurately count the length or cost of the ant path and reasonably distribute it according to the total amount of pheromone to strengthen the high-quality path and make the effective path easier to be selected by subsequent ants; The pheromone update formula is τ ij (new) = τ ij (old)+Δτ ij , ensuring that the pheromone matrix is ​​updated timely and accurately to reflect changes in the quality of the paths.

7. The method for optimizing chaos engineering application scenario experiments based on ant colony algorithm according to claim 1, characterized in that: In step S5: The fault path selection is performed by traversing the pheromone matrix, sorting the pheromone concentration from high to low, selecting the path with higher concentration as the fault path that may affect the stability of the system, and analyzing the dependency relationship between services on the path to determine the potential fault point; The fault impact analysis simulates fault injection into the service on the fault path, uses a test tool to accurately measure the service delay, availability degradation and fault recovery time, and compares the changes before and after the fault injection to evaluate the fault impact; The fault propagation analysis sets monitoring points in the system, tracks the propagation process of faults between services, records the fault propagation path and the affected services, and analyzes the blocking or mitigation effect of existing retry and degradation mechanisms on fault propagation when a fault occurs; The visualization chart is generated by data collection and graphics drawing tools, showing one or more of the CPU usage, memory usage changes, response time changes, and request frequency changes of the service node under the influence of a failure, providing developers with intuitive data display and assisting in system optimization and improvement.

8. The chaos engineering application scenario experiment optimization method based on ant colony algorithm according to any one of claims 1 to 7, characterized in that: The method also includes an environment migration step, in which, when migrating the experimental implementation plan from the development or test environment to the production environment, configuration file mapping or script conversion is used to handle the encryption obfuscation and log level adjustment that may exist in the application in the production environment, so as to ensure the smooth transfer and effective implementation of the experiment between different environments; Specifically, it includes: identifying the differences between configuration files in the development or test environment and the production environment, establishing a mapping relationship, and accurately converting the configuration parameters in the development or test environment into the parameters required for the production environment; for encrypted and obfuscated code, enabling the experimental tool to correctly identify and process it by parsing the encryption algorithm or using an adapted decryption tool; adjusting the log level according to the requirements of the production environment, while ensuring that sufficient log information can be obtained for analysis and evaluation during the experiment, and that the system performance is not affected by excessive log volume.

Images