A method and system for adaptive configuration adjustment applied to distributed systems

Abstract

The present invention provides an adaptive configuration adjustment method and system for distributed systems, and relates to the fields of computer science and software engineering. Through the adaptive distributed system configuration adjustment method and system, developers are allowed to select different monitoring indicators and thresholds according to actual needs, as well as dynamically add, modify, and delete adjustment strategies. The problem of poor adaptability of traditional configuration methods in dynamic environments is solved, so that the system can automatically adjust the configuration to adapt to real-time changing environmental conditions. Real-time performance monitoring and optimization are provided to ensure that the system can be adjusted according to real-time performance data during operation, rather than the periodic optimization of static configuration methods, to achieve the maintainability of the configuration. In addition, an intelligent analysis module is introduced, which uses machine learning algorithms and rule engines to conduct in-depth analysis of real-time monitoring data, generate intelligent adjustment strategies, and improve the system's adaptability and intelligence.

Description

Technical Field

[0001] The present invention relates to the fields of computer science and software engineering, and in particular to an adaptive configuration adjustment method and system applied to a distributed system. Background Art

[0002] In the fields of computer science, distributed systems and software engineering, adaptive distributed system configuration adjustment is an important technology that aims to dynamically adjust the configuration parameters of software applications in real time according to factors such as system requirements, workloads and environmental conditions to optimize system performance, resource utilization and adaptability. This technical field involves configuration management, software design, performance optimization, adaptive systems and other aspects, aiming to provide a more intelligent and adaptive configuration management method and system for software applications, so that they can run flexibly under different conditions.

[0003] Traditional software systems often use static configurations when deployed. Once the configuration is completed, the system is difficult to adapt to changes during operation. This approach cannot meet the needs of dynamic adaptation, resulting in inefficient system performance and resource utilization.

[0004] In addition, modern software applications often face variable workloads, changing user needs, and different environmental conditions. Traditional configuration management is difficult to cope with this dynamic nature, so a more adaptive configuration adjustment method is needed. Summary of the invention

[0005] Purpose of the invention: To propose an adaptive configuration adjustment method for distributed systems, and further propose an automated system that can implement the above configuration adjustment method, so as to solve the limitations of traditional configuration management methods and realize more flexible, intelligent and adaptive software system configuration management. This will provide a more efficient and maintainable configuration management solution for software applications in different scenarios.

[0006] The first aspect of the present invention provides an adaptive configuration adjustment method for a distributed system, comprising the following steps:

[0007] S1. Collect performance data of the distributed system to be configured;

[0008] S2, analyzing the performance data and outputting analysis result data;

[0009] S3. Formulate a dynamic adjustment strategy based on the analysis result data, and send a dynamic adjustment strategy notification message;

[0010] S4, sending the dynamic adjustment strategy notification information to the affected components in the distributed system;

[0011] S5. The affected component receives the notification information and performs corresponding configuration adjustment operations;

[0012] S6. After completing the current round of configuration adjustment operations, each component reports the latest performance data to the distributed system and updates the monitoring information of the configuration center for use in the next round of configuration adjustment.

[0013] In a further embodiment of the first aspect, the performance data includes the CPU usage, memory occupancy, and network traffic of each component in the distributed system; the CPU usage, memory occupancy, and network traffic are summarized in the form of timestamps, and each timestamp corresponds to the CPU usage, memory occupancy, and network traffic at a current timestamp; multiple timestamps are summarized to form a performance data table.

[0014] In a further embodiment of the first aspect, parsing the performance data in step S2 includes the following process:

[0015] S2-1. The collected performance data is analyzed using linear regression:

[0016] Assume that the current goal is to predict CPU usage as the target variable, and the input features include memory usage and network traffic. The input features are used as input variables; the linear regression model is expressed as:

[0017] y=b0+b1·x1+b2·x2+...+bn·xn+ε;

[0018] Where y represents the target variable; x1, x2, ..., xn represent input variables; b1, b2, ..., bn represent the coefficients of input features; b0 represents the intercept; ε represents the error.

[0019] S2-2, formulate predefined rules;

[0020] Rule 1: If CPU usage exceeds 80%, a warning will be issued;

[0021] Rule 2: When the memory usage exceeds 70%, perform memory cleanup.

[0022] Rule 3: If the network traffic exceeds 5MB / s and lasts for more than 10 minutes, the system capacity will be automatically expanded.

[0023] S2-3. Utilize the rule engine to execute the predefined rules, verify and match the performance data collected in real time, and output potential problems or abnormal situations.

[0024] In a further embodiment of the first aspect, the output analysis result data in step S2 includes a timestamp and an analysis result corresponding to the timestamp; wherein the analysis result includes at least the following contents:

[0025] node_scaling: Expand the nodes. It is recommended to expand the system and add 2 nodes.

[0026] restart: Restart the node. It is recommended to restart the node named ComputeNode-01. The restart time is: 2023-12-01T23:59:59.

[0027] start: Start the node. It is recommended to start the node named ComputeNode-01. The execution start time is: 2023-12-01T15:25:00.

[0028] stop: Stop the node. It is recommended to stop the node named ComputeNode-01. The execution stop time is: 2023-12-01T22:00:00.

[0029] task_optimization: Optimize tasks. It is recommended to optimize the distribution of task1 and task2 in the task list and adjust them to node3 and node5. The adjusted execution time is: 2023-12-01T12:45:00.

[0030] The above output data analysis result format is only a sample data, and the actual output result will vary depending on the actual situation of the system.

[0031] In a further embodiment of the first aspect, the dynamic adjustment strategy in step S3 includes:

[0032] A. Node restart strategy:

[0033] Rules: When the status of a system or component becomes unavailable or reaches the fault threshold, the health status of the system or component is detected by receiving CPU usage, memory usage, and network traffic;

[0034] Execution: Trigger node restart and use the restart script to restart;

[0035] B. Node startup strategy:

[0036] Rules: For systems or components that have previously failed, define a startup strategy so that they can be repaired the next time they are started;

[0037] Execution: trigger node startup, start using startup script;

[0038] C. Node stopping strategy:

[0039] Rules: When the status of a system or component becomes unavailable or reaches a failure threshold, such as a rule that requires the system or component to be stopped immediately in the case of a 100% CPU usage overload, regardless of its current status;

[0040] Execution: Trigger the node to stop and use the stop script to stop it;

[0041] D. Extension adjustment strategy:

[0042] Rules: Based on CPU usage and memory usage, when the system load exceeds a threshold, dynamically increase the number of nodes; when the system load drops below another threshold, reduce the number of nodes;

[0043] Execution: Adjust the number of nodes;

[0044] E. Task allocation strategy:

[0045] Rules: Based on the execution time and resource usage, when the execution time of a task exceeds the threshold or the resource usage is high, the task is reallocated;

[0046] Execution: Triggers load balancing to ensure even distribution of tasks in the system.

[0047] In a further embodiment of the first aspect, step S4 further comprises:

[0048] S4-1, receiving dynamic adjustment strategy notification information;

[0049] S4-2, parsing the dynamic adjustment strategy notification information to obtain the adjustment type, target component, and execution time;

[0050] S4-3. Notify the component adjustment execution module of the parsed dynamic adjustment policy notification information, and the component adjustment execution module executes corresponding configuration adjustment operations on the affected components in the distributed system.

[0051] As a second aspect of the present invention, an adaptive configuration adjustment system is proposed, which includes a real-time monitoring module, an intelligent analysis module, a dynamic adjustment strategy generation module, a configuration adjustment notification module, and an adjustment execution module.

[0052] The real-time monitoring module is used to collect performance data of the distributed system to be configured.

[0053] The intelligent analysis module is used to analyze the performance data collected by the real-time monitoring module and output analysis result data.

[0054] The dynamic adjustment strategy generation module is used to formulate a dynamic adjustment strategy according to the analysis result data and send dynamic adjustment strategy notification information.

[0055] The configuration adjustment notification module is used to notify the affected components in the distributed system of the dynamic adjustment strategy notification information.

[0056] The adjustment execution module is used to drive the affected components to receive notification information and perform corresponding configuration adjustment operations. After completing the current round of configuration adjustment operations, each component reports the latest performance data to the distributed system and updates the monitoring information of the configuration center for use in the next round of configuration adjustment.

[0057] As a third aspect of the present invention, a computer-readable storage medium is proposed, which stores at least one executable instruction. When the executable instruction is executed on an electronic device, the electronic device executes the adaptive configuration adjustment method applied to a distributed system as described in the first aspect.

[0058] The method and system based on dynamic configuration adjustment of the present invention have achieved the following significant technical effects:

[0059] 1) Real-time performance optimization: Through real-time monitoring and intelligent analysis, the system can optimize performance in real time during operation, improve the system's response speed and resource utilization efficiency, and ensure that the system can still maintain good performance under high load.

[0060] 2) Intelligent adaptability: Using machine learning algorithms and rule engines, the system generates intelligent adjustment strategies and makes intelligent decisions based on real-time environmental changes and historical data. This improves the system's adaptability, enabling it to better adapt to different workloads and environmental conditions.

[0061] 3) Flexibility and configurability: The system supports flexible configuration, allowing users to select monitoring indicators and thresholds to meet the needs of different application scenarios. Users can predefine multiple adjustment strategies or dynamically add new strategies, which improves the flexibility and configurability of the system.

[0062] 4) Improved adaptability to complex application scenarios: Through intelligent analysis and dynamic adjustment strategies, the system can better adapt to complex and changing workloads, user needs and environmental conditions, solving the problem of insufficient adaptability of traditional configuration management methods in complex application scenarios.

[0063] 5) Improve the automation level of the system: The system of the present invention realizes the automation of configuration management and system adjustment through real-time monitoring, intelligent analysis and flexible adjustment, which reduces the workload of operation and maintenance personnel and improves the automation level of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a runtime diagram of an adaptive configuration adjustment method applied to a distributed system.

[0065] Figure 2 1 is an architecture diagram of an adaptive configuration adjustment system in an embodiment of the present invention.

[0066] Figure 3 It is an architectural diagram of the dynamic adjustment strategy generation module in the adaptive configuration adjustment system. DETAILED DESCRIPTION

[0067] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0068] With the widespread application of distributed systems in fields such as large-scale data processing, cloud computing, and microservice architecture, traditional configuration management methods face a series of technical problems:

[0069] 1. Static configuration is not suitable for dynamic environments: Traditional software systems are usually statically configured during deployment, and this fixed configuration is difficult to adapt to dynamically changing workloads and environmental conditions.

[0070] 2. Lack of real-time performance optimization: For systems with high performance requirements, existing configuration adjustment methods are insufficient in real-time performance optimization, resulting in slow system response and low resource utilization.

[0071] 3. Lack of adaptability to complex application scenarios: Modern software applications face complex and changing workloads, user needs, and different environmental conditions. The existing configuration management methods are not adaptable enough, which makes configuration adjustment difficult.

[0072] 4. The configuration adjustment process lacks flexibility: The decision-making and execution in the configuration adjustment process usually lack flexibility and cannot effectively respond to changes in different application scenarios.

[0073] To this end, the present invention aims to provide an innovative method and system based on dynamic configuration adjustment, which solves the above-mentioned technical problems existing in the prior art through real-time monitoring, intelligent analysis and flexible adjustment, and realizes higher performance, better adaptability and more intelligent configuration management of the software system.

[0074] In summary, the method for adaptive distributed system configuration adjustment implemented by the adaptive configuration adjustment system includes the following steps:

[0075] 1) Deployment of real-time monitoring module: Deploy the real-time monitoring module in the distributed system. The module regularly collects the status and performance data of each component, including but not limited to CPU usage, memory usage, network traffic, etc.

[0076] 2) Modeling of intelligent analysis module: Use machine learning algorithms and rule engines to build intelligent analysis models. The model is trained with monitoring data to form the ability to predict changes in system performance. At the same time, it generates intelligent adjustment strategies based on predefined performance indicators and outputs the results to the dynamic adjustment strategy generation module.

[0077] 3) Configuration of dynamic adjustment strategy generation module: According to system requirements, configure dynamic adjustment strategy generation strategy and determine key indicators, thresholds and other parameters that need to be monitored. Predefine multiple adjustment strategies, including node expansion and reduction, expansion adjustment, task allocation optimization, etc.

[0078] 4) Setting of configuration adjustment notification module: Setting of configuration adjustment notification module ensures that the generated dynamic adjustment strategy can be notified to the adjustment execution module in a timely manner. The notification includes key information such as adjustment type, target component, execution time, etc.

[0079] 5) Implementation of the adjustment execution module: Deploy the adjustment execution module in the system component to ensure that it can receive and execute configuration adjustment notifications. The module is responsible for performing specific adjustment operations, such as starting or stopping nodes, scaling adjustments, and reallocating tasks.

[0080] 6) Status update and monitoring information update: The status update and monitoring information update module ensures that each component reports the latest status and performance data to the system after the adjustment is completed. Update the dynamic adjustment strategy generation module data to provide accurate data for the next round of intelligent analysis and adjustment.

[0081] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0082] Figure 1 The method based on adaptive distributed system configuration adjustment and the overall operation process of the system are presented in Figure 1 In the system, each module has a specific function, and the interaction process between modules can be clearly seen through the timing diagram. The real-time monitoring module collects performance data and passes it to the intelligent analysis module. The intelligent analysis module analyzes the data to generate dynamic adjustment strategies, and then notifies other components of the adjustment information through the configuration adjustment notification module, which helps to understand the workflow of each module in the system.

[0083] Figure 2 The schematic diagram shows a method for adaptive distributed system configuration adjustment and the overall structure of the system. Through real-time monitoring, intelligent analysis, and generation of dynamic adjustment strategies, the system can automatically adjust the configuration to adapt to changing workloads and environmental conditions. The modules work together to form a highly adaptive and intelligent dynamic configuration adjustment system.

[0084] from Figure 2It can be seen that this adaptive configuration adjustment system is responsible for real-time monitoring of the status, performance indicators and dynamic changes of the operating environment of each component in the system through the real-time monitoring module. The collected performance data is passed to the intelligent analysis module for intelligent analysis and adjustment strategy generation. According to the output results of the intelligent analysis module, a specific dynamic adjustment strategy is formulated to provide a flexible and intelligent dynamic adjustment solution for the system, thereby improving the maintainability and performance optimization capabilities of the system. Its functions and implementation involve real-time monitoring modules, intelligent analysis modules, dynamic adjustment strategy generation modules, configuration adjustment notification modules, and adjustment execution modules. The following are the various system modules and specific implementation processes of the patent, describing the key steps of the method and system based on dynamic configuration adjustment.

[0085] The real-time monitoring module is one of the basic components of the system. It aims to track and record the key performance indicators and dynamic changes of each component in the system in real time, and provide real-time data support for system adjustment.

[0086] The real-time monitoring module includes startup and initialization: the real-time monitoring module is activated when each system is started and completes the necessary initialization work.

[0087] The real-time monitoring module includes continuous monitoring of component status: This module is responsible for continuously monitoring the operating status of each component in the system, including but not limited to node load, memory usage, network traffic, etc.

[0088] The real-time monitoring module includes regular performance data collection: regular collection of performance data of each system component to ensure the latest operating status information. The collected performance data covers key indicators such as CPU usage, memory usage, network traffic, etc.

[0089] The real-time monitoring module includes data transmission to the intelligent analysis module: the real-time collected performance data is transmitted to the intelligent analysis module to provide a basis for subsequent data analysis and intelligent adjustment strategy generation.

[0090] The real-time monitoring module can perform the following data collection process:

[0091] 1) Module startup: The real-time monitoring module is started when the distributed system starts and begins its monitoring task;

[0092] 2) Continuous monitoring: This module continuously monitors each component in the system and captures status changes in real time;

[0093] 3) Data collection: This module regularly performs data collection tasks to obtain system performance data;

[0094] 4) Pass to the intelligent analysis module: The collected performance data is passed to the intelligent analysis module for use in intelligent analysis and adjustment strategy generation.

[0095] Through the above functions and implementations, the real-time monitoring module provides basic support for real-time performance monitoring and data collection for the system, and provides a reliable data source for subsequent intelligent analysis and dynamic adjustment strategies.

[0096] The above-mentioned intelligent analysis module is a key component of the system. It uses technologies such as machine learning algorithms and rule engines to conduct in-depth analysis of real-time monitoring data to generate intelligent adjustment strategies and improve system performance and efficiency.

[0097] The above-mentioned intelligent analysis module includes data transmission and reception: receiving the performance data transmitted by the real-time monitoring module to ensure timely acquisition of the latest system status information.

[0098] The above performance data is in the following table:

[0099]

[0100] The above intelligent analysis module includes a machine learning algorithm: For the above set of collected performance data, linear regression is used for analysis. Suppose our goal is to predict CPU usage as the target variable (y), and the input features include memory usage and network traffic as input variables (x1, x2, ..., xn). The linear regression model can be expressed as:

[0101] y=b0+b1·x1+b2·x2+…+bn·xn+ε

[0102] Where y represents the target variable; x1, x2, ..., xn represent input variables; b1, b2, ..., bn represent the coefficients of input features; b0 represents the intercept; ε represents the error.

[0103] This embodiment discloses a feasible implementation code of the above machine learning algorithm:

[0104]

[0105]

[0106] The above intelligent analysis module includes the following predefined rules:

[0107] A. If the CPU usage exceeds 80%, a warning is issued;

[0108] B. If the memory usage exceeds 70%, perform memory cleanup;

[0109] C. If the network traffic exceeds 5MB / s and lasts for more than 10 minutes, the system capacity will be automatically expanded. This embodiment discloses a set of feasible specific implementation codes of the above predefined rules:

[0110]

[0111]

[0112]

[0113] The above intelligent analysis module includes the application of a rule engine: the rule engine is used to execute the predefined rules mentioned above, verify and match real-time data, output potential problems or abnormal situations, and enhance the system's ability to identify abnormal situations.

[0114] The above-mentioned intelligent analysis module includes output analysis results: the output results of the intelligent analysis module are intelligent adjustment strategies obtained based on the analysis of machine learning algorithms and rule engines.

[0115] The data format of the output analysis results is as follows:

[0116]

[0117]

[0118] In the above output analysis result data, timestamp indicates the timestamp of the analysis result, and analysis_result contains the output result of the intelligent analysis module. The specific analysis results include:

[0119] A.node_scaling: Expand the nodes. It is recommended to expand the system and add 2 nodes.

[0120] B.restart: Restart the node. It is recommended to restart the node named ComputeNode-01. The restart time is: 2023-12-01T23:59:59.

[0121] C.start: Start the node. It is recommended to start the node named ComputeNode-01. The execution start time is: 2023-12-01T15:25:00.

[0122] D.stop: Stop the node. It is recommended to stop the node named ComputeNode-01. The execution stop time is: 2023-12-01T22:00:00.

[0123] E.task_optimization: Optimize tasks. It is recommended to optimize the distribution of task1 and task2 in the task list and adjust them to node3 and node5. The adjusted execution time is: 2023-12-01T12:45:00.

[0124] The above output data analysis result format is only a sample data, and the actual output result will vary depending on the actual situation of the system.

[0125] The data processing flow is as follows:

[0126] 1) Data reception: The intelligent analysis module receives the performance data transmitted by the real-time monitoring module;

[0127] 2) Application of machine learning algorithms: Use machine learning algorithms to analyze real-time performance data and learn the operating modes and rules of the system;

[0128] 3) Rule engine verification: Use the rule engine to verify real-time data to confirm whether it complies with predefined rules to identify potential problems or anomalies;

[0129] 4) Output analysis results: Analyze real-time performance data through machine learning algorithms, and use the rule engine to verify the intelligent adjustment strategy derived from the real-time data and analysis;

[0130] 5) Send analysis results: Send the intelligent adjustment strategy results obtained from the analysis to the dynamic adjustment strategy generation module.

[0131] The intelligent analysis module has the following advantages:

[0132] A. Intelligence: Through machine learning algorithms, the module can learn the behavior of the system and make intelligent decisions;

[0133] B. Accuracy: Combined with the verification of the rule engine, it improves the accurate identification of abnormal situations and reduces false positives;

[0134] C. Flexibility: The configuration of machine learning models and rule engines can be adjusted according to actual needs to adapt to different systems and environments.

[0135] Through the above functions, the intelligent analysis module can realize in-depth analysis of real-time monitoring data, provide intelligent adjustment suggestions for the system, and improve system performance and stability.

[0136] The dynamic adjustment strategy generation module formulates specific dynamic adjustment strategies based on the output results of the intelligent analysis module, covering various adjustment operations such as node management, expansion adjustment, and task allocation.

[0137] Figure 3 The diagram shows the architecture of the dynamic adjustment strategy generation module. As a core component, the dynamic adjustment strategy generation module establishes a real-time communication channel with various nodes, databases, tasks and other components in the distributed system to achieve real-time monitoring and control of the system status.

[0138] The specific functions of the dynamic adjustment strategy generation module and the strategy generation implementation process are as follows:

[0139] The dynamic adjustment strategy generation module includes user configuration and settings: allowing users to configure the dynamic adjustment strategy generation module, including setting monitoring indicators, thresholds and other parameters. Users can flexibly configure according to actual needs to adapt to different application scenarios;

[0140] User configuration and settings include user configuration interface: providing a user-friendly configuration interface that allows users to customize monitoring indicators and thresholds.

[0141] The above users can set indicators and thresholds through the configuration interface provided by the system, including:

[0142] A. Indicators: Select the indicators you want to monitor, such as CPU usage, memory usage, network traffic, etc.

[0143] B. Threshold: Set the threshold that triggers the adjustment, such as CPU usage exceeding 80% triggers node expansion.

[0144] The above user configuration and settings include configuration file support: support for configuration of system parameters through configuration files to meet higher-level user needs.

[0145] The data format of the configuration file is as follows:

[0146]

[0147] The above example shows how to define monitoring indicators and thresholds through configuration. This configuration method makes the system easier to expand and adapt to different usage scenarios. Configuration data Through the user configuration interface or configuration file, the administrator can specify monitoring indicators and thresholds as follows:

[0148] A. Indicators: CPUUsage, MemoryUsage, NetworkTraffic

[0149] B. Threshold: CPUUsage>80, MemoryUsage>90, NetworkTraffic>100

[0150] These configurations will guide the system to make dynamic adjustments during runtime to ensure system performance and stability.

[0151] The above dynamic adjustment strategy generation module includes intelligent adjustment strategy generation: based on the output of the intelligent analysis module, combined with user configuration and settings, a specific intelligent adjustment strategy is generated. The intelligent adjustment strategy includes specific adjustment operation suggestions, such as adding or reducing nodes, expansion adjustment, and optimizing task allocation.

[0152] The above dynamic adjustment strategy generation module includes rules and executions corresponding to different adjustment strategies:

[0153] A.Node restart strategy:

[0154] Rules: When the status of a system or component becomes unavailable or reaches a failure threshold, the health of the system or component is detected by receiving CPU, memory, network, etc.

[0155] Execution: Trigger node restart and use the restart script to restart.

[0156] B. Node startup strategy:

[0157] Rules: For systems or components that have previously failed, define a startup strategy so that they can be repaired the next time they are started.

[0158] Execution: Trigger node startup and start using the startup script.

[0159] C. Node stopping strategy:

[0160] Rules: When the status of a system or component becomes unavailable or reaches a failure threshold. For example, a rule that requires the system or component to be stopped immediately when the CPU usage reaches 100% overload, regardless of the current status.

[0161] Execution: Trigger node stop and use the stop script to stop.

[0162] D. Extension adjustment strategy:

[0163] Rule: When the system load exceeds a threshold based on CPU and memory usage, nodes are dynamically added; when the system load drops below another threshold, nodes are reduced.

[0164] Execution: Adjust the number of nodes.

[0165] E. Task allocation strategy:

[0166] Rule: Based on the execution time and resource usage, when the execution time of a task exceeds the threshold or the resource usage is high, reallocate the task.

[0167] Execution: Triggers load balancing to ensure that tasks are evenly distributed across the system, improving overall performance.

[0168] The strategy generation process specifically includes:

[0169] 1) User configuration: The user configures the dynamic adjustment strategy generation module and sets monitoring indicators, thresholds and other parameters;

[0170] 2) Intelligent analysis output: Receive the output results of the intelligent analysis module, including system performance status, potential problems, etc.;

[0171] 3) Strategy generation: Combine user configuration and settings to generate specific dynamic adjustment strategies. Intelligent adjustment strategies include node management, expansion adjustment, task allocation and other strategies.

[0172] Through the above functions, the dynamic adjustment strategy generation module realizes the combination of user customization and intelligent adjustment, provides a flexible and intelligent dynamic adjustment solution for the system, and improves the maintainability and performance optimization capabilities of the system.

[0173] The above configuration adjustment notification module is responsible for notifying the generated dynamic adjustment strategy to the adjustment execution module. The notification content includes key information such as the adjustment type, target component, and execution time. Its specific functions and notification implementation include:

[0174] The configuration adjustment notification module includes receiving an adjustment policy notification: when the dynamic adjustment policy generation module generates an adjustment policy, sending a notification to the configuration adjustment notification module;

[0175] The configuration adjustment notification module includes processing adjustment information: after receiving the adjustment policy, the configuration adjustment notification module parses the information therein, including the adjustment type, target component, execution time, etc.;

[0176] The configuration adjustment notification module includes a notification component adjustment execution module: based on the parsed information, the adjustment information is notified to the adjustment execution module. The notification content clearly specifies the adjustment type, target component, execution time and other key information.

[0177] The notification process specifically includes:

[0178] 1) Receiving adjustment policy notification: The configuration adjustment notification module receives the adjustment policy notification generated by the dynamic adjustment policy generation module;

[0179] 2) Parse the adjustment information: parse the information in the notification to obtain key information such as the adjustment type, target component, and execution time;

[0180] 3) Notify the component adjustment execution module: The parsed information is notified to the component adjustment execution module. The notification clearly states the details of the adjustment so that the component can make corresponding dynamic configuration adjustments.

[0181] The notification data content format is as follows:

[0182]

[0183] The above content shows a specific notification example, which clearly states that the adjustment type is to stop the node, the target component is ComputeNode-01, and the execution time is 2024-01-01 14:30:00. The clarity of the notification content helps system administrators and components to perform accurate and timely adjustment operations.

[0184] The configuration adjustment notification module has the following advantages:

[0185] A. Real-time asynchronous notification: The configuration adjustment notification module can receive and process the adjustment strategies generated by the intelligent analysis module in real time to ensure timely system adjustments;

[0186] B. Notification details: The notification content includes key information such as the adjustment type, target component, and execution time, providing detailed adjustment guidance;

[0187] C. Precise positioning: The notification clearly specifies the affected system components to avoid unnecessary impact on the entire system.

[0188] The above adjustment execution module is responsible for receiving configuration adjustment notifications and performing corresponding adjustment operations. These operations may include restarting, starting or stopping nodes, adjusting scale, and reallocating tasks. Its specific functions and adjustment processes include:

[0189] The above-mentioned component adjustment execution module includes receiving adjustment notification: the adjustment execution module receives adjustment information from the configuration adjustment notification module;

[0190] The component adjustment execution module includes parsing adjustment information: parsing information in the notification, including the type of adjustment, target component, execution time, etc.;

[0191] The above-mentioned component adjustment execution module includes executing adjustment operations: executing specific adjustment operations according to the parsed information. Possible operations include starting, restarting or stopping nodes, expansion adjustment, reallocation of tasks, etc.

[0192] The adjustment process specifically includes:

[0193] 1) Receiving notifications: The component adjustment execution module receives adjustment information from the configuration adjustment notification module in real time;

[0194] 2) Parsing information: Parse the key information in the notification to clarify the adjustment type, target component, execution time, etc.;

[0195] 3) Execute operations: Execute specific adjustment operations based on the parsed information. Operations may include but are not limited to starting or stopping nodes, adjusting scale, reallocating tasks, etc.;

[0196] 4) Update operation: Status update and monitoring information update After executing the adjustment operation, the adjustment execution module reports the latest status and performance data to the system. It also includes updating the data of the dynamic adjustment strategy generation module for the next round of adjustment strategy generation.

[0197] The latest data is as follows:

[0198]

[0199] The above content shows an example of the latest data reported by a component (ComputeNode-01), including the latest status (running) and performance data (CPU usage 30%, memory usage 40%). Such data can help the system better understand the operating status of each component.

[0200] The method and system based on adaptive distributed system configuration adjustment of the present invention effectively solve a series of technical problems existing in traditional configuration management methods, such as static configuration not adapting to dynamic environment, lack of real-time performance optimization, insufficient adaptability to complex application scenarios, lack of intelligent adaptability, and lack of flexibility in the configuration adjustment process. The average time from environmental change detection to configuration adjustment completion is increased by 30% compared with static configuration. In tests conducted in simulated or actual complex application scenarios, the stability of the system, such as error rate and number of crashes, is reduced by 20%. The average response time of the system in scenarios with high real-time performance requirements is reduced by 10% and the throughput is increased by 15%. The data shows that the performance before and after the implementation of dynamic adjustment is significantly improved, and significant technical effects have been achieved, which improves the performance, flexibility and intelligence level of the system.

[0201] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. An adaptive configuration adjustment method applied to a distributed system, characterized in that: The steps include: S1. Collect performance data of the distributed system to be configured; S2, analyzing the performance data and outputting analysis result data; S3. Formulate a dynamic adjustment strategy based on the analysis result data, and send a dynamic adjustment strategy notification message; The dynamic adjustment strategy includes: A. Node restart strategy: Rules: When the status of a system or component becomes unavailable or reaches the fault threshold, the health status of the system or component is detected by receiving CPU usage, memory usage, and network traffic; Execution: Trigger node restart and use the restart script to restart; B. Node startup strategy: Rules: For systems or components that have previously failed, define a startup strategy so that they can be repaired the next time they are started; Execution: trigger node startup, start using startup script; C. Node stopping strategy: Rules: When the state of a system or component becomes unavailable or reaches a failure threshold, immediately stop the current rule of the system or component, regardless of the current state; Execution: Trigger the node to stop and use the stop script to stop it; D. Extension adjustment strategy: Rule: The current system load is determined by CPU usage and memory usage. When the system load exceeds the first preset value, nodes are dynamically added; when the system load drops below the second preset value, nodes are reduced. Execution: Adjust the number of nodes; E. Task allocation strategy: Rules: Based on the execution time and resource usage, when the execution time of a task exceeds the threshold or the resource usage is high, the task is reallocated; Execution: triggers load balancing to ensure that tasks in the system are evenly distributed; S4. Sending the dynamic adjustment strategy notification information to the affected components in the distributed system, specifically including: S4-1, receiving dynamic adjustment strategy notification information; S4-2, parsing the dynamic adjustment strategy notification information to obtain the adjustment type, target component, and execution time; S4-3, notifying the component adjustment execution module of the parsed dynamic adjustment policy notification information, and the component adjustment execution module performs corresponding configuration adjustment operations on the affected components in the distributed system; S5. The affected component receives the notification information and performs corresponding configuration adjustment operations; S6. After completing the current round of configuration adjustment operations, each component reports the latest performance data to the distributed system and updates the monitoring information of the configuration center for use in the next round of configuration adjustment.

2. The adaptive configuration adjustment method applied to a distributed system according to claim 1, characterized in that: The performance data includes CPU usage, memory occupancy, and network traffic of each component in the distributed system; The CPU usage, memory occupancy, and network traffic are summarized in the form of timestamps, and each timestamp corresponds to the CPU usage, memory occupancy, and network traffic at the current timestamp; Multiple timestamps are aggregated to form a performance data table.

3. The adaptive configuration adjustment method applied to a distributed system according to claim 1, characterized in that: The step S2 of analyzing the performance data includes: Linear regression was used to analyze the collected performance data: Assume that the current goal is to predict CPU usage as the target variable, and the input features include memory usage and network traffic, and take the input features as input variables; The linear regression model is expressed as: ； Where y represents the target variable; x1, x2, ..., xn represent the input variables; b1, b2, ..., bn represent the coefficients of the input features; b0 represents the intercept; and ε represents the error.

4. The adaptive configuration adjustment method applied to a distributed system according to claim 3, characterized in that: The parsing of the performance data in step S2 further includes: formulating predefined rules; The predefined rules include: Rule 1: When the CPU usage exceeds the first threshold, a warning is issued; Rule 2: When the memory usage exceeds the second threshold, perform memory cleanup. Rule 3: When network traffic exceeds the third threshold and lasts longer than a predetermined period of time, automatic expansion of system capacity is triggered.

5. The adaptive configuration adjustment method applied to a distributed system according to claim 4, characterized in that: The parsing of the performance data in step S2 further includes: The predefined rules are executed using a rule engine to verify and match the performance data collected in real time, and output potential problems or abnormal situations.

6. The adaptive configuration adjustment method applied to a distributed system according to claim 1, characterized in that: The output analysis result data in step S2 includes a timestamp and an analysis result corresponding to the timestamp; the analysis result includes: Expand the node; Restart the node; Start the node; Stop the node; Optimize the task.

7. An adaptive configuration adjustment system, used to execute the adaptive configuration adjustment method applied to a distributed system according to any one of claims 1 to 6, characterized in that: The adaptive configuration adjustment system includes: Real-time monitoring module, used to collect performance data of the distributed system to be configured; An intelligent analysis module, used to analyze the performance data collected by the real-time monitoring module and output analysis result data; A dynamic adjustment strategy generation module, used to formulate a dynamic adjustment strategy according to the analysis result data, and send dynamic adjustment strategy notification information; A configuration adjustment notification module is used to notify the affected components in the distributed system of the dynamic adjustment strategy notification information; An adjustment execution module is used to drive the affected components to receive notification information and perform corresponding configuration adjustment operations; After completing the current round of configuration adjustment operations, each component reports the latest performance data to the distributed system and updates the monitoring information of the configuration center for use in the next round of configuration adjustment.

8. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device, the electronic device executes the adaptive configuration adjustment method applied to a distributed system according to any one of claims 1 to 6.

Images