Method and device for self-adaptive micro-service architecture

Abstract

The invention relates to the technical field of computer software and network, and particularly provides a method and device for self-adaptive micro-service architecture, and the method comprises the following steps: S1, service health examination; s2, dynamic resource adjustment; s3, an intelligent scheduling algorithm; and S4, a fault recovery mechanism. Compared with the prior art, the fault service can be quickly detected and isolated through an automatic service health check and fault recovery mechanism, and the influence of the fault on the whole system is reduced; through an intelligent scheduling algorithm, the number of service instances is dynamically adjusted according to real-time monitoring data and a prediction model, and effective utilization of resources is ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer software and network technology, and specifically provides a method and device for an adaptive microservice architecture. Background Art

[0002] With the continuous development of Internet applications, the traditional monolithic architecture can no longer meet the needs of modern applications. Microservice architecture is widely adopted due to its flexibility and scalability. However, in the microservice architecture, the management of service instances becomes more and more complicated, especially in a highly dynamic environment, where the number of service instances changes frequently, increasing the difficulty of management. Summary of the invention

[0003] The present invention aims to address the deficiencies of the above-mentioned prior art and provides a method for an adaptive microservice architecture with strong practicality.

[0004] A further technical task of the present invention is to provide a device for an adaptive microservice architecture that is reasonably designed, safe and applicable.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A method for an adaptive microservice architecture has the following steps:

[0007] S1. Service health check;

[0008] S2, dynamic resource adjustment;

[0009] S3, intelligent scheduling algorithm;

[0010] S4. Fault recovery mechanism.

[0011] Further, in step S1, Nginx is used as a load balancer, an address list of backend services is defined in the Nginx configuration file, and a health check policy is configured to regularly check the health status of the backend service instances;

[0012] Depending on the nature of the backend component business, a health check strategy is defined for each backend service, including check frequency, timeout, success and failure thresholds, and the results of each health check are recorded in a log file for subsequent decision-making and troubleshooting.

[0013] Furthermore, in step S2, when the health check finds a problem with a service instance, the traffic is automatically redirected to the healthy instance;

[0014] Dynamically adjust the number of service instances based on load conditions and forecasting models, and regularly generate service health status reports and resource adjustment reports to help operation and maintenance personnel understand the overall health of the system and overall resource status.

[0015] Furthermore, in step S3, a real-time monitoring tool is integrated to collect the status data of the service during runtime, and the ARIMA time series prediction model is used to train the intelligent scheduling model to dynamically adjust the number of service instances according to the load situation and service health status;

[0016] Among them, the ARIMA model formula is:

[0017] (1-φ1B-φ2B 2 -...-φ p B p )(1-B) d y t =c+(1+θ1B+θ2B 2 +...+θ q B q )∈ t

[0018] B is the backshift operator, defined as Byt=yt-1Byt=yt-1.

[0019] Furthermore, the ARIMA model training steps are:

[0020] (1) Collect historical monitoring data and handle missing values ​​and outliers through data cleaning;

[0021] (2) Use statistical test methods to check whether the time series is stationary.

[0022] If the column is not stationary, then difference processing is performed until the data becomes stationary;

[0023] (3) Select the order p of the autoregressive part and determine the order d of the difference, where d is between 0 and 1.

[0024] or 2, select the order q of the moving average part;

[0025] (4) Use the determined (p, d, q) parameters to construct the ARIMA model and use the maximum likelihood

[0026] The random estimation method estimates the model parameters and fits the model to the training data set;

[0027] (5) Check the white noise characteristics of the model residuals to ensure that the model has no obvious bias, and use the mean square error (MSE) or root mean square error (RMSE) indicators to evaluate the prediction performance of the model;

[0028] (6) Use the trained model to predict data for a period of time in the future. If

[0029] Longer time horizon forecasts are needed, but long-term forecasts are less accurate.

[0030] Furthermore, in step S4, it further includes:

[0031] S4.1. Error detection records all key events generated during system operation, including startup, shutdown, application startup, shutdown, and any abnormal events;

[0032] S4.2. Divide the system into independent modules, each module is responsible for a specific function;

[0033] S4.3. When a fault is detected, the system automatically restarts the faulty component or the entire system;

[0034] S4.4. Define a complete set of disaster recovery processes;

[0035] S4.5. Simulate loads under extreme conditions to check whether the system can work normally.

[0036] Furthermore, in step S4.1, the status of system components is checked regularly. In a distributed system, nodes confirm each other's survival status by sending heartbeat messages.

[0037] Furthermore, in step S4.3, the task is transferred from the failed node to a healthy node for continued execution, and the backup and log files are used to restore the lost or damaged data; in a distributed system, redundant storage of data is achieved by taking regular snapshots and replicating the data.

[0038] A device for an adaptive microservice architecture, comprising: at least one memory and at least one processor;

[0039] The at least one memory is used to store a machine-readable program;

[0040] The at least one processor is used to call the machine-readable program to execute a method for adaptive microservice architecture.

[0041] Compared with the method and device of the adaptive microservice architecture and the prior art, the method and device of the adaptive microservice architecture of the present invention has the following outstanding beneficial effects:

[0042] The present invention can quickly detect and isolate faulty services through automated service health checks and fault recovery mechanisms, reducing the impact of faults on the entire system; through intelligent scheduling algorithms, the number of service instances can be dynamically adjusted according to real-time monitoring data and prediction models to ensure efficient use of resources.

[0043] The health check mechanism ensures that only healthy service instances participate in load balancing, enhancing the security of the system.

[0044] The automated fault detection and recovery mechanism reduces the need for manual intervention and reduces the workload of operation and maintenance personnel.

[0045] The adaptive microservice architecture management system provided by the present invention not only improves the stability and availability of the system, but also reduces operation and maintenance costs, improves user experience, and has the advantages of easy expansion and maintenance, with significant economic and social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Attached Figure 1 It is a flow chart of an approach to adaptive microservice architecture;

[0048] Attached Figure 2 It is a timing diagram of an approach to adaptive microservice architecture;

[0049] Attached Figure 3 It is a network topology diagram of an adaptive microservice architecture approach. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] A best embodiment is given below:

[0052] like Figure 1-3 As shown, a method for adaptive microservice architecture in this embodiment has the following steps:

[0053] S1. Service health check;

[0054] Use Nginx as a load balancer, define the address list of the backend service in the Nginx configuration file, and configure the health check policy to regularly check the health status of the backend service instances.

[0055] According to the different business nature of the backend components, health check strategies are defined for each backend service, including parameters such as check frequency, timeout, success and failure thresholds, etc. The results of each health check are recorded in the log file for subsequent decision-making and troubleshooting.

[0056] S2, dynamic resource adjustment;

[0057] When the health check finds a problem with a service instance, traffic is automatically redirected to the healthy instance.

[0058] Dynamically adjust the number of service instances based on load conditions and prediction models, for example, increase the number of instances when the load is high and reduce the number of instances when the load is low.

[0059] Regularly generate service health status reports and resource adjustment reports to help operation and maintenance personnel understand the overall health and overall resource status of the system.

[0060] S3, intelligent scheduling algorithm;

[0061] By integrating real-time monitoring tools (such as Prometheus, Grafana, etc.), the status data of the service runtime is collected.

[0062] Use the ARIMA time series prediction model to train the intelligent scheduling model and dynamically adjust the number of service instances based on load conditions and service health;

[0063] The ARIMA model formula is:

[0064] (1-φ1B-φ2B 2 -...-φ p B p )(1-B d y t =c+(1+θ1B+θ2B 2 +...+θ q B q )∈ t

[0065] B is the backshift operator, defined as Byt=yt-1Byt=yt-1.

[0066] The ARIMA model training steps are:

[0067] (1) Collect historical monitoring data, such as CPU utilization and memory usage, and handle missing values ​​and outliers through data cleaning;

[0068] (2) Use statistical test methods (such as ADF test) to check whether the time series is flat.

[0069] If the time series is not stationary, then difference processing is performed until the data becomes stationary;

[0070] (3) Select the order p of the autoregressive part, determine the order d of the difference, d is between 0, 1 or 2, and select the order q of the moving average part;

[0071] (4) Use the determined (p, d, q) parameters to build an ARIMA model, estimate the model parameters using methods such as maximum likelihood estimation, and fit the model to the training data set;

[0072] (5) Check the white noise characteristics of the model residuals to ensure that the model has no obvious bias, and use indicators such as mean square error (MSE) or root mean square error (RMSE) to evaluate the prediction performance of the model;

[0073] (6) Use the trained model to predict data for a period of time in the future. If necessary, make predictions for a longer time range, but the accuracy of long-term predictions is lower.

[0074] Simple example:

[0075] # Load data

[0076] data=pd.read_csv('your_data.csv',parse_dates=['timestamp'],index_col='timestamp')

[0077] #Data preprocessing

[0078] data['value']=data['value'].fillna(method='ffill')#Fill missing values

[0079] #Stability test

[0080] result=adfuller(data['value'])

[0081] print('ADF Statistic:',result[0])

[0082] print('p-value:',result[1])

[0083] #Differential processing

[0084] data_diff=data.diff().dropna()

[0085] #Parameter determination

[0086] p=1#autoregressive order

[0087] d=1# difference order

[0088] q=1#moving average order

[0089] #Model training

[0090] model=ARIMA(data['value'],order=(p,d,q))

[0091] model_fit = model.fit()

[0092] #Model Diagnosis

[0093] residuals=pd.DataFrame(model_fit.resid)

[0094] residuals.plot()

[0095] plt.show()

[0096] print(residuals.describe())

[0097] #Model evaluation

[0098] predictions=model_fit.predict(start=100,end=len(data)-1)

[0099] mse=((predictions-data['value'][100:])**2).mean()

[0100] print('MSE:',mse)

[0101] #predict

[0102] forecast=model_fit.forecast(steps=10)

[0103] print('Forecast:',forecast)

[0104] S4, fault recovery mechanism;

[0105] Further including:

[0106] S4.1. Error detection records all key events generated during system operation, including system startup, shutdown, application startup, shutdown, and any abnormal events. Regularly check the status of system components, such as CPU usage, memory usage, disk space, etc. In a distributed system, nodes confirm each other's survival status by sending heartbeat messages.

[0107] S4.2. Divide the system into independent modules, each responsible for a specific function, so that even if a module fails, it will not affect other parts. Use redundant components to replace failed parts, such as dual power supplies, hot-swappable hard drives, etc. At the software level, partitioning technology can be used to isolate faults to prevent problems in one area from spreading to the entire system.

[0108] S4.3. When a fault is detected, the system automatically restarts the faulty component or the entire system; transfers tasks from the faulty node to a healthy node for continued execution, and uses backup and log files to recover lost or damaged data; in a distributed system, redundant storage of data is achieved by taking regular snapshots and replicating data.

[0109] S4.4. Define a complete disaster recovery process, including backup strategy, recovery point objective (RPO), recovery time objective (RTO), etc.

[0110] S4.5. Simulate loads under extreme conditions to check whether the system can work normally; intentionally introduce faults into the system to test the system's response capabilities.

[0111] Based on the above method, a device for an adaptive microservice architecture in this embodiment includes: at least one memory and at least one processor;

[0112] The at least one memory is used to store a machine-readable program;

[0113] The at least one processor is used to call the machine-readable program to execute a method for adaptive microservice architecture.

[0114] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0115] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for adaptive microservice architecture, characterized in that: The steps are as follows: S1. Service health check; S2, dynamic resource adjustment; S3, intelligent scheduling algorithm; S4. Fault recovery mechanism.

2. The method of an adaptive microservice architecture according to claim 1, characterized in that: In step S1, use Nginx as a load balancer, define the address list of the backend service in the Nginx configuration file, and configure the health check policy to regularly check the health status of the backend service instance; Depending on the nature of the backend component business, a health check strategy is defined for each backend service, including check frequency, timeout, success and failure thresholds, and the results of each health check are recorded in a log file for subsequent decision-making and troubleshooting.

3. The method of an adaptive microservice architecture according to claim 2, characterized in that: In step S2, when the health check finds a problem with a service instance, the traffic is automatically redirected to the healthy instance; Dynamically adjust the number of service instances based on load conditions and forecasting models, and regularly generate service health status reports and resource adjustment reports to help operation and maintenance personnel understand the overall health of the system and overall resource status.

4. The method of an adaptive microservice architecture according to claim 3, characterized in that: In step S3, real-time monitoring tools are integrated to collect status data of the service during runtime, and the ARIMA time series prediction model is used to train the intelligent scheduling model to dynamically adjust the number of service instances according to the load and service health status; Among them, the ARIMA model formula is: (1-φ1B-φ2B2-…-φ p B p (1-B) d y t =c+(1+θ1B+θ2B 2 +…+θ q B q )∈ t B is the backshift operator, defined as Byt=yt-1Byt=yt-1.

5. The method of an adaptive microservice architecture according to claim 4, characterized in that: The ARIMA model training steps are: (1) Collect historical monitoring data and handle missing values ​​and outliers through data cleaning; (2) Use statistical test methods to check whether the time series is stationary. If the time series is not stationary, perform difference processing until the data becomes stationary; (3) Select the order p of the autoregressive part, determine the order d of the difference, d is between 0, 1 or 2, and select the order q of the moving average part; (4) Use the determined (p, d, q) parameters to build an ARIMA model, estimate the model parameters using the maximum likelihood estimation method, and fit the model to the training data set; (5) Check the white noise characteristics of the model residuals to ensure that the model has no obvious bias, and use the mean square error (MSE) or root mean square error (RMSE) indicators to evaluate the prediction performance of the model; (6) Use the trained model to predict data for a period of time in the future. If necessary, make predictions for a longer time range, but the accuracy of long-term predictions is lower.

6. The method of an adaptive microservice architecture according to claim 5, characterized in that: In step S4, further comprising: S4.

1. Error detection records all key events generated during system operation, including startup, shutdown, application startup, shutdown, and any abnormal events; S4.

2. Divide the system into independent modules, each module is responsible for a specific function; S4.

3. When a fault is detected, the system automatically restarts the faulty component or the entire system; S4.

4. Define a complete set of disaster recovery processes; S4.

5. Simulate loads under extreme conditions to check whether the system can work normally.

7. The method of an adaptive microservice architecture according to claim 5, characterized in that: In step S4.1, the status of system components is checked regularly. In a distributed system, nodes confirm each other's survival status by sending heartbeat messages.

8. The method of an adaptive microservice architecture according to claim 5, characterized in that: In step S4.3, the task is transferred from the failed node to a healthy node for continued execution, and the backup and log files are used to recover the lost or damaged data; in a distributed system, redundant storage of data is achieved by taking regular snapshots and replicating the data.

9. A device for an adaptive microservice architecture, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 8.

Images