Method for realizing service migration by utilizing similarity algorithm based on ARMA

Abstract

The invention relates to a method for realizing service migration by utilizing a similarity algorithm based on ARMA, the similarity algorithm is established by analyzing historical operation data of each server, and the calculation result of the similarity is used as the basis for selecting a source server, the basis for service migration, the basis for selecting a target server and the basis for judging whether migration is reasonable or not. According to the method and the device, the problems that service migration is uncontrollable due to lack of data support and unreliable due to dependence on artificial experience are solved, the resource utilization efficiency of each server can be remarkably improved, the service migration can be standardized, programmed and materialized in experience, and server resources are reasonably utilized.

Description

Technical Field

[0001] The present application belongs to the technical field of server load optimization, and specifically relates to a method for realizing service migration based on ARMA using a similarity algorithm. Background Art

[0002] Many small and medium-sized enterprises lack a unified server resource management platform when running software services. The general technical solution is that operation and maintenance personnel manually monitor, deploy, and allocate server resources to each server based on experience. When server resources are limited, operation and maintenance personnel need to deploy different types of services in a mixed manner, which leads to resource competition between servers when multiple services are mixed and deployed on the same server. When the above phenomenon occurs, operation and maintenance personnel need to achieve a series of steps such as accurate optimization, manual monitoring, and estimated resources to ensure the accurate operation of the service. Furthermore, when a bottleneck occurs in server resources and service migration is required, operation and maintenance personnel also need to view the real-time status of the server through server instructions to select the target server.

[0003] The existing service migration methods have two main deficiencies: (1) The selection of the target server is accidental, that is, the selection of the target server lacks actual data support; (2) After the service migration, it still relies on the experience of the operation and maintenance personnel for management, and the hidden dangers before the migration are not eliminated. Summary of the invention

[0004] The present invention provides a method for realizing service migration by using a similarity algorithm based on ARMA, which solves the problem that service migration lacks data support and is uncontrollable, and relies on manual experience and is unreliable.

[0005] The technical solution of the present application is as follows: A method for implementing service migration based on ARMA using a similarity algorithm, comprising the following steps: S1) collecting the load data of the running servers and the load data of the services they run in a time series and at a fixed frequency; S2) Taking the server load data and demand configuration of the same period in any two collection days as input, a similarity algorithm is used to output the similarity between each server and the demand configuration in the application scenario, and the server with the highest similarity is selected as the source server for service migration; The similarity algorithm model in this step is: ; In the formula, L i This is the server load data for the first collection day; L j The server load data on the second collection day; Ui , U j They are L i , L j Demand configuration for the time period; C is the product coefficient; D is a constant coefficient; in, Represents the core attributes of server load data from two collection days; Represents the core attributes of the demand configuration in two collection days; Represents all attributes of server load data from two collection days; Represents all properties of the demand configuration in two collection days; S3) using the service load data of the source server and the source server load data on the two collection days selected in S2) as input, using a similarity algorithm to output the similarity between the source server and each service, and selecting the service with the highest similarity as the migration service; The similarity algorithm model in this step is: ; In the formula, A i This is the service load data for the first collection day; A j The service load data on the second collection day; B i , B j They are A i , A j Source server load data for the time period; E is the product coefficient; F is a constant coefficient; S4) After the source server and the service to be migrated are determined, the other servers are the target servers; the remaining configuration of each target server and the load data of the service to be migrated are used as input, and the similarity between each target server and the service to be migrated is output using a similarity algorithm, and the target server with the highest similarity is selected as the migration server; The similarity algorithm model in this step is: ; In the formula, M i The remaining configuration of the target server for the first collection day; M j The remaining configuration of the target server on the second acquisition day; S i、S j For M i , M j The corresponding load data of the service to be migrated, G is the product coefficient; H is a constant coefficient; S5) inputting the service load data into the ARMA time series prediction model, and outputting the service prediction load data; inputting the remaining configuration data of the migration server into the ARMA time series prediction model, and outputting the remaining configuration prediction data.

[0006] Taking the predicted load data and the remaining configuration predicted data as input, a similarity algorithm is used to output the similarity between the predicted load data and the remaining configuration predicted data, and judging whether the migration server can accommodate the migration service according to the similarity result; The similarity algorithm model in this step is: ; In the formula, T i For the future i The remaining configuration prediction data of the target server for the day; T j For the future j The remaining configuration prediction data of the target server for the day; R i For the future of the migration service i forecast load data for the day; R j For the future of the migration service j forecast load data for the day; i ≠ j ; P is the product coefficient; Q is a constant coefficient.

[0007] Furthermore, in step S1), the server load data and the service load data both include memory consumption, disk consumption, CPU load consumption and network bandwidth consumption.

[0008] Furthermore, in step S2), the required configuration includes the memory configuration, CPU configuration, disk IO configuration and network configuration of the server.

[0009] Furthermore, in step S5), the ARMA time series prediction model is as follows: y_t = c + φ 1 *y_(t- 1 ) + φ 2 *y_(t- 2 ) + ... + φ_p*y_(tp) + ε_t + θ 1 *e_ (t- 1) + i 2 *e_(t- 2 ) + ... + θ_q*ε_(tq) ; In the formula, t Indicates time; y_t For time t The observed value of; c is a constant term; f 1 ,f 2 ,…,φ_p is the autoregression coefficient, indicating t- 1, t -2, ..., t - p time y Value Pair t time y Linear influence of value; e _ t It's time t The error term of i 1, i 2, ..., i _ q is the moving average coefficient, indicating t -1, t -2,…, t -q time error t time y The impact of value.

[0010] Due to the adoption of the above technical solution, the beneficial effects of this application are as follows: 1. This application provides data support for each step in the service migration process, which can significantly improve the resource utilization efficiency of each server and enable service migration to be standardized, proceduralized, and empirically materialized.

[0011] 2. This application can significantly improve the controllability of services after migration to other servers. Since the server resource utilization rate fluctuates over time, this application can flexibly adjust the running services in the server to achieve service control, which not only helps to ensure the normal operation of the service, but also alleviates the unreasonable utilization of server resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a method for realizing service migration based on ARMA using a similarity algorithm provided in this application; Figure 2Training parameters for the ARMA time series forecasting model p and q Flowchart of the process. DETAILED DESCRIPTION

[0013] Based on the background technology, the present application provides a method for implementing service migration based on ARMA using a similarity algorithm, including the following steps: S1) Collecting the load data of the running servers and the load data of the services they run in a time series at a fixed frequency.

[0014] Multiple services can be run simultaneously in a server, such as service A, service B, etc. The running of services will consume the disk configuration, network configuration, memory configuration, and CPU configuration of the server. In this embodiment, the server load data and service load data are collected at a frequency of 1 hour, the data is matrixed, and stored in the database. When storing, it is stored in a relational data format. In step S1), the server load data and service load data both include memory consumption, disk consumption, CPU load consumption, and network bandwidth consumption.

[0015] S2) Taking the server load data and demand configuration of the same period in any two collection days as input, a similarity algorithm is used to output the similarity between each server and the demand configuration in the application scenario, and the server with the highest similarity is selected as the source server for service migration.

[0016] The similarity algorithm model in this step is: ; In the formula, L i This is the server load data for the first collection day; L j The server load data on the second collection day; U i , U j They are L i , L j Demand configuration for the time period; C is the product coefficient; D is a constant coefficient; The product coefficient and constant coefficient are adjusted during the calculation process according to the server migration priority and service importance to adapt to the application scenario. The algorithm model performs division, multiplication, and addition operations from left to right. The parameters C The adjustment ratio is higher, and the parameters DThe functions of the following product coefficients and constant coefficients are the same as those in this model. Different parameters are used to distinguish the models.

[0017] in, Represents the core attributes of server load data from two collection days; Represents the core attributes of the demand configuration in two collection days; Represents all attributes of server load data from two collection days; Represents all properties of the demand configuration for two collection days.

[0018] It should be noted that the two collection days are not limited to two consecutive days. The two collection days are determined according to the actual application scenario, and can be the day when the method is applied and a day before, such as two consecutive Saturdays.

[0019] In this embodiment, the matrix form of the collected server load data is as follows: ; The matrix form of the demand configuration is as follows: ; In the above two matrices, each column from left to right represents the memory load / configuration, CPU load / configuration, disk IO load / configuration, and network load / configuration. The similarity result range is 0-1. The closer the result is to 1, the higher the load. The similarity is used as the basis for the source server.

[0020] S3) Taking the service load data of the source server and the source server load data on any two collection days as input, a similarity algorithm is used to output the similarity between the source server and each service, and the service with the highest similarity is selected as the migration service; The similarity algorithm model in this step is: ; In the formula, A i This is the service load data for the first collection day; A j The service load data on the second collection day; B i , B j They are A i , A j Source server load data for the time period; E is the product coefficient; F is a constant coefficient.

[0021] The similarity algorithm model principle of this step is the same as that of the similarity algorithm model in step S2), and the intersection and union calculations therein still represent the extraction of the core attributes and all attributes between the two input data. Similarly, the similarity algorithm model principles in the following steps S4) and S5) are the same as those of this step.

[0022] The data collected on any two collection days in S2) and S3) can be the same or different from the collection day of S3). In general applications, the number of collection days is much greater than two, but the similarity is calculated by comparing two by two. Therefore, when there are multiple collection days, there will be multiple collection day combinations, and the final similarity result will also be in matrix form. In other words, S1)~S5) are repeated and rolled, and all combinations will always be obtained, and then the goal of each step will be achieved based on the similarity results.

[0023] S4) After the source server is determined, other servers are the target servers; the remaining configuration and required configuration of each target server are used as input, and the similarity between the remaining configuration and the configuration requirements of each target server is output using a similarity algorithm, and the target server with the highest similarity is selected as the migration server; The similarity algorithm model in this step is: ; In the formula, M i The remaining configuration of the target server for the first collection day; M j The remaining configuration of the target server on the second acquisition day; S i 、S j For M i , M j The corresponding load data of the service to be migrated, G is the product coefficient; H is a constant coefficient; The remaining quota of the server can be calculated or directly read by combining the data collected in S1) with the server configuration data. It should be noted that there can be multiple source servers, and the specific number can be determined by setting a similarity threshold.

[0024] S5) inputting the service load data into the ARMA time series forecasting model and outputting the service forecast load data; Taking the predicted load data and the remaining configuration of the migration server as input, a similarity algorithm is used to output the similarity between the predicted load data and the migration server, and whether the migration server can accommodate the migration service is determined based on the similarity result; The similarity algorithm model in this step is: ; In the formula, T i For the future i The remaining configuration prediction data of the target server for the day; T j For the future j The remaining configuration prediction data of the target server for the day; R i For the future of the migration service i forecast load data for the day; R j For the future of the migration service j forecast load data for the day; i ≠ j ; P is the product coefficient; Q is a constant coefficient.

[0025] The ARMA time series forecasting model is as follows: y_t = c + φ 1 *y_(t- 1 ) + φ 2 *y_(t- 2 ) + ... + φ_p*y_(tp) + ε_t + θ 1 *e_ (t- 1 ) + i 2 *e_(t- 2 ) + ... + θ_q*ε_(tq) ; In the formula, t Indicates time; y_t For time t The observed value of; c is a constant term; f 1 ,f 2 ,…,φ_p is the autoregression coefficient, indicating t- 1, t -2, ..., t - p time y Value Pair t time y Linear influence of value; e _ t It's time t The error term of i 1, i 2, ..., i _ q is the moving average coefficient, indicating t -1, t -2,…, t -q time error t timey The impact of value.

[0026] The ARMA model is suitable for stationary time series data and can predict the load at future time points. In the ARMA model, the key step is to determine p and q The specific value of can be obtained by analyzing the load data matrix using ACF and PACF. For the specific process, see the attached Figure 2 .

[0027] In this embodiment, p The value of is finally determined to be 2. q The value of is finally determined to be 1, and then other parameters are determined. The ARMA prediction method is a widely used sequence prediction model and can be implemented through existing technologies.

[0028] In this step, the purpose of predicting the service load data in the future is to verify whether the migrated service can run normally in the target server, that is, the similarity is used as the basis for judging whether the service is running normally. The higher the similarity, the better the adaptability between the service and the target server.

[0029] Anything not described in this application can be achieved by adopting or drawing on existing technologies. The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for realizing service migration using a similarity algorithm based on ARMA, characterized in that: The following steps are involved: S1) collecting the load data of the running servers and the load data of the services they run in a time series and at a fixed frequency; S2) Taking the server load data and demand configuration of the same period in any two collection days as input, a similarity algorithm is used to output the similarity between each server and the demand configuration in the application scenario, and the server with the highest similarity is selected as the source server for service migration; The similarity algorithm model in this step is: ； In the formula, L i This is the server load data for the first collection day; L j The server load data on the second collection day; U i , U j They are L i , L j Demand configuration for the time period; C is the product coefficient; D is a constant coefficient; in, Represents the core attributes of server load data from two collection days; Represents the core attributes of the demand configuration in two collection days; Represents all attributes of server load data from two collection days; Represents all properties of the demand configuration in two collection days; S3) Taking the service load data of the source server and the source server load data on any two collection days as input, a similarity algorithm is used to output the similarity between the source server and each service, and the service with the highest similarity is selected as the migration service; The similarity algorithm model in this step is: ； In the formula, A i This is the service load data for the first collection day; A j The service load data on the second collection day; B i , B j They are A i , A j Source server load data for the time period; E is the product coefficient; F is a constant coefficient; S4) After the source server and the service to be migrated are determined, the other servers are the target servers; the remaining configuration of each target server and the load data of the service to be migrated are used as input, and the similarity between each target server and the service to be migrated is output using a similarity algorithm, and the target server with the highest similarity is selected as the migration server; The similarity algorithm model in this step is: ； In the formula, M i The remaining configuration of the target server for the first collection day; M j The remaining configuration of the target server on the second acquisition day; S i 、S j For M i , M j The corresponding load data of the service to be migrated, G is the product coefficient; H is a constant coefficient; S5) inputting the service load data into the ARMA time series prediction model, and outputting the service prediction load data; inputting the remaining configuration data of the migration server into the ARMA time series prediction model, and outputting the remaining configuration prediction data; Taking the predicted load data and the remaining configuration predicted data as input, a similarity algorithm is used to output the similarity between the predicted load data and the remaining configuration predicted data, and judging whether the migration server can accommodate the migration service according to the similarity result; The similarity algorithm model in this step is: ； In the formula, T i For the future i The remaining configuration prediction data of the target server for the day; T j For the future j The remaining configuration prediction data of the target server for the day; R i For the future of the migration service i forecast load data for the day; R j For the future of the migration service j forecast load data for the day; i ≠ j ; P is the product coefficient; Q is a constant coefficient.

2. According to claim 1, a method for realizing service migration using a similarity algorithm based on ARMA is characterized in that: In step S1), the server load data and service load data both include memory consumption, disk consumption, CPU load consumption and network bandwidth consumption.

3. The method for realizing service migration based on ARMA and using similarity algorithm according to claim 1, characterized in that: In step S2), the required configuration includes the memory configuration, CPU configuration, disk IO configuration and network configuration of the server.

4. The method for realizing service migration based on ARMA and using similarity algorithm according to claim 1, characterized in that: In step S5), the ARMA time series forecasting model is as follows: y_t = c + φ 1 *y_(t- 1 ) + φ 2 *y_(t- 2 ) + ... + φ_p*y_(tp) + ε_t + θ 1 *ε_(t- 1 ) + θ 2 *ε_(t- 2 ) + ... + θ_q*ε_(tq) ； In the formula, t Indicates time; y_t For time t The observed value of; c is a constant term; φ 1 ,φ 2 ,…,φ_p is the autoregression coefficient, indicating t- 1, t -2, ..., t - p time y Value Pair t time y Linear influence of value; ε _ t It's time t The error term of θ 1, θ 2, ..., θ _ q is the moving average coefficient, indicating t -1, t -2,…, t -q time error t time y The impact of value.

Images