A remote automated operation and maintenance method and system for a big data storage server

Abstract

The present invention discloses a remote automated operation and maintenance method and system for a big data storage server, which relates to the field of automated operation and maintenance technology, and includes: determining a list of servers to be operated and maintained, and obtaining login key information of a target server, determining the corresponding target server using the login key information, obtaining a login fluctuation coefficient matrix, monitoring the target server, obtaining a performance fluctuation index matrix, and setting automated configuration information matching the target server based on the performance fluctuation index matrix, analyzing the performance of the target server under different conditions through the performance fluctuation index matrix, determining whether the server configuration needs to be adjusted, avoiding configuration failure or configuration mismatch due to different performance of the system or hardware of the big data storage server and different resource utilization of the same system or hardware, and more effectively allocating and managing server resources to avoid resource waste.

Description

Technical Field

[0001] The present invention relates to the field of automated operation and maintenance technology, and in particular to a remote automated operation and maintenance method and system for a big data storage server. Background Art

[0002] The remote automated operation and maintenance method and system of big data storage servers are designed to improve the efficiency and reliability of large-scale data processing. In the context of rapid growth in data volume, traditional manual operation and maintenance methods are difficult to meet the requirements of efficiency and real-time performance. Remote automated operation and maintenance can reduce human errors, improve system stability, and reduce operation and maintenance costs through automated monitoring and fault handling. The application of this system helps ensure the continuous and stable operation of the big data environment, which is of great significance to enterprise decision-making and business development.

[0003] However, the system or hardware performance of existing big data storage servers is different, and the resource utilization of the same system or hardware is different. When the automated operation and maintenance system is used to configure and manage them, configuration failure or configuration incompatibility may occur, resulting in reduced operation and maintenance efficiency of the automated operation and maintenance system or waste of resources. Therefore, it is necessary to provide a remote automated operation and maintenance method and system for big data storage servers to solve the above-mentioned problems. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, a remote automated operation and maintenance method and system for a big data storage server are provided. The technical solution solves the problem that the existing big data storage servers proposed in the above-mentioned background technology have different system or hardware performance, and the resource utilization of the same system or hardware is different. When the automated operation and maintenance system is used to configure and manage them, configuration failure or configuration incompatibility may occur, resulting in reduced operation and maintenance efficiency of the automated operation and maintenance system, or causing waste of resources.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A remote automated operation and maintenance method for a big data storage server, comprising:

[0007] Determine the list of servers to be maintained and obtain the login key information of the target server;

[0008] Use the login key information to determine the corresponding target server, log in and connect to the target server, and obtain the login fluctuation coefficient matrix;

[0009] Refer to the login fluctuation coefficient matrix to optimize the login key information and strengthen the connection with the target server;

[0010] Send the highest authority acquisition instruction to the target server to obtain the highest authority;

[0011] Monitor the target server, obtain a performance fluctuation index matrix, and set automatic configuration information matching the target server based on the performance fluctuation index matrix;

[0012] Configure the target server based on the automated configuration information, and obtain the risk assessment index of the target server in real time during the configuration process;

[0013] Optimize the automated configuration information based on the risk assessment index and complete the configuration of the target server;

[0014] After the configuration of the target server is completed, an operation and maintenance request is sent to the target server, and the operation and maintenance of the target server is performed;

[0015] Using the automation configuration information of the target server, an automation configuration information base is established, and an automation configuration script template is created through the automation configuration information base;

[0016] The servers to be operated and maintained in the server list information are configured in sequence based on the automated configuration script template, and operation and maintenance requests are sent to the servers to be operated and maintained in the server list information, so as to implement operation and maintenance on the servers to be operated and maintained in the server list information.

[0017] In an optional embodiment, the method of determining the corresponding target server by using the login key information, logging in and connecting to the target server, and obtaining a login fluctuation coefficient matrix specifically includes:

[0018] Obtaining information of the server to be operated and maintained and key information, and establishing an information base of the server to be operated and maintained and a key information base, wherein the key information includes key pair information, and the key pair information consists of a public key and a private key;

[0019] Extract key pair information from the key information in the key information repository, and randomly distribute the public key contained in the key pair information to the relevant files of the server to be operated and maintained in the server to be operated and maintained information repository;

[0020] Re-sort the server information base and key information base to be maintained, and determine the list of servers to be maintained;

[0021] Based on the information of the list of servers to be maintained, the server to be maintained that is ranked first in the list of servers to be maintained is used as the target server;

[0022] Based on the key information, determining login key information corresponding to the target server;

[0023] According to the login key information corresponding to the target server, a private key corresponding to the target server is obtained;

[0024] Use the private key corresponding to the target server to connect to the target server and obtain the connection status information;

[0025] Extracting key connection information from the connection status information, and using the key connection information to establish a login fluctuation coefficient matrix;

[0026] Among them, the expression formula of the login fluctuation coefficient matrix is:

[0027] ( );

[0028] In the formula, is the expression symbol of the login volatility matrix, It is the quantitative value of whether the connection is successful or not. is the quantitative value of the authentication status, is the server response time value, is the network connection quality index;

[0029] The calculation formula of the network connection quality index is:

[0030]

[0031] In the formula, is the network connection quality index, is the actual network delay, is the ideal value of network delay, is the actual bandwidth, is the ideal value of bandwidth, is the actual packet loss rate, is the ideal value of packet loss rate, For the actual jitter, is the ideal value of jitter, , , and are the weights of delay, bandwidth, packet loss rate and jitter, respectively. and is the exponential decay coefficient, which adjusts the decay speed of the impact factor. is the impact index of packet loss on network quality, It is the combined impact index of delay and jitter.

[0032] In an optional embodiment, the step of tuning the login key information with reference to the login fluctuation coefficient matrix to strengthen the connection with the target server specifically includes:

[0033] The login key information is optimized by using the quantized value of connection success or failure and the quantized value of authentication status in the login fluctuation coefficient matrix as the first tuning reference value of the login key information;

[0034] The login key information is tuned using the server response time value as a second tuning reference value of the login key information;

[0035] The login key information is optimized by using the network connection quality index as a third optimization reference value of the login key information;

[0036] Integrate the first tuning reference value, the second tuning reference value, and the third tuning reference value to obtain a login key tuning parameter matrix;

[0037] The corresponding login key information is tuned using the login key tuning parameter matrix.

[0038] In an optional embodiment, monitoring the target server, obtaining a performance fluctuation index matrix, and setting automatic configuration information matching the target server based on the performance fluctuation index matrix specifically includes:

[0039] Through monitoring, the target server can be obtained in real time from 0 to performance data at each moment, the performance data including a server system evaluation index, a server response time comprehensive index, and a server load index;

[0040] Performing statistical analysis on the server system evaluation index, the server response time comprehensive index and the server load index of the target server respectively to obtain a first mean value, a second mean value and a third mean value;

[0041] Taking the first mean, the second mean and the third mean as the standard, the volatility corresponding to the server system evaluation index, the server response time comprehensive index and the server load index is obtained, and a performance volatility index matrix is ​​established;

[0042] The calculation formula of the server system evaluation index is:

[0043] ;

[0044] In the formula, For the target server The server system evaluation index at the moment, For the target server The real-time network delay value of the system at the moment, For the target server The system load value at the moment, For the target server The configuration optimization value at the moment, For the target server CPU performance value at the moment, For the target server The memory capacity value at the moment, For the target server Disk IO value at the moment, For the target server The comprehensive index of the system version at the moment, , , and are weight coefficients;

[0045] The calculation formula of the server load index is:

[0046] ;

[0047] In the formula, For the target server The server load index at the moment, For the target server The actual CPU usage at the moment, is the minimum value of CPU usage, is the maximum value of CPU usage. For the target server The actual memory utilization at the moment, is the minimum memory utilization, is the maximum value of memory utilization, For the target server The actual disk IO at the moment, is the minimum value of disk IO, is the maximum value of disk IO. For the target server The actual network traffic at the moment, is the minimum value of network traffic, is the maximum value of network traffic, , , and They are the weights of CPU usage, memory utilization, disk IO, and network traffic, respectively. is the exponential decay coefficient, To monitor the time;

[0048] The expression formula of the performance fluctuation index matrix is:

[0049] ;

[0050] In the formula, is the first configuration reference value, is the second configuration reference value, is the third configuration reference value, From 0 to The average value of the server system evaluation index at time, From 0 to The average value of the comprehensive index of server response time at time, From 0 to The average server load index at the time.

[0051] In an optional embodiment, the step of optimizing the automated configuration information based on the risk assessment index and completing the configuration of the target server specifically includes:

[0052] Determine basic configuration information of the target server according to the first configuration reference value, the second configuration reference value, and the third configuration reference value in the performance fluctuation index matrix;

[0053] Determine configuration script compatibility information based on basic configuration information of the target server;

[0054] Determine configuration item information matching the target server based on the configuration script compatibility information, and obtain automated configuration information;

[0055] The target server is configured step by step based on the automated configuration information, and the performance fluctuation index matrix of the target server after each configuration step is obtained, and the historical configuration information of the target server before each configuration step is recorded;

[0056] Evaluate the performance of the target server based on the performance fluctuation index matrix, obtain the risk assessment index of the target server, and determine the risk assessment index threshold according to the performance fluctuation index matrix;

[0057] According to the risk assessment index threshold, determine whether the risk assessment index of the target server exceeds the threshold. If not, the target server is in a normal state, and the target server is configured further based on the automated configuration information. If yes, the target server is in an abnormal state, and the state of the target server can be rolled back based on the historical configuration information.

[0058] After rolling back the state of the target server, the risk assessment index and its corresponding performance fluctuation index matrix are used to trace the automation configuration information and optimize the automation configuration;

[0059] The calculation formula of the risk assessment index is:

[0060] ;

[0061] In the formula, is the risk assessment index, is the weight coefficient of the risk assessment index, is the error term.

[0062] In an optional embodiment, the use of the automatic configuration information of the target server to establish an automatic configuration information library and automatically configure the script template specifically includes:

[0063] By using the risk assessment index when the target server is configured and its corresponding performance fluctuation index matrix, the automation configuration information that makes the target server state abnormal is traced back, and the abnormal automation configuration information and its corresponding performance fluctuation index matrix are obtained;

[0064] Based on the abnormal automation configuration information and its corresponding performance fluctuation index matrix, an automation configuration information library is established;

[0065] Collect historical configuration file information and configuration specification information, merge the data, obtain script generation basic data, and import the script generation basic data into the automated configuration information library;

[0066] Clean the basic data generated by the scripts in the automated configuration information library and standardize the cleaned data;

[0067] Extract important features from the script-generated basic data after standardization to obtain relevant feature information;

[0068] Using the random forest method, the regression model is trained based on the standardized script generation basic data and related feature information to obtain an automated configuration script generation model;

[0069] Generate an automated configuration script template through an automated configuration script generation model, and update an automated configuration information base when the automated configuration script template sequentially configures the servers to be operated and maintained in the server list information to be operated and maintained;

[0070] Optimize the automation configuration script templates using the updated automation configuration repository.

[0071] Furthermore, a remote automated operation and maintenance system for a big data storage server is proposed, which is used to implement any of the above operation and maintenance methods, including:

[0072] A main control module, which is used to receive data transmitted by the configuration module, the operation and maintenance module, and the display module through the transmission module, and to process the received data, and at the same time to evaluate various information and indexes that appear in the data processing process, and to control the configuration module, the operation and maintenance module, the transmission module, and the display module;

[0073] A configuration module, the configuration module is used to configure the target server in the list of servers to be operated and maintained, and to configure the login key information of the target server;

[0074] An operation and maintenance module, which is used to monitor and operate the target server in the list of servers to be operated and maintained, and issue an alarm when the target server is in an abnormal state;

[0075] A transmission module, which is used to transmit data and information between the main control module, the configuration module, the operation and maintenance module and the display module;

[0076] The display module is used to display the data processing results of the main control module, the configuration process and results of the configuration module, and the operation and maintenance process and results of the operation and maintenance module.

[0077] In an optional embodiment, the main control module includes:

[0078] The data receiving unit is used to determine the list of servers to be operated and maintained, and obtain the login key information of the target server, and is used to receive the login fluctuation coefficient matrix, the performance fluctuation index matrix and the risk assessment index, and receive the target server from 0 to Performance data at the moment, server system evaluation index, server response time comprehensive index and server load index;

[0079] A data processing unit, the data processing unit is used to obtain information about servers to be operated and maintained and key information, establish an information base of servers to be operated and maintained and a key information base, perform statistical analysis on a server system evaluation index, a server response time comprehensive index, and a server load index of a target server, respectively, obtain a first mean, a second mean, and a third mean, optimize automatic configuration information based on a risk assessment index, and determine a risk assessment index threshold according to a performance fluctuation index matrix;

[0080] An evaluation unit, the evaluation unit is used to evaluate the performance of the target server based on the performance fluctuation index matrix, obtain the risk assessment index of the target server, and determine whether the risk assessment index of the target server exceeds the threshold according to the risk assessment index threshold;

[0081] A control unit, wherein the control unit is used to control the configuration module, the operation and maintenance module, the transmission module and the display module.

[0082] In an optional embodiment, the configuration module includes:

[0083] A server configuration unit, the server configuration unit is used to send a highest authority acquisition instruction to a target server, obtain the highest authority, set automatic configuration information matching the target server based on a performance fluctuation index matrix, configure the target server based on the automatic configuration information, and complete the configuration of the target server, and sequentially configure the servers to be operated and maintained in the list information of servers to be operated and maintained based on an automatic configuration script template;

[0084] A key configuration unit is used to determine the corresponding target server using the login key information, log in and connect to the target server, tune the login key information with reference to the login fluctuation coefficient matrix, extract the key pair information of the key information in the key information library, and randomly distribute the public key contained in the key pair information to the relevant files of the server to be operated and maintained in the server information library to be operated and maintained.

[0085] In an optional embodiment, the operation and maintenance module includes:

[0086] The monitoring unit is used to obtain the login fluctuation coefficient matrix, monitor the target server, obtain the performance fluctuation index matrix, obtain the risk assessment index of the target server in real time during the configuration process, and obtain the target server in real time through monitoring. Performance data at all times;

[0087] The operation and maintenance unit is used to send an operation and maintenance request to the target server after the configuration of the target server is completed, and perform operation and maintenance on the target server, send an operation and maintenance request to the server to be operated and maintained in the list of servers to be operated and maintained, and perform operation and maintenance on the server to be operated and maintained in the list of servers to be operated and maintained;

[0088] The alarm unit is used to issue an alarm when the risk assessment index corresponding to the server to be operated and maintained exceeds the risk assessment index threshold value when the server to be operated and maintained in the server list information is operated and maintained.

[0089] Compared with the prior art, the present invention has the following beneficial effects:

[0090] This scheme proposes a remote automated operation and maintenance method and system for a big data storage server. The method obtains a first mean, a second mean and a third mean by performing statistical analysis on the server system evaluation index, the server response time comprehensive index and the server load index of the target server, further obtains the corresponding volatility of the server system evaluation index, the server response time comprehensive index and the server load index, and establishes a performance volatility index matrix. The performance of the target server under different conditions is analyzed through the performance volatility index matrix to reveal which factors cause the fluctuation of server performance and determine whether the server configuration needs to be adjusted to optimize performance, so as to avoid configuration failure or configuration incompatibility due to different performance of the system or hardware of the big data storage server and different resource utilization of the same system or hardware. At the same time, through in-depth analysis of performance fluctuations, server resources can be more effectively allocated and managed to avoid resource waste and improve resource utilization.

[0091] This scheme proposes a remote automated operation and maintenance method and system for a big data storage server. The performance of the target server is evaluated through a performance fluctuation index matrix to obtain the risk assessment index of the target server. At the same time, the risk assessment index threshold is determined according to the performance fluctuation index matrix to determine whether the risk assessment index of the target server exceeds the threshold. If not, the target server is in a normal state, and the target server is configured for the next step based on the automated configuration information. If so, the target server is in an abnormal state, and the state of the target server can be rolled back according to the historical configuration information. Through the performance fluctuation index matrix, the automated operation and maintenance system can monitor the server performance in real time, detect abnormalities in time, and reduce the risk of system instability. The system automatically adjusts the configuration according to the evaluation results, optimizes resource usage, and improves server performance. When a performance abnormality is found, it can also automatically roll back to the historical stable configuration, reducing the impact on the business and downtime, and also helping to reduce the fluctuation of server performance and improve the overall stability and reliability of the system.

[0092] This scheme proposes a remote automated operation and maintenance method and system for a big data storage server. The method connects to the target server through login key information, obtains connection status information, extracts connection key information from the connection status information, and uses the connection key information to establish a login fluctuation coefficient matrix. The login key information is tuned through the login fluctuation coefficient matrix to strengthen the connection with the target server. The login key information can also be adjusted through the login fluctuation coefficient matrix to optimize the connection stability with the target server. The fluctuation coefficient matrix is ​​used to analyze the fluctuation of connection performance. Adjusting the key can improve encryption efficiency and connection stability, thereby improving the overall communication quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 A flowchart of a remote automated operation and maintenance method for a big data storage server proposed by the present invention;

[0094] Figure 2 The flowchart of establishing the login fluctuation coefficient matrix in the present invention;

[0095] Figure 3 A flowchart of obtaining and evaluating the risk assessment index in the present invention;

[0096] Figure 4 A flowchart for obtaining the automation script template in the present invention;

[0097] Figure 5 This is a system framework diagram of a remote automated operation and maintenance system for a big data storage server proposed by the present invention. DETAILED DESCRIPTION

[0098] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0099] Reference Figure 1 - Figure 5 As shown, a remote automated operation and maintenance method for a big data storage server in an embodiment of the present invention includes:

[0100] Determine the list of servers to be maintained and obtain the login key information of the target server;

[0101] Use the login key information to determine the corresponding target server, log in and connect to the target server, and obtain the login fluctuation coefficient matrix;

[0102] Refer to the login fluctuation coefficient matrix to optimize the login key information and strengthen the connection with the target server;

[0103] Send the highest authority acquisition instruction to the target server to obtain the highest authority;

[0104] Monitor the target server, obtain a performance fluctuation index matrix, and set automatic configuration information matching the target server based on the performance fluctuation index matrix;

[0105] Configure the target server based on the automated configuration information, and obtain the risk assessment index of the target server in real time during the configuration process;

[0106] Optimize the automated configuration information based on the risk assessment index and complete the configuration of the target server;

[0107] After the configuration of the target server is completed, an operation and maintenance request is sent to the target server, and the operation and maintenance of the target server is performed;

[0108] Using the automation configuration information of the target server, an automation configuration information base is established, and an automation configuration script template is created through the automation configuration information base;

[0109] The servers to be operated and maintained in the server list information are configured in sequence based on the automated configuration script template, and operation and maintenance requests are sent to the servers to be operated and maintained in the server list information, so as to implement operation and maintenance on the servers to be operated and maintained in the server list information.

[0110] Specifically, the list of servers to be maintained includes information such as the server name and IP address of the servers to be maintained. The server name is the identification name or number of the server, which is used for identification and management. The IP address is the network address of the server, which is used for remote connection and management. The login key information refers to the key or credential used for identity authentication and authorization. The key information includes key pair information, key length information, key algorithm information and key type information, and the key pair information includes public key and private key. The public key is used to encrypt information and can be paired with the private key for decryption. The private key is used to decrypt information or sign and needs to be properly kept to prevent leakage. Tools such as ssh-keygen can be used to generate key pairs.

[0111] Further, the corresponding target server is determined using the login key information, and the target server is logged in and connected to obtain a login fluctuation coefficient matrix, which specifically includes:

[0112] Obtain the server information and key information to be maintained, and establish a server information library and a key information library to be maintained. The key information includes key pair information, and the key pair information consists of a public key and a private key.

[0113] Extract key pair information from the key information in the key information repository, and randomly distribute the public key contained in the key pair information to the relevant files of the server to be operated and maintained in the server to be operated and maintained information repository;

[0114] Re-sort the server information base and key information base to be maintained, and determine the list of servers to be maintained;

[0115] Based on the information of the list of servers to be maintained, the server to be maintained that is ranked first in the list of servers to be maintained is used as the target server;

[0116] Based on the key information, determining login key information corresponding to the target server;

[0117] According to the login key information corresponding to the target server, a private key corresponding to the target server is obtained;

[0118] Use the private key corresponding to the target server to connect to the target server and obtain the connection status information;

[0119] Extracting key connection information from the connection status information, and using the key connection information to establish a login fluctuation coefficient matrix;

[0120] Among them, the expression formula of the login volatility coefficient matrix is:

[0121] ( );

[0122] In the formula, is the expression symbol of the log volatility matrix, It is the quantitative value of whether the connection is successful or not. is the quantitative value of the authentication status, is the server response time value, is the network connection quality index;

[0123] The calculation formula of network connection quality index is:

[0124]

[0125] In the formula, is the network connection quality index, is the actual network delay, is the ideal value of network delay, is the actual bandwidth, is the ideal value of bandwidth, is the actual packet loss rate, is the ideal value of packet loss rate, For the actual jitter, is the ideal value of jitter, , , and are the weights of delay, bandwidth, packet loss rate and jitter, respectively. and is the exponential decay coefficient, is the impact index of packet loss on network quality, It is the combined impact index of delay and jitter.

[0126] Specifically, the public key contained in the key pair information is randomly distributed to the relevant files of the server to be operated and maintained in the server information library to be operated and maintained, such as the SSH public key file (file location: ~ / .ssh / authorized_keys). The login fluctuation coefficient matrix is ​​some data generated by the automated operation and maintenance system connecting to or logging into the target server, including the authentication status, the fluctuation data of the login success rate, the fluctuation data of the delay time, etc. The target server refers to the server corresponding to the server list information to be operated and maintained determined according to the login key information. and is the exponential decay coefficient, which is used to adjust the decay speed of the impact factor. It is a quantitative value indicating whether the connection is successful or not. The value can be 0 or 1, 1 means the connection is successful, and 0 means otherwise. It is the quantitative value of the authentication status, which can be compared with same.

[0127] Furthermore, referring to the login fluctuation coefficient matrix, the login key information is optimized to strengthen the connection with the target server, including:

[0128] The login key information is optimized by using the quantized value of connection success or failure and the quantized value of authentication status in the login fluctuation coefficient matrix as the first tuning reference value of the login key information;

[0129] The login key information is tuned using the server response time value as a second tuning reference value of the login key information;

[0130] The login key information is optimized by using the network connection quality index as a third optimization reference value of the login key information;

[0131] Integrate the first tuning reference value, the second tuning reference value, and the third tuning reference value to obtain a login key tuning parameter matrix;

[0132] The corresponding login key information is tuned using the login key tuning parameter matrix.

[0133] Specifically, the first to third tuning benchmark values ​​are the tuning parameters or tuning method information corresponding to all coefficients in the login fluctuation coefficient matrix. For example, the quantified value of the connection success or failure and the quantified value of the authentication state in the login fluctuation coefficient matrix are used as the first tuning benchmark value of the login key information to tune the login key information. According to the quantified value of the authentication state, the key management strategy is optimized. For example, multi-factor authentication or dynamic key update strategy is implemented to improve the authentication success rate. The complexity and update frequency of the key are adjusted according to the success rate data. Increasing the complexity of the key may improve security, but it may also cause more connection failures. Therefore, a balance is needed, that is, the first tuning benchmark value is used. The first tuning benchmark value is the complexity and update frequency of the key; the server response time value is used as the second tuning benchmark value of the login key information to tune the login key information. The key processing algorithm can be optimized to reduce the response time, or the key usage frequency can be adjusted to reduce the impact on the server load. That is, the second tuning benchmark value is the quantitative value of the key processing algorithm; the network connection quality index is used as the third tuning benchmark value of the login key information to tune the login key information to ensure that the transmission channel of the key information is encrypted and stable. If the connection quality is poor, consider using a stronger encryption algorithm or improving the fault tolerance of the connection. That is, the third tuning benchmark value is the level of the encryption algorithm.

[0134] It can be understood that the first tuning benchmark value, the second tuning benchmark value and the third tuning benchmark value are integrated to obtain the login key tuning parameter matrix, which includes the first tuning benchmark value, the second tuning benchmark value and the third tuning benchmark value, that is, the complexity and update frequency of the key in the first tuning benchmark value, the quantization value of the key processing algorithm in the second tuning benchmark value, and the level of the encryption algorithm in the third tuning benchmark value. Finally, the login key tuning parameter matrix can be used to tune the corresponding login key information.

[0135] Furthermore, the target server is monitored to obtain a performance fluctuation index matrix, and automatic configuration information matching the target server is set based on the performance fluctuation index matrix, specifically including:

[0136] Through monitoring, the target server can be obtained in real time from 0 to Performance data at each moment, including server system evaluation index, server response time comprehensive index and server load index;

[0137] Performing statistical analysis on the server system evaluation index, the server response time comprehensive index and the server load index of the target server respectively to obtain a first mean value, a second mean value and a third mean value;

[0138] Taking the first mean, the second mean and the third mean as the standard, the volatility corresponding to the server system evaluation index, the server response time comprehensive index and the server load index is obtained, and a performance volatility index matrix is ​​established;

[0139] The calculation formula of the server system evaluation index is:

[0140] ;

[0141] In the formula, For the target server The server system evaluation index at the moment, For the target server The real-time network delay value of the system at the moment, For the target server The system load value at the moment, For the target server The configuration optimization value at the moment, For the target server CPU performance value at the moment, For the target server The memory capacity value at the moment, For the target server Disk IO value at the moment, For the target server The comprehensive index of the system version at the moment, , , and are weight coefficients;

[0142] The server load index is calculated as:

[0143] ;

[0144] In the formula, For the target server The server load index at the moment, For the target server The actual CPU usage at the moment, is the minimum value of CPU usage, is the maximum value of CPU usage. For the target server The actual memory utilization at the moment, is the minimum memory utilization, is the maximum value of memory utilization, For the target server The actual disk IO at the moment, is the minimum value of disk IO, is the maximum value of disk IO. For the target server The actual network traffic at the moment, is the minimum value of network traffic, is the maximum value of network traffic, , , and They are the weights of CPU usage, memory utilization, disk IO, and network traffic, respectively. is the exponential decay coefficient, To monitor the time;

[0145] The expression formula of the performance fluctuation index matrix is:

[0146] ;

[0147] In the formula, is the first configuration reference value, is the second configuration reference value, is the third configuration reference value, From 0 to The average value of the server system evaluation index at time, From 0 to The average value of the comprehensive index of server response time at time, From 0 to The average server load index at the time.

[0148] Furthermore, based on the risk assessment index, the automated configuration information is optimized and the configuration of the target server is completed, including:

[0149] Determine basic configuration information of the target server according to the first configuration reference value, the second configuration reference value, and the third configuration reference value in the performance fluctuation index matrix;

[0150] Determine configuration script compatibility information based on basic configuration information of the target server;

[0151] Determine configuration item information matching the target server based on the configuration script compatibility information, and obtain automated configuration information;

[0152] The target server is configured step by step based on the automated configuration information, and the performance fluctuation index matrix of the target server after each configuration step is obtained, and the historical configuration information of the target server before each configuration step is recorded;

[0153] Evaluate the performance of the target server based on the performance fluctuation index matrix, obtain the risk assessment index of the target server, and determine the risk assessment index threshold according to the performance fluctuation index matrix;

[0154] According to the risk assessment index threshold, determine whether the risk assessment index of the target server exceeds the threshold. If not, the target server is in a normal state, and the target server is configured further based on the automated configuration information. If yes, the target server is in an abnormal state, and the state of the target server can be rolled back based on the historical configuration information.

[0155] After rolling back the state of the target server, the risk assessment index and its corresponding performance fluctuation index matrix are used to trace the automation configuration information and optimize the automation configuration;

[0156] The calculation formula of the risk assessment index is:

[0157] ;

[0158] In the formula, is the risk assessment index, is the weight coefficient of the risk assessment index, is the error term.

[0159] Specifically, the performance fluctuation index matrix is ​​used to measure and evaluate the stability and changes of server performance over a certain period of time. The performance fluctuation index matrix includes the server system evaluation index, the server response time comprehensive index and the server load index. The risk assessment index is a comprehensive indicator of the various states of the target server, which can be used to evaluate and identify risks that may affect system stability and security. This index combines various factors to better predict and respond to potential problems. If the state of the target server is abnormal, the state of the target server can be rolled back based on historical configuration information. The historical configuration information is the configuration information of the target server before the system is configured by the automated operation and maintenance system, including configuration parameters, system settings, file configuration, etc. Rolling back the state of the target server based on historical configuration information can be understood as using historical configuration information to roll back the target server and restore the target server to the configuration state before the abnormal state.

[0160] Furthermore, the automatic configuration information of the target server is used to establish an automatic configuration information library and an automatic configuration script template, which specifically includes:

[0161] By using the risk assessment index when the target server is configured and its corresponding performance fluctuation index matrix, the automation configuration information that makes the target server state abnormal is traced back, and the abnormal automation configuration information and its corresponding performance fluctuation index matrix are obtained;

[0162] Based on the abnormal automation configuration information and its corresponding performance fluctuation index matrix, an automation configuration information library is established;

[0163] Collect historical configuration file information and configuration specification information, merge the data, obtain script generation basic data, and import the script generation basic data into the automated configuration information library;

[0164] Clean the basic data generated by the scripts in the automated configuration information library and standardize the cleaned data;

[0165] Extract important features from the script-generated basic data after standardization to obtain relevant feature information;

[0166] Using the random forest method, the regression model is trained based on the standardized script generation basic data and related feature information to obtain an automated configuration script generation model;

[0167] Generate an automated configuration script template through an automated configuration script generation model, and update an automated configuration information base when the automated configuration script template sequentially configures the servers to be operated and maintained in the server list information to be operated and maintained;

[0168] Optimize the automation configuration script templates using the updated automation configuration repository.

[0169] Specifically, the automated configuration script template can be understood as a preset script framework for automating the configuration and management of systems or applications. It provides a common structure and steps that can be customized according to specific needs. The template usually includes environment settings, configuration parameters, installation steps, etc., where the environment settings define the necessary environment variables and dependencies, the configuration parameters are predefined configuration options, such as server addresses, port numbers, authentication information, etc., and the installation steps are the steps for automating the installation and configuration of the required software or services.

[0170] Furthermore, a remote automated operation and maintenance system for a big data storage server is proposed, which is used to implement any of the above operation and maintenance methods, including:

[0171] The main control module is used to receive data transmitted by the configuration module, operation and maintenance module and display module through the transmission module, and process the received data, evaluate various information and indexes that appear in the data processing process, and control the configuration module, operation and maintenance module, transmission module and display module;

[0172] The configuration module is used to configure the target server in the list of servers to be operated and maintained, and to configure the login key information of the target server;

[0173] The operation and maintenance module is used to monitor and operate the target servers in the operation and maintenance server list information, and issue an alarm when the target server is in an abnormal state;

[0174] Transmission module, which is used to transmit data and information between the main control module, configuration module, operation and maintenance module and display module;

[0175] Display module, the display module is used to display the data processing results of the main control module, the configuration process and results of the configuration module, and the operation and maintenance process and results of the operation and maintenance module.

[0176] Furthermore, the main control module includes:

[0177] The data receiving unit is used to determine the list of servers to be operated and maintained, and obtain the login key information of the target server, and is used to receive the login fluctuation coefficient matrix, the performance fluctuation index matrix and the risk assessment index, and receive the target server from 0 to Performance data at the moment, server system evaluation index, server response time comprehensive index and server load index;

[0178] A data processing unit, the data processing unit is used to obtain information about servers to be operated and maintained and key information, establish an information base of servers to be operated and maintained and a key information base, perform statistical analysis on a server system evaluation index, a server response time comprehensive index and a server load index of a target server, respectively, obtain a first mean, a second mean and a third mean, optimize automatic configuration information based on a risk assessment index, and determine a risk assessment index threshold according to a performance fluctuation index matrix;

[0179] An evaluation unit, the evaluation unit is used to evaluate the performance of the target server based on the performance fluctuation index matrix, obtain the risk assessment index of the target server, and determine whether the risk assessment index of the target server exceeds the threshold according to the risk assessment index threshold;

[0180] Control unit, which is used to control the configuration module, operation and maintenance module, transmission module and display module.

[0181] Furthermore, the configuration module includes:

[0182] The server configuration unit is used to send a highest authority acquisition instruction to the target server to obtain the highest authority, set the automatic configuration information matching the target server based on the performance fluctuation index matrix, configure the target server based on the automatic configuration information, and complete the configuration of the target server, and sequentially configure the servers to be operated and maintained in the list information of the servers to be operated and maintained based on the automatic configuration script template;

[0183] The key configuration unit is used to determine the corresponding target server using the login key information, log in and connect to the target server, tune the login key information with reference to the login fluctuation coefficient matrix, extract the key pair information of the key information in the key information library, and randomly distribute the public key contained in the key pair information to the relevant files of the server to be operated and maintained in the server information library to be operated and maintained.

[0184] Furthermore, the operation and maintenance module includes:

[0185] The monitoring unit is used to obtain the login fluctuation coefficient matrix, monitor the target server, obtain the performance fluctuation index matrix, obtain the risk assessment index of the target server in real time during the configuration process, and obtain the target server in real time through monitoring. Performance data at all times;

[0186] The operation and maintenance unit is used to send an operation and maintenance request to the target server after the configuration of the target server is completed, and perform operation and maintenance on the target server, send an operation and maintenance request to the server to be operated and maintained in the list of servers to be operated and maintained, and perform operation and maintenance on the server to be operated and maintained in the list of servers to be operated and maintained;

[0187] The alarm unit is used to issue an alarm when the risk assessment index corresponding to the server to be operated and maintained exceeds the risk assessment index threshold value when the server to be operated and maintained in the server list information is operated and maintained.

[0188] In summary, the advantages of the present invention are: by establishing a performance fluctuation index matrix, analyzing the performance of the target server under different conditions, revealing which factors cause the fluctuation of server performance, determining whether it is necessary to adjust the configuration of the server to optimize performance, avoiding configuration failure or configuration mismatch due to different performance of the system or hardware of the big data storage server, and different resource utilization of the same system or hardware. At the same time, through in-depth analysis of performance fluctuations, server resources can be more effectively allocated and managed, avoiding resource waste and improving resource utilization;

[0189] Through the performance fluctuation index matrix, the automated operation and maintenance system can monitor server performance in real time, detect anomalies in a timely manner, and reduce the risk of system instability. The system automatically adjusts the configuration based on the evaluation results, optimizes resource usage, and improves server performance. It can also automatically roll back to the historical stable configuration when performance anomalies are found, reducing the impact on business and downtime, and also helping to reduce server performance fluctuations and improve the overall stability and reliability of the system.

[0190] By tuning the login key information through the login fluctuation coefficient matrix, the connection with the target server can be strengthened. The login key information can also be adjusted through the login fluctuation coefficient matrix to optimize the connection stability with the target server. The fluctuation coefficient matrix is ​​used to analyze the fluctuation of connection performance. Adjusting the key can improve encryption efficiency and connection stability, thereby improving the overall communication quality.

[0191] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A remote automated operation and maintenance method for a big data storage server, characterized in that: include: Determine the list of servers to be maintained and obtain the login key information of the target server; Use the login key information to determine the corresponding target server, log in and connect to the target server, and obtain the login fluctuation coefficient matrix; Refer to the login fluctuation coefficient matrix to optimize the login key information and strengthen the connection with the target server; Send the highest authority acquisition instruction to the target server to obtain the highest authority; Monitor the target server, obtain a performance fluctuation index matrix, and set automatic configuration information matching the target server based on the performance fluctuation index matrix; Configure the target server based on the automated configuration information, and obtain the risk assessment index of the target server in real time during the configuration process; Optimize the automated configuration information based on the risk assessment index and complete the configuration of the target server; After the configuration of the target server is completed, an operation and maintenance request is sent to the target server, and the operation and maintenance of the target server is performed; Using the automation configuration information of the target server, an automation configuration information base is established, and an automation configuration script template is created through the automation configuration information base; The servers to be operated and maintained in the list of servers to be operated and maintained are configured in sequence based on the automated configuration script template, and an operation and maintenance request is sent to the servers to be operated and maintained in the list of servers to be operated and maintained, so as to implement operation and maintenance on the servers to be operated and maintained in the list of servers to be operated and maintained; The method of using the login key information to determine the corresponding target server, logging in and connecting to the target server, and obtaining a login fluctuation coefficient matrix specifically includes: Obtaining information of the server to be operated and maintained and key information, and establishing an information base of the server to be operated and maintained and a key information base, wherein the key information includes key pair information, and the key pair information consists of a public key and a private key; Extract key pair information from the key information in the key information repository, and randomly distribute the public key contained in the key pair information to the relevant files of the server to be operated and maintained in the server to be operated and maintained information repository; Re-sort the server information base and key information base to be maintained, and determine the list of servers to be maintained; Based on the information of the list of servers to be maintained, the server to be maintained that is ranked first in the list of servers to be maintained is used as the target server; Based on the key information, determining login key information corresponding to the target server; According to the login key information corresponding to the target server, a private key corresponding to the target server is obtained; Use the private key corresponding to the target server to connect to the target server and obtain the connection status information; Extracting key connection information from the connection status information, and using the key connection information to establish a login fluctuation coefficient matrix; Among them, the expression formula of the login fluctuation coefficient matrix is: ( )； In the formula, is the expression symbol of the log volatility matrix, It is the quantitative value of whether the connection is successful or not. is the quantitative value of the authentication status, is the server response time value, is the network connection quality index; The calculation formula of the network connection quality index is: In the formula, is the network connection quality index, is the actual network delay, is the ideal value of network delay, is the actual bandwidth, is the ideal value of bandwidth, is the actual packet loss rate, is the ideal value of packet loss rate, For the actual jitter, is the ideal value of jitter, , , and are the weights of delay, bandwidth, packet loss rate and jitter, respectively. and is the exponential decay coefficient, is the impact index of packet loss on network quality, is the combined impact index of delay and jitter; The monitoring of the target server, obtaining the performance fluctuation index matrix, and setting the automatic configuration information matching the target server based on the performance fluctuation index matrix specifically includes: Through monitoring, the target server can be obtained in real time from 0 to performance data at each moment, the performance data including a server system evaluation index, a server response time comprehensive index, and a server load index; Performing statistical analysis on the server system evaluation index, the server response time comprehensive index and the server load index of the target server respectively to obtain a first mean value, a second mean value and a third mean value; Taking the first mean, the second mean and the third mean as the standard, the volatility corresponding to the server system evaluation index, the server response time comprehensive index and the server load index is obtained, and a performance volatility index matrix is ​​established; The calculation formula of the server system evaluation index is: ； In the formula, For the target server The server system evaluation index at the moment, For the target server The real-time network delay value of the system at the moment, For the target server The system load value at the moment, For the target server The configuration optimization value at the moment, For the target server CPU performance value at the moment, For the target server The memory capacity value at the moment, For the target server Disk IO value at the moment, For the target server The comprehensive index of the system version at the moment, , , and are weight coefficients; The calculation formula of the server load index is: ； In the formula, For the target server The server load index at the moment, For the target server The actual CPU usage at the moment, is the minimum value of CPU usage, is the maximum value of CPU usage. For the target server The actual memory utilization at the moment, is the minimum memory utilization, is the maximum value of memory utilization, For the target server The actual disk IO at the moment, is the minimum value of disk IO, is the maximum value of disk IO. For the target server The actual network traffic at the moment, is the minimum value of network traffic, is the maximum value of network traffic, , , and They are the weights of CPU usage, memory utilization, disk IO, and network traffic, respectively. is the exponential decay coefficient, To monitor the time; The expression formula of the performance fluctuation index matrix is: ； In the formula, is the first configuration reference value, is the second configuration reference value, is the third configuration reference value, From 0 to The average value of the server system evaluation index at time, From 0 to The average value of the comprehensive index of server response time at time, From 0 to The average value of the server load index at the moment; The step of optimizing the automatic configuration information based on the risk assessment index and completing the configuration of the target server specifically includes: Determine basic configuration information of the target server according to the first configuration reference value, the second configuration reference value, and the third configuration reference value in the performance fluctuation index matrix; Determine configuration script compatibility information based on basic configuration information of the target server; Determine configuration item information matching the target server based on the configuration script compatibility information, and obtain automated configuration information; The target server is configured step by step based on the automated configuration information, and the performance fluctuation index matrix of the target server after each configuration step is obtained, and the historical configuration information of the target server before each configuration step is recorded; Evaluate the performance of the target server based on the performance fluctuation index matrix, obtain the risk assessment index of the target server, and determine the risk assessment index threshold according to the performance fluctuation index matrix; According to the risk assessment index threshold, determine whether the risk assessment index of the target server exceeds the threshold. If not, the target server is in a normal state, and the target server is configured further based on the automated configuration information. If yes, the target server is in an abnormal state, and the state of the target server can be rolled back based on the historical configuration information. After rolling back the state of the target server, the risk assessment index and its corresponding performance fluctuation index matrix are used to trace the automation configuration information and optimize the automation configuration; The calculation formula of the risk assessment index is: ； In the formula, is the risk assessment index, is the weight coefficient of the risk assessment index, is the error term.

2. A remote automated operation and maintenance method for a big data storage server according to claim 1, characterized in that: The step of tuning the login key information by referring to the login fluctuation coefficient matrix and strengthening the connection with the target server specifically includes: The login key information is optimized by using the quantized value of connection success or failure and the quantized value of authentication status in the login fluctuation coefficient matrix as the first tuning reference value of the login key information; The login key information is tuned using the server response time value as a second tuning reference value of the login key information; The login key information is optimized by using the network connection quality index as a third optimization reference value of the login key information; Integrate the first tuning reference value, the second tuning reference value, and the third tuning reference value to obtain a login key tuning parameter matrix; The corresponding login key information is tuned using the login key tuning parameter matrix.

3. A remote automated operation and maintenance method for a big data storage server according to claim 1, characterized in that: The method of using the automatic configuration information of the target server to establish an automatic configuration information library and automatically configure the script template specifically includes: By using the risk assessment index when the target server is configured and its corresponding performance fluctuation index matrix, the automation configuration information that makes the target server state abnormal is traced back, and the abnormal automation configuration information and its corresponding performance fluctuation index matrix are obtained; Based on the abnormal automation configuration information and its corresponding performance fluctuation index matrix, an automation configuration information library is established; Collect historical configuration file information and configuration specification information, merge the data, obtain script generation basic data, and import the script generation basic data into the automated configuration information library; Clean the basic data generated by the scripts in the automated configuration information library and standardize the cleaned data; Extract important features from the script-generated basic data after standardization to obtain relevant feature information; Using the random forest method, the regression model is trained based on the standardized script generation basic data and related feature information to obtain an automated configuration script generation model; Generate an automated configuration script template through an automated configuration script generation model, and update an automated configuration information base when the automated configuration script template sequentially configures the servers to be operated and maintained in the server list information to be operated and maintained; Optimize the automation configuration script templates using the updated automation configuration repository.

4. A remote automated operation and maintenance system for a big data storage server, used to implement the operation and maintenance method according to any one of claims 1 to 3, characterized in that: include: A main control module, which is used to receive data transmitted by the configuration module, the operation and maintenance module, and the display module through the transmission module, and to process the received data, and at the same time to evaluate various information and indexes that appear in the data processing process, and to control the configuration module, the operation and maintenance module, the transmission module, and the display module; A configuration module, the configuration module is used to configure the target server in the list of servers to be operated and maintained, and to configure the login key information of the target server; An operation and maintenance module, which is used to monitor and operate the target server in the list of servers to be operated and maintained, and issue an alarm when the target server is in an abnormal state; A transmission module, which is used to transmit data and information between the main control module, the configuration module, the operation and maintenance module and the display module; The display module is used to display the data processing results of the main control module, the configuration process and results of the configuration module, and the operation and maintenance process and results of the operation and maintenance module.

5. A remote automated operation and maintenance system for a big data storage server according to claim 4, characterized in that: The main control module comprises: The data receiving unit is used to determine the list of servers to be operated and maintained, and obtain the login key information of the target server, and is used to receive the login fluctuation coefficient matrix, the performance fluctuation index matrix and the risk assessment index, and receive the target server from 0 to Performance data at the moment, server system evaluation index, server response time comprehensive index and server load index; A data processing unit, the data processing unit is used to obtain information about servers to be operated and maintained and key information, establish an information base of servers to be operated and maintained and a key information base, perform statistical analysis on a server system evaluation index, a server response time comprehensive index, and a server load index of a target server, respectively, obtain a first mean, a second mean, and a third mean, optimize automatic configuration information based on a risk assessment index, and determine a risk assessment index threshold according to a performance fluctuation index matrix; An evaluation unit, the evaluation unit is used to evaluate the performance of the target server based on the performance fluctuation index matrix, obtain the risk assessment index of the target server, and determine whether the risk assessment index of the target server exceeds the threshold according to the risk assessment index threshold; A control unit, wherein the control unit is used to control the configuration module, the operation and maintenance module, the transmission module and the display module.

6. A remote automated operation and maintenance system for a big data storage server according to claim 4, characterized in that: The configuration module includes: A server configuration unit, the server configuration unit is used to send a highest authority acquisition instruction to a target server, obtain the highest authority, set automatic configuration information matching the target server based on a performance fluctuation index matrix, configure the target server based on the automatic configuration information, and complete the configuration of the target server, and sequentially configure the servers to be operated and maintained in the list information of servers to be operated and maintained based on an automatic configuration script template; A key configuration unit is used to determine the corresponding target server using the login key information, log in and connect to the target server, tune the login key information with reference to the login fluctuation coefficient matrix, extract the key pair information of the key information in the key information library, and randomly distribute the public key contained in the key pair information to the relevant files of the server to be operated and maintained in the server information library to be operated and maintained.

7. A remote automated operation and maintenance system for a big data storage server according to claim 4, characterized in that: The operation and maintenance module includes: The monitoring unit is used to obtain the login fluctuation coefficient matrix, monitor the target server, obtain the performance fluctuation index matrix, obtain the risk assessment index of the target server in real time during the configuration process, and obtain the target server in real time through monitoring. Performance data at all times; The operation and maintenance unit is used to send an operation and maintenance request to the target server after the configuration of the target server is completed, and perform operation and maintenance on the target server, send an operation and maintenance request to the server to be operated and maintained in the list of servers to be operated and maintained, and perform operation and maintenance on the server to be operated and maintained in the list of servers to be operated and maintained; The alarm unit is used to issue an alarm when the risk assessment index corresponding to the server to be operated and maintained exceeds the risk assessment index threshold value when the server to be operated and maintained in the server list information is operated and maintained.

Images