Node capacity expansion method and device, electronic equipment and program product

Abstract

The invention discloses a node capacity expansion method and device, electronic equipment and a program product, and relates to the field of distributed systems.The method comprises the steps that index data of N nodes in a cluster are collected, configuration information of the N nodes is obtained, and historical index data are obtained; constructing a prediction model based on the historical index data, and performing model inspection on the constructed prediction model to obtain a target prediction model passing the inspection; predicting table capacity occupation conditions of the N nodes based on the target prediction model to obtain a prediction result; and expanding the table capacity of the N nodes based on the prediction result. According to the method and the device, the problem that the cluster performance is reduced due to the fact that the prediction result is inaccurate due to the fact that the prediction result is easily influenced by short-term business when the occupation condition of the future capacity of the cluster node is predicted in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of distributed systems, and in particular, to a method, device, electronic device, and program product for expanding the capacity of nodes. Background Art

[0002] Due to regulatory considerations, bank performance capacity management cannot directly participate in business discussions, so it is impossible to analyze future business requirements and judge business growth. Often, due to insufficient communication with the business department in the early stage and ineffective coordination with upstream units, serious consequences of IT systems hindering business development will occur. The performance capacity index system constructed based on various performance capacity data has not been refined, and most of the indexes are formulated for the entire IT system. With the continuous growth of banking business, the applications carried on the IT system are also increasing. If the indexes cannot be refined, it is difficult to effectively evaluate the advantages and disadvantages of new applications and comprehensively grasp the current production status.

[0003] In terms of capacity prediction, there is a lack of an effective capacity prediction model. Bank performance capacity management uses a simple univariate linear regression model for trend prediction of various index capacities, which is easily affected by short-term business and results in too high or too low expectations, and manually judges whether the model prediction results are normal. In this process, manual judgment is prone to overlooking important risk points.

[0004] Aiming at the problem that when predicting the future capacity occupancy of cluster nodes in related technologies, the prediction results are easily affected by short-term business and are inaccurate, leading to a decline in cluster performance, no effective solution has been proposed yet. Summary of the Invention

[0005] The main purpose of the present application is to provide a method, device, electronic device, and program product for expanding the capacity of nodes, so as to solve the problem that when predicting the future capacity occupancy of cluster nodes in related technologies, the prediction results are easily affected by short-term business and are inaccurate, leading to a decline in cluster performance.

[0006] To achieve the above object, according to one aspect of the present application, a method for expanding the capacity of nodes is provided. The method includes: collecting index data of N nodes in a cluster, and obtaining configuration information of the N nodes to obtain historical index data, where N is a positive integer; constructing a prediction model based on the historical index data, and performing model verification on the constructed prediction model to obtain a target prediction model that passes the verification; predicting the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result; and expanding the table capacity of the N nodes based on the prediction result.

[0007] Further, a prediction model is constructed based on the historical indicator data, and the constructed prediction model is tested to obtain a target prediction model that passes the test, including: performing a difference operation on the historical indicator data to convert the historical indicator data into a stationary difference sequence and obtaining a difference order; performing a white noise test on the stationary difference sequence according to a preset statistic to obtain a first test result. Wherein, the first test result at least includes: a first variable; when the value of the first variable is less than or equal to a preset significance level, a maximum likelihood ratio model is used to determine a first parameter and a second parameter; determining the constructed prediction model according to the difference order, the first parameter, and the second parameter, and obtaining the target prediction model when the constructed prediction model passes the model test.

[0008] Further, performing a difference operation on the historical indicator data to convert the historical indicator data into a stationary difference sequence and obtaining a difference order includes: performing a stationarity test on the historical indicator data using a unit root test method to obtain a second test result, where the second test result at least includes: a second variable; when the value of the second variable is less than or equal to a preset significance level, determining that the historical indicator data of the N nodes belongs to a stationary sequence and determining the difference order; when the value of the first variable is greater than the preset significance level, determining that the historical indicator data of the N nodes does not belong to a stationary sequence, and performing a difference operation on the historical indicator data until the historical indicator data belongs to a stationary sequence to obtain the difference order.

[0009] Further, collecting the indicator data of N nodes in the cluster and obtaining the configuration information of the N nodes to obtain historical indicator data includes: collecting the indicator data of the N nodes and collecting the batch task information of the N nodes, where the indicator data at least includes first-class data and second-class data, the first-class data represents the indicator data collected every first preset time period, and the second-class data represents the indicator data different from the first-class data; obtaining the configuration information of the N nodes and constructing a topology graph of the N nodes according to the configuration information; updating the topology graph of the N nodes according to the indicator data of the N nodes and the batch task information of the N nodes to obtain the historical indicator data.

[0010] Further, update the topology graph of the N nodes based on the index data of the N nodes and the batch task information of the N nodes to obtain the historical index data, including: determining the IP information of the N nodes according to the configuration information of the N nodes; respectively performing data preprocessing on the index data of the N nodes and the batch task information of the N nodes, and storing them in a target database to obtain the processed index data; determining the time series information of the processed index data; updating the topology graph of the N nodes based on the processed index data, the time series information of the processed index data, and the IP information of the N nodes to obtain the historical index data.

[0011] Further, predict the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result, including: calculating the table capacity occupancy of the N nodes within a preset time period based on the target prediction model for the historical index data; processing the table capacity occupancy of the N nodes within the preset time period to obtain the prediction result, and displaying the prediction result in a target device, where the prediction result at least includes: a first result and a second result, the first result is a result obtained by analyzing the table capacity occupancy of the N nodes from a business perspective, and the second result is a result obtained by analyzing the table capacity occupancy of the N nodes from a resource perspective.

[0012] Further, expand the table capacity of the N nodes based on the prediction result, including: determining target nodes to be expanded among the N nodes based on the prediction result every preset time duration; determining the resource information of the cluster, where the resource information at least includes: currently available resource information; determining an expansion strategy based on the resource information of the cluster and the target nodes; expanding the table capacity of the target nodes according to the expansion strategy.

[0013] To achieve the above object, according to another aspect of the present application, there is provided an expansion device for node capacity, the device includes: a collection unit, configured to collect index data of N nodes in a cluster and obtain the configuration information of the N nodes to obtain historical index data, where N is a positive integer; a construction unit, configured to construct a prediction model based on the historical index data and perform model verification on the constructed prediction model to obtain a target prediction model that passes the verification; a prediction unit, configured to predict the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result; and an expansion unit, configured to expand the table capacity of the N nodes based on the prediction result.

[0014] Further, the construction unit includes: a first calculation subunit, configured to perform a difference operation on the historical index data, convert the historical index data into a stationary difference sequence, and obtain a difference order; a test subunit, configured to perform a white noise test on the stationary difference sequence according to a preset statistic to obtain a first test result, where the first test result at least includes: a first variable; a first determination subunit, configured to determine a first parameter and a second parameter by using a maximum likelihood ratio model when the value of the first variable is less than or equal to a preset significance level; a second determination subunit, configured to determine the constructed prediction model according to the difference order, the first parameter, and the second parameter, and obtain the target prediction model when the constructed prediction model passes a model test.

[0015] Further, the calculation subunit includes: a test module, configured to perform a stationarity test on the historical index data by using a unit root test method to obtain a second test result, where the second test result at least includes: a second variable; a first determination module, configured to determine that the historical index data of the N nodes belongs to a stationary sequence and determine the difference order when the value of the second variable is less than or equal to a preset significance level; a calculation module, configured to determine that the historical index data of the N nodes does not belong to a stationary sequence when the value of the first variable is greater than the preset significance level, and perform a difference operation on the historical index data until the historical index data belongs to a stationary sequence to obtain the difference order.

[0016] Further, the acquisition unit includes: an acquisition subunit, configured to acquire the index data of the N nodes and acquire the batch task information of the N nodes, where the index data at least includes first-class data and second-class data, the first-class data represents the index data acquired every first preset time period, and the second-class data represents index data different from the first-class data; a construction subunit, configured to obtain the configuration information of the N nodes and construct a topology map of the N nodes according to the configuration information; an update subunit, configured to update the topology map of the N nodes according to the index data of the N nodes and the batch task information of the N nodes to obtain the historical index data.

[0017] Further, the update subunit includes: a second determination module, configured to determine the IP information of the N nodes according to the configuration information of the N nodes; a processing module, configured to perform data preprocessing on the metric data of the N nodes and the batch task information of the N nodes respectively, and store them in a target database to obtain the processed metric data; a third determination module, configured to determine the time series information of the processed metric data; an update module, configured to update the topology diagrams of the N nodes according to the processed metric data, the time series information of the processed metric data, and the IP information of the N nodes, to obtain the historical metric data.

[0018] Further, the prediction unit includes: a second calculation subunit, configured to calculate the table capacity occupancy of the N nodes within a preset time period based on the target prediction model; a processing subunit, configured to process the table capacity occupancy of the N nodes within the preset time period to obtain the prediction result, and display the prediction result in a target device, where the prediction result at least includes: a first result and a second result, the first result is a result obtained by analyzing the table capacity occupancy of the N nodes from a business perspective, and the second result is a result obtained by analyzing the table capacity occupancy of the N nodes from a resource perspective.

[0019] Further, the capacity expansion unit includes: a third determination subunit, configured to determine a target node to be expanded among the N nodes based on the prediction result every preset duration; a fourth determination subunit, configured to determine the resource information of the cluster, where the resource information at least includes: currently available resource information; a fifth determination subunit, configured to determine a capacity expansion strategy according to the resource information of the cluster and the target node; a capacity expansion subunit, configured to expand the table capacity of the target node according to the capacity expansion strategy.

[0020] To achieve the above object, according to one aspect of the present application, there is provided a computer program product, including a computer program, where when the computer program is executed by a processor, it implements the method for expanding the capacity of a node as described in any one of the above, and when the computer program is executed by a processor, it implements the steps of the method for expanding the capacity of a node in various embodiments of the present application.

[0021] To achieve the above object, according to one aspect of the present application, there is provided a computer-readable storage medium, where the computer-readable storage medium includes stored computer instructions, and when the computer instructions are executed by a processor, the method for expanding the capacity of a node as described in any one of the above is implemented.

[0022] To achieve the above object, according to one aspect of the present application, there is provided an electronic device, including one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for expanding the node capacity described in any one of the above.

[0023] In an embodiment of the present application, by collecting the metric data of N nodes in a cluster and obtaining the configuration information of the N nodes, historical metric data is obtained, where N is a positive integer; a prediction model is constructed based on the historical metric data, and the constructed prediction model is tested to obtain a target prediction model that passes the test; the table capacity occupancy of the N nodes is predicted based on the target prediction model to obtain a prediction result; the table capacity of the N nodes is expanded based on the prediction result, thereby solving the technical problem that when predicting the future capacity occupancy of cluster nodes, the prediction result is inaccurate due to being easily affected by short-term services, resulting in a decline in cluster performance.

[0024] By collecting the metric data and configuration information of nodes in real time to construct historical metric data, using the ARIMA model to construct a target prediction model, and performing model testing, the effectiveness of the model can be automatically verified, the reliability of the prediction result can be ensured, the uncertainty of human judgment can be reduced, the scientificity and accuracy of the prediction can be improved. At the same time, by predicting the future table capacity usage of cluster nodes through the target prediction model, automatically identifying the nodes that need to be expanded, and formulating corresponding expansion strategies, the operation and maintenance personnel can focus on more valuable work, that is, planning resources in advance, avoiding performance problems caused by resource bottlenecks, ensuring the timely replenishment of table capacity resources, and reducing the risk of system interruption or degradation caused by insufficient resources, thereby improving the business continuity and customer experience of financial institutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0026] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing the method for expanding the node capacity according to Embodiment 1 of the present application;

[0027] Figure 2 is a flowchart of an optional method for expanding the node capacity according to Embodiment 1 of the present application;

[0028] Figure 3 is a schematic diagram of the structure of an optional data acquisition layer according to Embodiment 1 of the present application;

[0029] Figure 4 It is a schematic flowchart of the construction and verification of an optional ARIMA prediction model provided in Embodiment 1 of the present application;

[0030] Figure 5 It is a schematic flowchart of an optional dynamic expansion of the node table capacity based on the ARIMA model provided in Embodiment 1 of the present application;

[0031] Figure 6 It is a schematic diagram of an expansion device for node capacity provided in Embodiment 2 of the present application;

[0032] Figure 7 It is a schematic diagram of an expansion electronic device for node capacity provided in Embodiment 5 of the present application. Detailed implementation manners

[0033] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0034] It should be noted that the processing methods, devices, storage media, and methods and devices for determining electronic devices in the present application documents can be used in the field of fintech to improve the cluster performance during the process of predicting the cluster node table capacity and expanding the cluster node table capacity based on the prediction results, and can also be used in any field other than the fintech field. The application fields of the processing methods, devices, storage media, and methods and devices of the present application documents are not limited.

[0035] It should be noted that the information collected in the present application (including but not limited to user device information, user personal information, collected data, used data, generated data, processed data, etc.) and data (including but not limited to data for analysis, stored data, displayed data, collected information, used information, generated information, processed information, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure, and application of relevant data and other processing all comply with the relevant laws, regulations, and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set between the present system and relevant users or institutions to provide corresponding operation entrances for users to choose to agree or refuse the automated decision-making results; if the user chooses to refuse, the expert decision-making process will be entered.

[0036] Embodiment 1

[0037] According to an embodiment of the present application, there is also provided an embodiment of a method for expanding the node capacity. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0038] The method embodiment provided by the first embodiment of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Figure 1 The following shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing the method for expanding the node capacity. As Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more (shown as 102a, 102 for expanding the node capacity,..., 102n in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the USB bus for expanding the node capacity), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only illustrative and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0039] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in whole or in part as software, hardware, firmware, or any arbitrary combination. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is used for processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for expanding the node capacity in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned method for expanding the node capacity. The memory 104 may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0043] Under the above operating environment, the present application provides a method for expanding the node capacity as shown in Figure 2 the following. Figure 2 is a flowchart of the method for expanding the node capacity according to Embodiment 1 of the present application.

[0044] Step S201: Collect the metric data of N nodes in the cluster, and obtain the configuration information of the N nodes to obtain historical metric data, where N is a positive integer.

[0045] In the first embodiment, in the business system of a financial institution, there is one or more clusters composed of multiple servers, and these servers are responsible for different business processing tasks or data processing tasks. The operation and maintenance system automatically collects performance metric data (for example, metric data such as CPU utilization rate, memory usage, I / O operation frequency, network bandwidth utilization rate, disk utilization rate, and tablespace utilization rate of the database) from each node in the cluster to reflect the running status and resource utilization of the cluster nodes. Here, "N" represents the total number of nodes in the cluster and is a positive integer.

[0046] In addition to performance metric data, the operation and maintenance system also needs to collect the configuration information of each node, which may include but is not limited to server hardware specifications (such as processor type, memory size, hard disk type and capacity), operating system version, network configuration (such as IP address, network interface type), database configuration (such as version, tablespace size, log file size), etc.

[0047] The collected performance metric data and configuration information are processed and stored to obtain the above-mentioned historical metric data. By analyzing these historical metric data, resource usage patterns, trends, and potential anomalies can be identified, enabling accurate prediction of future resource requirements and performance, achieving reasonable resource planning and automatic scaling, and avoiding service interruptions in financial institutions or performance degradation of the cluster due to insufficient resources.

[0048] Step S202: Build a prediction model based on the historical metric data and conduct model verification on the built prediction model to obtain a target prediction model that passes the verification.

[0049] In the first embodiment, the time series analysis algorithm (for example, the Autoregressive Integrated Moving Average model, hereinafter simply referred to as the ARIMA model) is used to calculate the historical metric data to build a prediction model. The ARIMA model is a statistical model widely used for predicting non-stationary time series data and can capture trends and seasonal variations in the data.

[0050] After building the prediction model, it is necessary to conduct a Ljung-Box test on the residuals of the prediction model (that is, the difference between the actual value and the predicted value), which is a statistical test method for testing whether the residual sequence shows autocorrelation, so as to determine whether the built prediction model can be used to predict future resource usage.

[0051] By using the historical metric data to build and optimize the prediction model, the effective prediction and management of the resource requirements of the business system can be achieved, thus ensuring the stability and business continuity of the business system of financial institutions.

[0052] Step S203: Predict the table capacity occupancy of N nodes based on the target prediction model to obtain a prediction result.

[0053] In the first embodiment, after determining the target prediction model, it is applied to the historical table capacity data of N nodes (i.e., the records of the database table space usage). By analyzing the patterns and trends of the table capacity occupancy in the historical metric data, the model predicts the possible usage of the database table space of each node at a future time point (such as half an hour later, one day later, etc.). N is the number of nodes in the cluster, and each node may host different databases or applications. Therefore, the target prediction model can be used to perform an independent prediction process for each node to ensure the accuracy and pertinence of the prediction results.

[0054] Predicting the table capacity occupancy of N nodes through the target prediction model and obtaining the prediction results helps financial institutions manage IT resources more effectively and timely, improving the stability and business continuity of the business systems of financial institutions.

[0055] Step S204, expand the table capacity of N nodes based on the prediction results.

[0056] In the first embodiment, when the prediction results indicate that the table capacity of a certain node is close to or exceeds the warning value, the business system will automatically expand the capacity according to the existing resource amount and the preset expansion strategy. The expansion strategy may include increasing the hard disk space, adjusting the table space configuration parameters (such as the pre-allocation size), optimizing the database table structure to reduce the storage requirements, etc.

[0057] Through this automated process, financial institutions can manage their business systems more efficiently, especially during peak business periods or when the data volume is growing rapidly, and can respond to resource requirements in a timely manner to ensure the stability and business continuity of the business systems.

[0058] Optionally, in the method for expanding the node capacity provided in the first embodiment of the present application, a prediction model is constructed based on historical metric data, and the constructed prediction model is subjected to model verification to obtain a target prediction model that passes the verification, including: performing a difference operation on the historical metric data to convert the historical metric data into a stationary difference sequence and obtaining the difference order; performing a white noise test on the stationary difference sequence according to a preset statistic to obtain a first test result, where the first test result at least includes: a first variable; in the case where the value of the first variable is less than or equal to the preset significance level, a maximum likelihood ratio model is used to determine a first parameter and a second parameter; determining the constructed prediction model according to the difference order, the first parameter, and the second parameter, and obtaining the target prediction model in the case where the constructed prediction model passes the model verification.

[0059] In the first embodiment, the time series data included in the collected historical indicator data is often not stationary, that is, its statistical characteristics (such as mean and variance) change over time. However, a prerequisite for constructing a prediction model using the ARIMA model is that the data series must be stationary. Therefore, first, it is necessary to perform a differencing operation on the collected historical indicator data, that is, calculate the sequential change of the data series until the series exhibits statistical stationarity. The differencing operation can eliminate the trend or periodic components in the data series, making its mean and variance stable over time.

[0060] By performing multiple differencing operations on the historical indicator data, a non-stationary series can be transformed into a stationary series. The number of differencing operations is called the differencing order d, and this order needs to be determined through a stationarity test using the unit root test method (Augmented Dickey-Fuller, which can be abbreviated as the ADF algorithm). If the P-value in the ADF test statistic calculated using the ADF algorithm is less than the significance level (such as 0.05), the series is considered stationary; otherwise, further differencing is required until the series becomes stationary. The differencing order d reflects the complexity of the trend or periodic changes in the original series.

[0061] Then, the Ljung-Box statistic (abbreviated as the LB statistic) is used for the white noise test. The white noise test is used to check whether the differenced series has any predictable patterns or structures, that is, whether its autocorrelation coefficients are significantly non-zero. If the P-value corresponding to the LB statistic (i.e., the above-mentioned first variable) is greater than the significance level, then within the confidence interval, the differenced series is considered a white noise series, indicating that there is no correlation between any point in the series and the previous time points, and the values of the series are random.

[0062] Secondly, if the differenced series is not a white noise series, that is, there is autocorrelation, then the maximum likelihood ratio model (information criterion method, such as BIC) can be used to determine the autoregressive parameter p (i.e., the above-mentioned first parameter) and the moving average parameter q (i.e., the above-mentioned second parameter) of the ARIMA model.

[0063] Finally, after determining the difference order d, the autoregressive parameter p, and the moving average parameter q, a specific ARIMA prediction model can be constructed. Based on the parameters p, d, and q of the ARIMA model, the type of the prediction model is determined. This model can capture the trend, periodicity, and random fluctuation components in the time series and is used to predict future metric data. After constructing the prediction model, it needs to be tested to confirm the effectiveness of the model. The model test includes the Ljung-Box model test, which is used to verify whether the model residuals are a white noise sequence, that is, whether the model has captured all the predictable information in the sequence. If the P value of the Ljung-Box test is greater than the significance level, then the model passes the test, and the model at this time is regarded as the target prediction model and can be used for future prediction and decision-making.

[0064] Through the above steps, an ARIMA prediction model based on historical metric data and tested (i.e., the above-mentioned target prediction model) can be obtained, which is used to predict the table capacity occupancy of N nodes in the cluster, so as to realize the automatic expansion of the table capacity of the cluster nodes to cope with the fluctuations and growth of business requirements, achieving the effect of improving the stability of the business system.

[0065] Optionally, in the method for expanding the node capacity provided in the first embodiment of the present application, performing a difference operation on the historical metric data to convert the historical metric data into a stationary difference sequence and obtaining the difference order includes: using the unit root test method to perform a stationarity test on the historical metric data to obtain a second test result, where the second test result at least includes: a second variable; in the case where the value of the second variable is less than or equal to a preset significance level, determining that the historical metric data of the N nodes belongs to a stationary sequence and determining the difference order; in the case where the value of the first variable is greater than the preset significance level, determining that the historical metric data of the N nodes does not belong to a stationary sequence and performing a difference operation on the historical metric data until the historical metric data belongs to a stationary sequence to obtain the difference order.

[0066] In the first embodiment, the ADF algorithm can be used to determine whether the historical metric data belongs to a stationary sequence. The ADF test will give a statistic and a corresponding P value (i.e., the above-mentioned second variable), which are used to judge the stationarity of the time series.

[0067] If the P-value in the second test result is less than or equal to a preset significance level (for example, it can be set to 0.05), it indicates that the historical indicator data is stationary and can enter the subsequent modeling process without the need for differencing operations, and the differencing order can be set to a preset value (i.e., 0). If the P-value of the ADF test is greater than the preset significance level, it indicates that the historical indicator data series is non-stationary. To make the series stationary, differencing operations are required, that is, calculating the successive changes in the series values until the series becomes stationary, that is, the P-value of the ADF test is less than or equal to the significance level, and the value of the differencing order d is determined according to the number of differencing operations.

[0068] By performing differencing operations, it is possible to ensure that the data series processed by the model is stationary, thereby improving the prediction accuracy and reliability of the model. Through the above steps, it is possible to determine whether differencing operations are required for the historical indicator data and the number of differences based on the second test result (the P-value of the ADF test), so as to obtain stationary series data that can be used for ARIMA model modeling, which in turn helps to construct a more accurate target prediction model.

[0069] Optionally, in the method for expanding the node capacity provided in the first embodiment of the present application, the indicator data of N nodes in the cluster is collected, and the configuration information of the N nodes is obtained to obtain historical indicator data, including: collecting the indicator data of the N nodes and collecting the batch task information of the N nodes, where the indicator data at least includes first-class data and second-class data, the first-class data represents the indicator data collected every first preset time period, and the second-class data represents the indicator data different from the first-class data; obtaining the configuration information of the N nodes and constructing a topology map of the N nodes according to the configuration information; updating the topology map of the N nodes according to the indicator data of the N nodes and the batch task information of the N nodes to obtain historical indicator data.

[0070] In the first embodiment, the first-class data represents the performance indicator data that needs to be collected in real time, that is, the performance indicator data automatically collected every first preset time period (for example, every 1 minute, every 5 minutes, every hour, etc.). The first-class data may include, but is not limited to, the usage conditions of key resources such as CPU utilization, memory usage, I / O operation frequency, network bandwidth utilization rate, and disk usage, which are used to continuously monitor the running status of the nodes. The second-class data represents the performance indicator data that does not need to be collected in real time. For example, the tablespace usage of a certain database, the transaction volume or response time of a specific service, etc.

[0071] The batch task information represents the information of the batch tasks executed by the business system. For example, the running status and resource consumption conditions of the nodes when completing daily batch jobs (such as settlement, report generation, etc.). Collecting batch task information helps to identify potential performance bottlenecks or resource demand peaks of the cluster nodes.

[0072] The configuration information of N nodes may include the hardware specifications of the nodes (such as CPU model, memory size, disk type), software configuration (such as operating system version, database type and version), network configuration (such as IP address, network bandwidth), etc. The configuration information is used to create a topology map of the N nodes. The topology map shows the logical and physical connections between the nodes, as well as the configuration information of each node. This helps the operation and maintenance personnel understand the overall architecture and resource distribution of the system, providing a structured perspective for data mining and resource management.

[0073] Finally, the first type of data of the N nodes, the second type of data of the N nodes, and the batch task information of the N nodes are updated to the topology map of the N nodes to obtain the historical metric data of the N nodes that changes over time.

[0074] By constructing the historical metric data, it is ensured that the operation and maintenance system can capture and reflect the resource usage and business load of each node changing over time in real time and accurately, providing a solid data foundation for subsequent time series analysis, prediction model construction, and automatic scaling. Through the continuous update of the historical metric data, the operation and maintenance system can continuously learn and adapt to the changes in node behavior and business models, improving the accuracy of prediction and the efficiency of resource management.

[0075] Optionally, in the method for expanding the node capacity provided in the first embodiment of this application, the topology map of the N nodes is updated according to the metric data of the N nodes and the batch task information of the N nodes to obtain historical metric data, including: determining the IP information of the N nodes according to the configuration information of the N nodes; respectively preprocessing the metric data of the N nodes and the batch task information of the N nodes, and storing them in the target database to obtain the processed metric data; determining the time series information of the processed metric data; and updating the topology map of the N nodes according to the processed metric data, the time series information of the processed metric data, and the IP information of the N nodes to obtain historical metric data.

[0076] In the first embodiment, according to the configuration information of each node, the IP address information of each node is extracted. The IP information is used to identify each node in the network, which helps to associate the performance metric data and batch task information collected from different data sources to the correct node.

[0077] Preprocess the metric data (including CPU utilization, memory usage, I / O operation frequency, network bandwidth utilization, disk usage, transaction volume or response time of specific services, etc.) and batch task information (involving the resource consumption of nodes when completing daily batch jobs) collected from N nodes. The steps of data preprocessing may include: cleaning the data, that is, removing outliers, missing values or error records to ensure the accuracy and integrity of the data; data alignment, that is, aligning the data collected from different nodes in terms of time to ensure that all data is within the same time frame; format unification, that is, converting the data into a unified format or unit for subsequent data storage and analysis; data aggregation, for example, calculating statistical information such as averages and maximum values.

[0078] The preprocessed metric data and batch task information are stored in a target database. The target database is a place for centralized storage and management of all processed data. It not only stores the current performance metrics but also retains historical data, providing a data basis for subsequent time series analysis and prediction. Determine the time series information of the processed metric data in the target database, and these time series information describe the changes of the metric data over time. The time series information usually includes the timestamps of data collection.

[0079] Finally, combine the configuration information of N nodes (including IP information), the processed metric data and the time series information to update the topology map of N nodes to obtain historical metric data. The topology map reflects the logical and physical connections between nodes, as well as the real-time performance status and resource usage of each node. By dynamically updating the topology map, the current state of the entire network can be more accurately reflected. The historical metric data includes not only the current metric data but also the data on the changes in node performance over a period of time in the past, which is also the basis for time series analysis and the construction of prediction models, thus helping financial institutions achieve effective resource management and reasonable capacity planning.

[0080] Through the automated data processing process, the accuracy, consistency and timeliness of the performance capacity metric data are ensured, providing reliable data support for the performance capacity management platform, and then enabling efficient and accurate analysis and automated resource management.

[0081] Optionally, in the method for expanding the node capacity provided in the first embodiment of this application, predicting the table capacity occupancy of N nodes based on a target prediction model to obtain a prediction result, including: calculating the table capacity occupancy of N nodes within a preset time period based on the target prediction model for historical metric data; processing the table capacity occupancy of N nodes within the preset time period to obtain a prediction result, and displaying the prediction result in a target device, where the prediction result at least includes: a first result and a second result, the first result is the result obtained by analyzing the table capacity occupancy of N nodes from a business perspective, and the second result is the result obtained by analyzing the table capacity occupancy of N nodes from a resource perspective.

[0082] In the first embodiment, a target prediction model is used to calculate historical metric data. The target prediction model predicts the table capacity occupancy and growth trend of N nodes within a future preset time period (such as the next few hours, days, or weeks) based on the patterns and trends learned from the historical metric data. Subsequently, the table capacity occupancy data predicted by the model can be further processed and analyzed to interpret the prediction result from different perspectives and provide an in-depth understanding of the impact on business and resources.

[0083] Exemplarily, the prediction result can be combined with the business scenario to analyze how the predicted table capacity occupancy affects the operation of a specific business. For example, determining that a certain business may face performance bottlenecks or storage shortages due to rapid growth of the table space, or whether the predicted growth of the table space is consistent with the growth trend of the transaction volume of a specific business, thus helping the business department prepare countermeasures in advance to ensure business continuity and service quality.

[0084] Exemplarily, the predicted table capacity occupancy can be analyzed from the perspective of resource management to evaluate whether the storage resources of the nodes in the cluster are sufficient to meet future demands, or which nodes' table space growth is approaching their storage limits and require advance capacity expansion planning. This helps the operation and maintenance system optimize resource allocation and avoid system performance degradation caused by resource bottlenecks.

[0085] The processed prediction result is displayed on a target device (such as the monitoring workstation of an operation and maintenance personnel, a mobile device, or other terminals). The display method of the prediction result may include charts, reports, or real-time views to enable operation and maintenance personnel to quickly understand the prediction information. The displayed prediction result can include both the business perspective (the first result) and the resource perspective (the second result).

[0086] By performing model prediction, data analysis, and result presentation, it can help the operation and maintenance system of financial institutions understand the future table capacity occupancy situation from multiple dimensions, so as to make more accurate decisions, ensure the reasonable allocation and efficient utilization of system resources, and avoid affecting the normal operation of the business due to improper capacity management. At the same time, through dynamic prediction and multi-angle analysis, it can better cope with the challenges brought by business growth and data fluctuations, and ensure the stability and scalability of the business system of financial institutions.

[0087] Optionally, in the method for expanding the capacity of nodes provided in Embodiment 1 of this application, expanding the table capacity of N nodes based on the prediction result includes: determining the target nodes to be expanded among the N nodes based on the prediction result every preset time period; determining the resource information of the cluster, where the resource information at least includes: the currently available resource information; determining the expansion strategy according to the resource information of the cluster and the target nodes; and expanding the table capacity of the target nodes according to the expansion strategy.

[0088] In Embodiment 1, the operation and maintenance system will analyze the table capacity occupancy of N nodes based on the previously obtained prediction result every preset time period (such as every half hour, daily, weekly, etc.). Through the prediction result, the operation and maintenance system can identify that the table space usage may approach or exceed its storage capacity in the future time period, that is, the target nodes to be expanded. Exemplarily, information such as "expected support time" in the prediction result will be used to determine whether the table capacity of the node needs to be increased immediately.

[0089] After determining the target nodes, it is necessary to determine the resource information of the entire cluster, and these resource information at least include: the currently available resource information, such as the free disk space of each node in the cluster, the status of storage devices, the usage of network bandwidth, etc. According to the resource information of the cluster and the specific situation of the target nodes, formulate an expansion strategy. The formulation of the strategy may include determining the expansion priority, selecting the type of resources to be expanded (such as disks, storage arrays, etc.), calculating the storage capacity that needs to be increased, planning the time window for the expansion operation (to reduce the impact on the business), selecting the expansion method (such as physically adding hard disks, adjusting the storage resource allocation, etc.). The expansion strategy needs to ensure the effective utilization of resources, and at the same time, consider the feasibility and cost-effectiveness of the expansion operation.

[0090] After determining the expansion strategy, expand the table capacity of the target nodes according to the expansion strategy. This may involve interacting with the storage management system to automatically perform operations such as disk space allocation, resource reconfiguration, or data migration. The execution of the expansion operation needs to ensure the minimum interference to the existing business and be completed as soon as possible to meet the predicted resource requirements.

[0091] Through automated capacity expansion, the efficiency and accuracy of the bank's performance capacity management have been greatly improved, the workload of operation and maintenance personnel has been reduced, improper resource allocation caused by human judgment errors has been avoided, and the dynamic optimization and adjustment of resources have been ensured to cope with the fluctuations and growth of business requirements.

[0092] Optionally, in the first embodiment, the schematic diagram of the structure of the data acquisition layer of this solution can be as Figure 3 shown. The business volume statistics system and application logs, that is, the above-mentioned first type of data, are collected through collection tools, and the key path information of TWS (Transaction Workload Scheduling), that is, the above-mentioned configuration information, is collected through self-developed software tools, and the RMFIII (Resource Management Facility) data, that is, the above-mentioned second type of data, is collected through self-developed software tools. The design of the data acquisition layer ensures the real-time and comprehensiveness of data, providing a solid foundation for subsequent data preprocessing and model construction. Figure 3 In addition to the data acquisition layer, it also shows the data display layer, unified portal, unified interface, and unified authentication system. In the data display layer, the system can provide an intuitive view of performance capacity management, facilitating managers to monitor and understand the performance status of the bank's IT system from different dimensions. The unified portal and unified interface ensure the consistency of the operation interface of the entire system, simplifying the user's operation process, while the unified authentication system enhances the security of the system, ensuring that only authenticated users can access sensitive data and perform critical operations. Performance capacity-related data interacts with the centralized monitoring system through an interface, and the centralized monitoring system instantaneously processes data and performance capacity-related events.

[0093] Optionally, in the first embodiment, the schematic diagram of the process of constructing and verifying the ARIMA prediction model of this solution can be as Figure 4As shown below. First, the observation value sequence N is introduced, representing the collected original performance metric data. Next, the ADF stationarity test is performed on the system to check whether the time series has stationary characteristics. If it is not stationary, differencing operations will be carried out until the sequence meets the stationarity requirements. The differenced sequence also needs to pass the white noise test to verify whether the model residuals conform to the white noise characteristics, which is one of the key indicators of model validity. After that, the maximum likelihood ratio is used for automatic order determination to determine the optimal order configuration of the ARIMA model. If the model order determination result does not meet the preset conditions, the ARIMA model construction process will be repeated until a suitable model order is found. Finally, the Ljung-Box model test is used to ensure the validity of the ARIMA prediction model, avoiding overfitting or underfitting of the model and being able to accurately reflect the performance trend of the bank IT system. If the model test passes, the prediction step will be initiated, and the constructed ARIMA model will be used to predict future performance metrics, providing a basis for resource planning and capacity expansion. After the prediction is completed, if the system requires further analysis, it will re-enter steps such as differencing operations and white noise tests to optimize and adjust the model until the analysis is completed, realizing the closed-loop management of the performance prediction of the bank IT system.

[0094] Optionally, in the first embodiment, the schematic flow diagram of the dynamic capacity expansion of the node table based on the ARIMA model in this solution can be as Figure 5 shown. Process various performance metrics and configuration information to generate historical metric data. And calculate the historical metric data based on the ARIMA model to construct a target prediction model. Introduce expert rules and compare the detection results of the expert rules with the prediction results of the target prediction model to ensure the rationality of the prediction results. If the prediction results generated by the target prediction model are inaccurate, the target prediction model needs to be optimized and reconstructed, and the model parameters will be adjusted according to the expert rules and data analysis results to improve the prediction accuracy. Immediately afterwards, the resource expansion module dynamically adjusts the resources of devices or nodes, including CPU, memory, and storage resources, based on the prediction results and the existing resource volume, ensuring that the resource requirements of the business system are met in a timely manner and avoiding the occurrence of performance bottlenecks. Finally, the resource expansion operation realizes the optimal allocation of resources, reduces the operating cost, and improves the stability and response speed of the banking business system.

[0095] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0096] In summary, the method for expanding the node capacity provided by the embodiments of the present application collects the index data of N nodes in the cluster, obtains the configuration information of the N nodes, and obtains historical index data, where N is a positive integer; constructs a prediction model based on the historical index data, and performs model verification on the constructed prediction model to obtain a target prediction model that passes the verification; predicts the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result; expands the table capacity of the N nodes based on the prediction result, solving the problem that in the related art, when predicting the future capacity occupancy of cluster nodes, the prediction result is easily affected by short-term services and is inaccurate, resulting in a decline in cluster performance.

[0097] By collecting the index data and configuration information of nodes in real time to construct historical index data, using the ARIMA model to construct a target prediction model, and performing model verification, the effectiveness of the model can be automatically verified, the reliability of the prediction result can be ensured, the uncertainty of human judgment can be reduced, the scientificity and accuracy of the prediction can be improved. At the same time, by using the target prediction model to predict the future table capacity usage of cluster nodes, automatically identifying the nodes that need to be expanded, and formulating corresponding expansion strategies, the operation and maintenance personnel can focus on more valuable work, that is, planning resources in advance, avoiding performance problems caused by resource bottlenecks, ensuring the timely replenishment of table capacity resources, and reducing the risk of system interruption or degradation caused by insufficient resources, thereby improving the business continuity and customer experience of financial institutions.

[0098] Embodiment 2

[0099] The embodiments of the present application also provide a device for expanding the node capacity. It should be noted that the device for expanding the node capacity in the embodiments of the present application can be used to execute the method for expanding the node capacity provided by the embodiments of the present application. The following introduces the device for expanding the node capacity provided by the embodiments of the present application.

[0100] According to the embodiments of the present application, there is also provided a device for implementing the above method for expanding the node capacity, as Figure 6 shown, the device includes:

[0101] Specifically, a collection unit 601 is configured to collect the index data of N nodes in the cluster, obtain the configuration information of the N nodes, and obtain historical index data, where N is a positive integer.

[0102] A construction unit 602 is configured to construct a prediction model based on the historical index data, and perform model verification on the constructed prediction model to obtain a target prediction model that passes the verification.

[0103] A prediction unit 603 is configured to predict the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result.

[0104] The capacity expansion unit 604 is used to expand the table capacity of N nodes based on the prediction result.

[0105] In the node capacity expansion device provided by the embodiment of the present application, the acquisition unit 601 acquires the index data of N nodes in the cluster, and obtains the configuration information of the N nodes to obtain historical index data, where N is a positive integer; the construction unit 602 constructs a prediction model based on the historical index data, and performs model verification on the constructed prediction model to obtain a target prediction model that passes the verification; the prediction unit 603 predicts the table capacity occupancy of N nodes based on the target prediction model to obtain a prediction result; the capacity expansion unit 604 expands the table capacity of N nodes based on the prediction result, solving the problem that in the related art, when predicting the future capacity occupancy of cluster nodes, the prediction result is easily affected by short-term services and is inaccurate, resulting in a decline in cluster performance.

[0106] By collecting the index data and configuration information of nodes in real time to construct historical index data, using the ARIMA model to construct a target prediction model, and performing model verification, the effectiveness of the model can be automatically verified, the reliability of the prediction result can be ensured, the uncertainty of human judgment can be reduced, the scientificity and accuracy of the prediction can be improved. At the same time, the future table capacity usage of cluster nodes is predicted through the target prediction model, and the nodes that need to be expanded are automatically identified, and corresponding capacity expansion strategies are formulated, enabling operation and maintenance personnel to focus on more valuable work, that is, planning resources in advance, avoiding performance problems caused by resource bottlenecks, ensuring the timely replenishment of table capacity resources, and reducing the risk of system interruption or degradation caused by insufficient resources, thereby improving the business continuity and customer experience of financial institutions.

[0107] Optionally, in the node capacity expansion device provided in the second embodiment of the present application, the above construction unit 602 includes: a first calculation subunit, configured to perform a difference operation on the historical index data to convert the historical index data into a stationary difference sequence and obtain a difference order; a test subunit, configured to perform a white noise test on the stationary difference sequence according to a preset statistic to obtain a first test result, where the first test result at least includes: a first variable; a first determination subunit, configured to determine a first parameter and a second parameter by using a maximum likelihood ratio model when the value of the first variable is less than or equal to a preset significance level; a second determination subunit, configured to determine the constructed prediction model according to the difference order, the first parameter, and the second parameter, and obtain a target prediction model when the constructed prediction model passes the model verification.

[0108] Optionally, in the node capacity expansion device provided in the second embodiment of the present application, the above-mentioned calculation subunit includes: a verification module, configured to perform a stationarity test on historical index data by using the unit root test method to obtain a second test result, where the second test result at least includes: a second variable; a first determination module, configured to determine that the historical index data of N nodes belongs to a stationary sequence and determine the difference order when the value of the second variable is less than or equal to a preset significance level; a calculation module, configured to determine that the historical index data of N nodes does not belong to a stationary sequence when the value of the first variable is greater than the preset significance level, and perform a difference operation on the historical index data until the historical index data belongs to a stationary sequence to obtain the difference order.

[0109] Optionally, in the node capacity expansion device provided in the second embodiment of the present application, the above-mentioned acquisition unit 601 includes: an acquisition subunit, configured to acquire index data of N nodes and acquire batch task information of N nodes, where the index data at least includes first-class data and second-class data, the first-class data represents index data acquired every first preset time period, and the second-class data represents index data different from the first-class data; a construction subunit, configured to obtain configuration information of N nodes and construct a topology graph of N nodes according to the configuration information; an update subunit, configured to update the topology graph of N nodes according to the index data of N nodes and the batch task information of N nodes to obtain historical index data.

[0110] Optionally, in the node capacity expansion device provided in the second embodiment of the present application, the above-mentioned update subunit includes: a second determination module, configured to determine the IP information of N nodes according to the configuration information of N nodes; a processing module, configured to perform data preprocessing on the index data of N nodes and the batch task information of N nodes respectively and store them in a target database to obtain processed index data; a third determination module, configured to determine the time series information of the processed index data; an update module, configured to update the topology graph of N nodes according to the processed index data, the time series information of the processed index data, and the IP information of N nodes to obtain historical index data.

[0111] Optionally, in the node capacity expansion device provided in the second embodiment of the present application, the above-mentioned prediction unit 603 includes: a second calculation subunit, configured to calculate the table capacity occupancy of N nodes within a preset time period based on a target prediction model; a processing subunit, configured to process the table capacity occupancy of N nodes within a preset time period to obtain a prediction result, and display the prediction result in a target device, where the prediction result at least includes: a first result and a second result, the first result is a result obtained by analyzing the table capacity occupancy of N nodes from a business perspective, and the second result is a result obtained by analyzing the table capacity occupancy of N nodes from a resource perspective.

[0112] Optionally, in the node capacity expansion device provided in the second embodiment of the present application, the above-mentioned expansion unit 604 includes: a third determination subunit, configured to determine a target node to be expanded among N nodes based on the prediction result every preset time period; a fourth determination subunit, configured to determine the resource information of the cluster, where the resource information at least includes: currently available resource information; a fifth determination subunit, configured to determine an expansion policy based on the resource information of the cluster and the target node; an expansion subunit, configured to expand the table capacity of the target node according to the expansion policy.

[0113] It should be noted here that the above-mentioned acquisition unit 601, construction unit 602, prediction unit 603, and expansion unit 604 correspond to steps S201 to S204 in Embodiment 1. The two modules and the corresponding steps have the same implemented examples and application scenarios, but are not limited to the content disclosed in the above-mentioned Embodiment 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102 for node capacity expansion,..., 102n). The above-mentioned modules may also be part of a device and can run in the computer terminal 10 provided in Embodiment 1.

[0114] Embodiment 3

[0115] An embodiment of the present application may provide an electronic device. Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present application. As Figure 7 shown, the electronic device may include: one or more ( Figure 7 only one is shown in the figure) processors 702, a memory 704, a storage controller, and a peripheral interface, where the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0116] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned methods. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely provided with respect to the processor, and these remote memories can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0117] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: collect the metric data of N nodes in the cluster, and obtain the configuration information of the N nodes to obtain historical metric data, where N is a positive integer; build a prediction model based on the historical metric data, and perform model verification on the built prediction model to obtain a target prediction model that passes the verification; predict the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result; expand the table capacity of the N nodes based on the prediction result.

[0118] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: build a prediction model based on the historical metric data, and perform model verification on the built prediction model to obtain a target prediction model that passes the verification, including: performing a difference operation on the historical metric data to convert the historical metric data into a stationary difference sequence to obtain the difference order; performing a white noise test on the stationary difference sequence according to a preset statistic to obtain a first test result, where the first test result at least includes: a first variable; in the case where the value of the first variable is less than or equal to a preset significance level, determine a first parameter and a second parameter by using a maximum likelihood ratio model; determine the built prediction model according to the difference order, the first parameter, and the second parameter, and obtain the target prediction model in the case where the built prediction model passes the model verification.

[0119] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: perform a difference operation on the historical indicator data, convert the historical indicator data into a stationary difference sequence, and obtain the difference order, including: performing a stationarity test on the historical indicator data using the unit root test method to obtain a second test result, where the second test result at least includes: a second variable; in the case where the value of the second variable is less than or equal to a preset significance level, determining that the historical indicator data of N nodes belongs to a stationary sequence and determining the difference order; in the case where the value of the first variable is greater than the preset significance level, determining that the historical indicator data of N nodes does not belong to a stationary sequence, and performing a difference operation on the historical indicator data until the historical indicator data belongs to a stationary sequence to obtain the difference order.

[0120] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: collect the indicator data of N nodes in the cluster and obtain the configuration information of the N nodes to obtain historical indicator data, including: collecting the indicator data of N nodes and collecting the batch task information of N nodes, where the indicator data at least includes a first type of data and a second type of data, the first type of data represents the indicator data collected every first preset time period, and the second type of data represents indicator data different from the first type of data; obtain the configuration information of N nodes and construct a topology map of N nodes according to the configuration information; update the topology map of N nodes according to the indicator data of N nodes and the batch task information of N nodes to obtain historical indicator data.

[0121] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: update the topology map of N nodes according to the indicator data of N nodes and the batch task information of N nodes to obtain historical indicator data, including: determining the IP information of N nodes according to the configuration information of N nodes; respectively perform data preprocessing on the indicator data of N nodes and the batch task information of N nodes and store them in the target database to obtain the processed indicator data; determine the time series information of the processed indicator data; update the topology map of N nodes according to the processed indicator data, the time series information of the processed indicator data, and the IP information of N nodes to obtain historical indicator data.

[0122] The processor can call the information and application programs stored in the memory through a transmission device to perform the following steps: Predict the table capacity occupancy of N nodes based on a target prediction model to obtain a prediction result, including: calculating the table capacity occupancy of N nodes within a preset time period based on the target prediction model for historical metric data; processing the table capacity occupancy of N nodes within the preset time period to obtain a prediction result, and displaying the prediction result in a target device, where the prediction result at least includes: a first result and a second result, the first result is the result obtained by analyzing the table capacity occupancy of N nodes from a business perspective, and the second result is the result obtained by analyzing the table capacity occupancy of N nodes from a resource perspective.

[0123] The processor can call the information and application programs stored in the memory through a transmission device to perform the following steps: Expand the table capacity of N nodes based on the prediction result, including: determining target nodes to be expanded among N nodes based on the prediction result every preset time duration; determining the resource information of the cluster, where the resource information at least includes: currently available resource information; determining an expansion strategy based on the resource information of the cluster and the target nodes; and expanding the table capacity of the target nodes according to the expansion strategy.

[0124] By adopting the embodiment of the present application, a method for expanding the capacity of nodes is provided. Historical metric data is obtained by collecting the metric data of N nodes in a cluster and acquiring the configuration information of the N nodes, where N is a positive integer; a prediction model is constructed based on the historical metric data, and the constructed prediction model is subjected to model verification to obtain a target prediction model that passes the verification; the table capacity occupancy of N nodes is predicted based on the target prediction model to obtain a prediction result; and the table capacity of N nodes is expanded based on the prediction result, thereby solving the technical problem that when predicting the future capacity occupancy of cluster nodes, the prediction result is easily affected by short-term services and is inaccurate, resulting in a decline in cluster performance.

[0125] By collecting the metric data and configuration information of nodes in real time to construct historical metric data, using an ARIMA model to construct a target prediction model, and performing model verification, the effectiveness of the model can be automatically verified, the reliability of the prediction result can be ensured, the uncertainty of human judgment can be reduced, the scientificity and accuracy of the prediction can be improved. At the same time, by predicting the future table capacity usage of cluster nodes through the target prediction model, automatically identifying the nodes that need to be expanded, and formulating corresponding expansion strategies, the operation and maintenance personnel can focus on more valuable work, that is, planning resources in advance, avoiding performance problems caused by resource bottlenecks, ensuring the timely replenishment of table capacity resources, and reducing the risk of system interruption or degradation caused by insufficient resources, thereby enhancing the business continuity and customer experience of financial institutions.

[0126] Those of ordinary skill in the art can understandFigure 7 The structure shown is only schematic. The electronic device can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a personal digital assistant, and mobile Internet devices (MID), a PAD, and other terminal devices. Figure 7 It does not limit the structure of the above-mentioned electronic device. For example, the electronic device may further include more or fewer components (such as a network interface, a display device, etc.) than those shown Figure 7 in the figure, or have a different configuration from that shown Figure 7 in the figure.

[0127] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program. The program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0128] Embodiment 4

[0129] An embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the above storage medium can be used to store the program code executed by the method for expanding the node capacity provided in the first embodiment above.

[0130] Optionally, in this embodiment, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0131] The present application also provides a computer program product, which is adapted to execute a program for the steps of the method for expanding the node capacity when executed on a data processing device.

[0132] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0133] In the above embodiments of the present application, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0135] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0137] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0138] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for expanding node capacity, characterized in that: include: Collecting indicator data of N nodes in the cluster and obtaining configuration information of the N nodes to obtain historical indicator data, where N is a positive integer; Constructing a prediction model based on the historical indicator data, and performing a model test on the constructed prediction model to obtain a target prediction model that passes the test; Predicting the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result; The table capacity of the N nodes is expanded based on the prediction result.

2. The method according to claim 1, characterized in that A prediction model is constructed based on the historical indicator data, and a model test is performed on the constructed prediction model to obtain a target prediction model that passes the test, including: Performing a differential operation on the historical indicator data, converting the historical indicator data into a stable differential sequence, and obtaining a differential order; Performing a white noise test on the stationary difference sequence according to a preset statistic to obtain a first test result, wherein the first test result at least includes: a first variable; When the value of the first variable is less than or equal to a preset significance level, a maximum likelihood ratio model is used to determine the first parameter and the second parameter; The constructed prediction model is determined according to the difference order, the first parameter and the second parameter, and the target prediction model is obtained when the constructed prediction model passes the model test.

3. The method according to claim 2, characterized in that Performing a differential operation on the historical indicator data to convert the historical indicator data into a stable differential sequence to obtain a differential order, including: A unit root test method is used to perform a stationarity test on the historical indicator data to obtain a second test result, wherein the second test result at least includes: a second variable; When the value of the second variable is less than or equal to a preset significance level, determining that the historical indicator data of the N nodes belongs to a stationary sequence, and determining the difference order; When the value of the first variable is greater than the preset significance level, it is determined that the historical indicator data of the N nodes do not belong to a stationary sequence, and a differential operation is performed on the historical indicator data until the historical indicator data belongs to a stationary sequence to obtain the differential order.

4. The method according to claim 1, characterized in that: Collect the indicator data of N nodes in the cluster, obtain the configuration information of the N nodes, and obtain historical indicator data, including: Collecting indicator data of the N nodes and collecting batch task information of the N nodes, wherein the indicator data includes at least a first type of data and a second type of data, the first type of data represents indicator data collected every first preset time period, and the second type of data represents indicator data different from the first type of data; Acquire configuration information of the N nodes, and construct a topology map of the N nodes according to the configuration information; The topology map of the N nodes is updated according to the indicator data of the N nodes and the batch task information of the N nodes to obtain the historical indicator data.

5. The method according to claim 4, characterized in that Updating the topology map of the N nodes according to the indicator data of the N nodes and the batch task information of the N nodes to obtain the historical indicator data includes: Determining IP information of the N nodes according to the configuration information of the N nodes; Preprocess the index data of the N nodes and the batch task information of the N nodes respectively, and store them in a target database to obtain processed index data; Determining time series information of the processed indicator data; The topological map of the N nodes is updated according to the processed indicator data, the time series information of the processed indicator data and the IP information of the N nodes to obtain the historical indicator data.

6. The method according to claim 1, characterized in that Predicting the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result includes: Calculate the historical indicator data based on the target prediction model to obtain the table capacity occupancy of the N nodes within a preset time period; The table capacity occupancy of the N nodes within the preset time period is processed to obtain the prediction result, and the prediction result is displayed in the target device, wherein the prediction result at least includes: a first result and a second result, the first result is a result obtained by analyzing the table capacity occupancy of the N nodes from a business perspective, and the second result is a result obtained by analyzing the table capacity occupancy of the N nodes from a resource perspective.

7. The method according to claim 1, characterized in that Expanding the table capacity of the N nodes based on the prediction result includes: Determine a target node to be expanded among the N nodes based on the prediction result after each preset time period; Determine resource information of the cluster, wherein the resource information at least includes: currently available resource information; Determining a capacity expansion strategy based on the resource information of the cluster and the target node; The table capacity of the target node is expanded according to the expansion strategy.

8. A device for expanding node capacity, characterized in that: include: A collection unit, used to collect indicator data of N nodes in the cluster, and obtain configuration information of the N nodes to obtain historical indicator data; A construction unit, used to construct a prediction model based on the historical indicator data, and perform a model test on the constructed prediction model to obtain a target prediction model that passes the test; A prediction unit, configured to predict the table capacity occupancy of the N nodes based on the target prediction model to obtain a prediction result; A capacity expansion unit is used to expand the table capacity of the N nodes based on the prediction result.

9. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 7 when running.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images