Data processing method and device

Abstract

The embodiments of this specification provide a data processing method and device, wherein the data processing method includes obtaining operating status parameters of a first data node, and extracting parameter features of the operating status parameters based on preset feature extraction rules; inputting the parameter features into a data node decision model to obtain a data node decision result corresponding to the parameter features within a preset time length; sending a data node adjustment request to a data node resource pool based on the data node decision result; determining a target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and performing data processing based on the target data node.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly to a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a computing device, and a computer-readable storage medium. Background Art

[0002] In the Internet era, where the demand for real-time analysis of big data is increasing, the size of user data will also fluctuate with their own projects. For example, the data volume will explode during the Double Eleven project. Or many e-commerce projects will have many flash sales projects, and the pressure on the database will also increase sharply during the flash sales activities. For cloud databases, user projects are diverse, and the fluctuations of the projects are also different. For the flash sales projects of Double Eleven, the pressure on the database will change with the changes in the flash sales projects. For example, when the flash sales activities are in progress, the database may need to be expanded to ensure the normal data processing of the database; and after the flash sales project ends, the database needs to be reduced to avoid wasting database resources.

[0003] Therefore, how to adaptively expand or shrink the database has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the present specification provides a data processing method. One or more embodiments of the present specification also relate to a data processing device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a data processing method is provided, including:

[0006] Acquire an operating status parameter of the first data node, and extract parameter features of the operating status parameter based on a preset feature extraction rule;

[0007] Input the parameter feature into the data node decision model to obtain the data node decision result corresponding to the parameter feature within a preset time length;

[0008] Sending a data node adjustment request to a data node resource pool based on the data node decision result, wherein the data node resource pool is composed of a first data node and a second data node;

[0009] Determine a target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and perform data processing based on the target data node.

[0010] According to a second aspect of an embodiment of this specification, there is provided a data processing device, including:

[0011] A feature extraction module is configured to obtain an operating state parameter of the first data node and extract parameter features of the operating state parameter based on a preset feature extraction rule;

[0012] A node strategy determination module is configured to input the parameter characteristics into a data node decision model to obtain a data node decision result corresponding to the parameter characteristics within a preset time period;

[0013] A request sending module is configured to send a data node adjustment request to a data node resource pool based on the data node decision result, wherein the data node resource pool is composed of a first data node and a second data node;

[0014] The node adjustment module is configured to determine a target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and perform data processing based on the target data node.

[0015] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0016] Memory and processor;

[0017] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, the steps of the data processing method are implemented.

[0018] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the data processing method are implemented.

[0019] An embodiment of the present specification implements a data processing method and device, wherein the data processing method includes obtaining the operating status parameters of the first data node, and extracting the parameter features of the operating status parameters based on a preset feature extraction rule; inputting the parameter features into a data node decision model to obtain a data node decision result corresponding to the parameter features within a preset duration; sending a data node adjustment request to a data node resource pool based on the data node decision result; determining the target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and performing data processing based on the target data node. Specifically, the data processing method predicts the data node decision result within a future duration based on the parameter features of the operating status parameters of the current data node and a pre-established data node decision model, and adjusts the number of data nodes within a future duration based on the data node decision result. On the basis of avoiding frequent adjustments to the number of data nodes, the expansion and contraction of data nodes can be effectively achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of a specific scenario in which a data processing method provided by an embodiment of this specification is applied to a distributed cloud database;

[0021] Figure 2 is a flow chart of a data processing method provided by an embodiment of this specification;

[0022] Figure 3 It is a specific application diagram of a data node decision model in a data processing method provided by an embodiment of this specification;

[0023] Figure 4 This is a flow chart of the application of a data processing method provided by an embodiment of this specification in a distributed cloud database;

[0024] Figure 5 is a structural schematic diagram of a data processing device provided by an embodiment of this specification;

[0025] Figure 6 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0026] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0027] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0028] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0029] First, the terms involved in one or more embodiments of this specification are explained.

[0030] Distributed database: A distributed database is a logically identical database composed of multiple physically dispersed databases connected by a computer network, and is managed by a unified database management system (distributed database management system).

[0031] OSS: Object Storage Service (OSS) is a massive, secure, low-cost, and highly reliable cloud storage service. Its data durability is designed to be no less than 99.9999999999% (12 nines), and its service availability (or project continuity) is designed to be no less than 99.995%.

[0032] Coordinate: Coordinator is the control node of the distributed cloud database, used to manage and schedule data nodes and distribute data.

[0033] In practical applications, in order to ensure that in a flash sale project, the database (the database includes multiple data nodes) can be expanded or reduced in capacity to adapt to the database's processing of data during the flash sale project. For example, the operating status parameters of each data node can be monitored in real time by means of threshold monitoring. When it is determined that the operating status parameters of the data node are greater than or equal to the preset parameter threshold, the data node is expanded to meet the data processing requirements; and when the operating status parameters of the data node are less than the preset parameter threshold, the data node is reduced in capacity to avoid wasting data resources; however, this method will make the expansion and contraction of data nodes very frequent, and frequent expansion and contraction of data nodes will greatly affect data processing performance. Therefore, based on this solution, the following improvements are made in the embodiments of this specification.

[0034] In this specification, a data processing method is provided. This specification also relates to a data processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0035] See also Figure 1 , Figure 1 A schematic diagram of a specific scenario in which a data processing method provided according to an embodiment of the present specification is applied to a distributed cloud database is shown.

[0036] Figure 1 The system includes a coordinator 102 (Coordinate), a plurality of data nodes 104, a resource pool 106 including the plurality of data nodes 104, and an object storage service platform 108 (OSS).

[0037] Among them, the coordinator 102 is the control node of the distributed cloud database, which is mainly used to manage and schedule the data node 104 and distribute data. The data node 104 is the core of the distributed cloud database, which is mainly used to calculate and store data. For the elastically scalable distributed cloud database provided in this specification, the shared storage is the key to elastic contraction. The data is readable after being written, that is, the data will be saved in the object storage service platform 108 after being written, and the data will not be unreadable due to the shrinkage of the data node 104; in addition, the data node 104 will asynchronously index the real-time data and upload it to the object storage service platform 108 in a package. The resource pool 106 is the management and control part, which is used to manage the available resources (data node 104) of the distributed cloud database, as well as the interactive scheduling between the expansion and contraction of the distributed cloud database and the coordinator 102.

[0038] In specific implementation, the distributed cloud database is started, the coordinator 102 is started, the coordinator 102 holds the heartbeat and operation status information of all data nodes 104, and is responsible for the distribution and routing of all queries / writes (commands); at the same time, the scheduling tasks of the data nodes 104 are also the responsibility of the coordinator 102. The coordinator 102 is started, and then after all the data nodes 104 are started, the heartbeat detection and data transmission are always performed with the data nodes 104. The data nodes 104 are started, and at this time, the data nodes 104 under the default configuration are started. If the default configuration is 64 nodes, 64 data nodes 104 are started. The data nodes 104 can be expanded or reduced according to the actual application later, that is, the data nodes 104 are added or deleted.

[0039] In actual applications, after the distributed cloud database is started, the coordinator 102 is started, and the data node 104 is started, the data processing method provided in the embodiment of this specification is applied to the distributed cloud database. At this time, the coordinator 102 obtains the operating status information of the current data node 104, that is, the operating status parameters; then the operating status parameters of the current data node 104 are analyzed, for example, based on a preset threshold analysis or using a pre-established model analysis; when the analysis determines that the current data node 104 needs to be expanded, a capacity expansion request is sent to the resource pool 106, and an appropriate number of data nodes 104 are scheduled from the resource pool 106 to perform data processing together with the current data node 104; and when the analysis determines that the current data node 104 needs to be reduced, it is analyzed which data node 104 needs to be reduced, and a reduction request is also sent to the resource pool 106, and then the data processing service of the data node 104 that needs to be reduced is stopped, and the resource pool 106 is waited for the data node 104 that needs to be reduced to be recovered; wherein, data processing includes but is not limited to data query, data writing, etc.

[0040] In order to ensure that data loss does not occur when the data node 104 is expanded or reduced in capacity, the data node 104 will asynchronously index the real-time data and package it for transmission to the object storage service platform 108 during its operation, so that data can be read as soon as it is written, which can meet the elastic shrinkage of the subsequent distributed cloud database.

[0041] See also Figure 2 , Figure 2 A flow chart of a data processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0042] Step 202: Acquire the operating status parameters of the first data node, and extract parameter features of the operating status parameters based on preset feature extraction rules.

[0043] The first data node is a running data node. When the data processing method is applied to a distributed cloud database, the first data node can be understood as a plurality of running data nodes.

[0044] Specifically, during the operation of the distributed cloud database, the coordinator obtains the operating status parameters of the first data node, and extracts parameter features of the operating status parameters based on preset feature extraction rules; wherein the operating status parameters include but are not limited to CPU (Central Processing Unit) utilization, current IOPS (Input / Output Operations Per Second, input and output or read and write times per second), current QPS (query rate per second), the number of connections of the current data node, current disk utilization, and the like.

[0045] Among them, the preset feature extraction rules can be set according to actual applications, and this manual does not impose any restrictions on this, such as extracting parameter features such as the mean, maximum, minimum, median and 90th percentile of the operating status parameters.

[0046] For example, if the operating status parameters are CPU usage, current IOPS, current QPS, the number of connections of the current data node, and the current disk usage; the preset feature extraction rule is to extract parameter features such as the mean, maximum, minimum, median, and 90th percentile of the operating status parameters; then the operating status parameters of the first data node are obtained, and the parameter features of the operating status parameters are extracted based on the preset feature extraction rules. It can be understood that the CPU usage, current IOPS, current QPS, the number of connections of the current data node, and the current disk usage of each first data node are obtained, and then based on the preset feature extraction rules, the parameter features of the operating status parameters are extracted. The rule extracts the mean, maximum, minimum, median and 90th percentile of the CPU utilization of all data nodes; the mean, maximum, minimum, median and 90th percentile of the current IOPS of all data nodes; the mean, maximum, minimum, median and 90th percentile of the current QPS of all data nodes; the mean, maximum, minimum, median and 90th percentile of the number of connections of the current data nodes of all data nodes; the mean, maximum, minimum, median and 90th percentile of the current disk utilization of all data nodes as parameter characteristics of the operating status parameters of the first data node.

[0047] In addition, the obtaining of the operating status parameter of the first data node includes:

[0048] A first data node is determined, and when all first data nodes are started, an operating status parameter of the first data node is obtained.

[0049] Specifically, when acquiring the running status parameters, it is necessary to first determine which data nodes' running status parameters need to be acquired, and then, when it is determined that all these data nodes are started, the running status parameters of these data nodes are acquired periodically.

[0050] In the embodiments of this specification, the operating status parameters of the first data nodes are obtained only when all first data nodes (i.e., data nodes configured to perform data processing) are started, so as to ensure that more accurate data node decision results can be obtained based on the obtained operating status parameters.

[0051] In a specific implementation, before obtaining the operating status parameter of the first data node, the method further includes:

[0052] Acquire, at a preset time interval, the first data node, historical parameter characteristics of the historical operating state parameters within the preset time period, and the data node decision result corresponding to the historical parameter characteristics;

[0053] Using the historical parameter features as training data, using the data node decision results as labels, and forming training samples based on the training data and the labels;

[0054] Inputting the training samples into an initial data node decision model to obtain network parameters of the initial data node decision model;

[0055] The initial data node decision model is trained based on the network parameters to obtain a data node decision model.

[0056] The preset time interval and the preset duration can be set according to actual application, for example, the preset time interval and the preset duration are set to 1 hour.

[0057] In practical applications, the first data node can be understood as the data node currently in operation. During the actual operation process, the data node will extract parameter features of its operating status parameters in real time, and then store the extracted parameter features. When training the model, the historical parameter features of the historical operating status parameters newly stored by the data node can be used to train and update the model to maintain the stability of the model.

[0058] During specific implementation, the historical operating status parameters of the first data node are substantially the same as the current operating status parameters, and are not described in detail here.

[0059] Taking the preset time interval and preset duration of 1 hour as an example, the historical parameter characteristics of all historical operating status parameters of the first data node (i.e., the currently running data node) within one hour before the current moment are regularly obtained every hour, as well as the data node decision result corresponding to the historical parameter characteristics, wherein the data node decision result can be understood as whether to expand or shrink the data node, and the number of expanded data nodes or the number of shrunk data nodes, etc., and the historical parameter characteristics include the time point at which the data node was expanded or shrunk (for example, it was expanded in the 6th minute within the 1 hour) and the characteristic value at the time point (i.e., the mean, maximum, minimum, median and / or 90th percentile of each operating status parameter, etc.), and the labels are expansion and the number of expanded data nodes, shrinkage and the number of shrunk data nodes.

[0060] Then, the historical parameter features are used as training data, and the data node decision results corresponding to the historical parameter features are used as labels. Training samples are formed based on the training data and the labels, that is, each training sample consists of training data and the labels corresponding to the training data. For example, a training sample includes: training data: the first data node was expanded at 11:20 within the hour from 11:00 to 12:00. At 11:20, the characteristic value of the operating status parameter of the first data node was A; the label corresponding to the training data: expansion, and the number of expanded data nodes is 2.

[0061] The training samples are then input into the initial data node decision model to obtain the network parameters of the initial data node decision model, and finally the initial data node decision model is trained based on the network parameters to obtain the data node decision model.

[0062] In specific implementation, in order to enable the data processing method provided in the embodiment of this specification to more accurately determine whether the data node needs to be expanded or reduced within a preset time period based on the operating status parameters of the current data node, it is necessary to collect the operating status parameters of the currently running data nodes in real time, and then establish a data node decision model based on these collected historical operating status parameters of the currently running data nodes, so as to accurately obtain a strategy for whether the data node needs to be expanded or reduced.

[0063] In actual use, the coordinator obtains the offline operating status parameters of the currently running data node (i.e., the first data node), such as CPU usage, current IOPS, current QPS, current number of connections, current disk usage, etc.; and then summarizes the operating status parameters of all currently running data nodes. Since the distributed cloud database system is very large, and the monitoring data indicators (i.e., operating status parameters) detected each time by the scheduled scheduling task (schedule) cannot all be held in the database, otherwise it will consume a lot of resources. Therefore, the hourly statistical data is used as a feature for preservation, that is, the characteristic values ​​of the operating status parameters of the data nodes counted every hour are saved. In order to reduce the computational complexity of the data node decision model, the main features can be extracted from the parameter features corresponding to the operating status parameters as training data for subsequent data node decision model training, such as extracting parameter features such as the mean, maximum, minimum, median and / or 90th percentile in the parameter features corresponding to the operating status parameters.

[0064] Specifically, the coordinator obtains the offline data of the currently running data nodes (such as data node system data and data node traffic data, i.e., operating status parameters) to prepare the model training samples, and then uses the parameter characteristics of the operating status parameters of the data nodes obtained regularly every hour and the corresponding data node policy results to train the data node decision model. For example, the parameter characteristics of the operating status parameters of the data nodes in the past hour and the corresponding data node policy results are obtained every 2 hours.

[0065] In specific implementation, the data node decision model can be an SVM multi-classifier model. The parameter characteristics of the operating status parameters of the data nodes obtained regularly every hour and the corresponding data node strategy results are used as training samples to form a training data set. The SVM multi-classifier model is trained to obtain the final data node strategy model. The principle of the SVM multi-classifier model is as follows:

[0066] Given a linearly separable training data set, the hyperplane trained by the training data set is:

[0067] w * ·x+b * =0

[0068] Among them, w is the normal vector, x is the eigenvector, and b is the displacement;

[0069] The distance from the data point to the hyperplane is:

[0070]

[0071] The function distance from the sample point (xi, yi) to the hyperplane (w, b) is:

[0072]

[0073] The geometric distance from the sample point (xi, yi) to the hyperplane (w, b) is:

[0074]

[0075] Among them, x i is the ith feature vector, and yi is the class label.

[0076] The minimum geometric distance is:

[0077] γ=min i =1,...nγ i

[0078] In order to better divide the hyperplane, find a set of w, b to maximize these intervals, that is:

[0079]

[0080] Through Lagrange derivation, we can get:

[0081]

[0082] The constraints are:

[0083]

[0084] Among them, α is the Lagrange multiplier and is greater than 0;

[0085] By substituting the characteristics of all data nodes and the corresponding data node decision results (i.e., the training data set) into the above-mentioned SVM optimization function, the network parameters of the SVM are obtained through the above-mentioned SVM operation, and then the SVM is adjusted through the network parameters to obtain the final data node decision model.

[0086] In an embodiment of the present specification, the coordinator will periodically obtain historical parameter characteristics of the historical operating status parameters of the currently running data node in the past period of time and the corresponding data node decision results, and train and update the data node decision model so that accurate data node decision results within a preset period of time can be obtained through the data node decision model in the future. Based on the data node decision results, the expansion and contraction of the data node within the preset period of time can be determined to avoid frequent adjustments to the data nodes, which affects the data processing performance of the database.

[0087] In another embodiment of the present specification, after obtaining the data node decision model, the method further includes:

[0088] Storing the data node decision model in an object storage service platform;

[0089] Accordingly, the step of inputting the parameter features into the data node decision model includes:

[0090] The data node decision model is called from the object storage service platform, and the parameter characteristics are input into the data node decision model.

[0091] Specifically, after the data node decision model is obtained through training, the data node decision model is stored in the object storage service platform (OSS). When used subsequently, the coordinator can obtain the data node decision model from the object storage service platform and obtain the data node decision results of the data node.

[0092] In actual applications, in order to avoid the loss of the data node decision model when the coordinator fails, the data node decision model is constructed and stored in the object storage service platform after each update. When the coordinator is used subsequently, it only needs to call the latest data node decision model each time, which greatly improves the user experience.

[0093] In another embodiment of the present specification, in the cold start phase, that is, when the data processing system is started for the first time, the data nodes are not running yet, and the data node decision model in the coordinator is not generated, in order to ensure the scalability of the data nodes to support project needs, the expansion and contraction of the data nodes can be adjusted based on the preset parameter thresholds. The specific implementation method is as follows:

[0094] After obtaining the operating status parameter of the first data node, the method further includes:

[0095] If there is no data node decision model and no historical operating status parameters of the first data node, the historical operating status parameters are compared with a preset parameter threshold, and a data node adjustment request is sent to the data node resource pool based on the comparison result.

[0096] The preset parameter threshold can be set according to actual application, and this specification does not impose any limitation on this.

[0097] Specifically, when the data node decision model is not constructed in the coordinator and there are no historical operating status parameters for the currently running data nodes, the operating status parameters of the currently running nodes are compared with the preset parameter threshold. If they are greater than or equal to the preset parameter threshold, a data node expansion request is sent to the data node resource pool, and the data node expansion request carries the number of data nodes to be expanded according to the higher operating status parameters, for example, 3 data nodes need to be added; if they are less than the preset parameter threshold, a data node reduction request is sent to the data node resource pool, and the data node reduction request carries the number of data nodes to be reduced according to the lower operating status parameters, for example, 3 data nodes need to be deleted.

[0098] In addition, after obtaining the operating status parameter of the first data node and extracting the parameter feature of the operating status parameter based on a preset feature extraction rule, the method further includes:

[0099] Storing the parameter characteristics of the operating status parameters in the object storage service platform;

[0100] Accordingly, the acquisition of the historical parameter characteristics of the historical operating state parameters of the first data node within the preset time period according to the preset time interval includes:

[0101] Obtain historical parameter characteristics of historical operating status parameters of the first data node within the preset time period from the object storage service platform at preset time intervals.

[0102] In actual applications, in order to ensure that the coordinator can regularly obtain the parameter characteristics of the historical operating state parameters of the first data node, after the coordinator obtains the current operating state parameters of the first data node each time, the parameter characteristics required for model update are extracted from the current operating state parameters and stored in the object storage service platform. The coordinator will also regularly obtain the parameter characteristics of the historical operating state parameters of the first data node from the object storage service platform and regularly update the data node decision model to ensure the prediction accuracy of the data node decision model.

[0103] Step 204: Input the parameter feature into the data node decision model to obtain the data node decision result corresponding to the parameter feature within a preset time period.

[0104] The data node decision results include data node expansion and the number of data node expansions, as well as data node reduction and the number of data node reductions.

[0105] Specifically, the coordinator will obtain the operating status parameters of the currently running data node once every period of time (for example, 5 seconds), and then extract the parameter characteristics of the operating status parameters based on the preset extraction rules, and obtain the data node decision result of the currently running data node based on the parameter characteristics, that is, whether the currently running data node needs to be expanded or reduced, as well as the number of data nodes to be expanded and the number of data nodes to be reduced.

[0106] See also Figure 3 , Figure 3 A specific application schematic diagram of a data node decision model in a data processing method provided by an embodiment of this specification is shown.

[0107] Figure 3 In the model training phase:

[0108] Step 1: The coordinator periodically obtains historical data node system data and historical data node process data (i.e., historical operating status parameters of the first data node) of the first data node within a preset time period;

[0109] Step 2: Based on the acquired historical operating status parameters of the first data node, an SVM multi-classifier (ie, a data node decision model) is trained;

[0110] Step 3: Store the data node decision model in the object storage service platform.

[0111] Model application phase:

[0112] Step 4: The coordinator calls the trained data node decision model from the object storage service platform.

[0113] Step 5: The coordinator periodically obtains the data node system data and data node process data of the first data node (i.e., the current operating status parameters of the first data node), and then inputs the current operating status parameters of the first data node into the data node decision model to obtain the data node decision result corresponding to the current operating status parameters of the first data node.

[0114] In actual applications, the coordinator obtains the parameter characteristics of the operating status parameters of the currently running data node once at regular intervals, and then obtains the data node decision results corresponding to the parameter characteristics within a preset time period based on the data node decision model. For example, the coordinator obtains the parameter characteristics of the operating status parameters of the currently running data node once every 5 seconds, and then obtains the data node decision results corresponding to the parameter characteristics within 1 hour based on the data node decision model, that is, the first data node needs to be expanded or reduced in the next hour, the number of data nodes to be expanded, and the number of data nodes to be reduced. That is, the data node decision model will internally determine at which point in the next hour the first data node needs to be expanded and the number of data node expansions, at which point in the next hour the first data node needs to be reduced and the number of data nodes reduced, and then comprehensively consider the expansion and reduction situations to predict the final adjustment strategy of the first data node in the next hour. For example, the query rate of the first data node is now 12w, and it will drop to 8w in 20 minutes, and 4 data nodes need to be reduced. However, the query rate rises to 12w in 30 minutes, and 4 data nodes need to be expanded. Then the final prediction result of the data node decision model may be that the first data node will not be expanded or reduced in the next hour, so as to avoid frequent additions and deletions of the first data node.

[0115] However, the data node decision model predicts the expansion or contraction of the first data node, and in actual applications, there may be a sharp increase in the operating status parameters in a certain time period within the predicted time period. Therefore, the coordinator will periodically obtain the operating status parameters of the first data node, and then determine the node policy results of the first data node based on the operating status parameters of the first data node. For example, the operating status parameter of the first data node is 12W query rate, the current number of first data nodes is 12, and 4 data nodes need to be reduced in capacity; then the data node policy results determined based on the operating status parameters are matched with the data node policy results obtained based on the data node decision model. For example, the data node policy results obtained based on the data node decision model are that the data node needs to be expanded by 2 data nodes in the next hour. Then it can be determined that if the current data node policy result determined based on the operating status parameters shrinks the first data node, it may need to be expanded later, so no adjustment is made to the first data node. If the node strategy result of the first data node determined based on the operating status parameters of the first data node is: 4 data nodes need to be expanded; and the data node strategy result obtained based on the data node decision model is that the data node needs to be expanded by 2 data nodes within the next hour; then it can be determined that the first data node is insufficient to support the current data processing, and a data node adjustment request is sent to the data node resource pool based on the data node strategy result determined based on the operating status parameters, and the specific implementation method is as follows:

[0116] The sending a data node adjustment request to a data node resource pool based on the data node decision result includes:

[0117] When the data node policy result determined based on the operating status parameters does not match the data node policy result obtained based on the data node decision model, a data node adjustment request is sent to the data node resource pool based on the data node policy result determined based on the operating status parameters.

[0118] In the embodiments of the present specification, after obtaining the data node policy result based on the data node policy model, in order to avoid the occurrence of emergencies, the coordinator will still periodically obtain the operating status parameters of the first data node to judge the data node policy result, so as to ensure that when a certain indicator of the first data node suddenly increases, it will not cause the first data node to run at a high load.

[0119] Step 206: Send a data node adjustment request to a data node resource pool based on the data node decision result, wherein the data node resource pool consists of a first data node and a second data node.

[0120] Specifically, after obtaining the data node decision result, a data node adjustment request is sent to the data node resource pool based on the data node decision result.

[0121] Step 208: Determine a target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and perform data processing based on the target data node.

[0122] Among them, the data node decision results include two situations, one is to expand the data node, and the other is to reduce the data node. Therefore, when the data node decision results are different, the data node adjustment request sent by the coordinator to the data node resource pool and the final adjusted target data node are different.

[0123] In specific implementation, the data node decision result includes data node expansion and the number of data node expansions;

[0124] Correspondingly, sending a data node adjustment request to a data node resource pool based on the data node decision result includes:

[0125] Sending a data node expansion request to a data node resource pool based on the data node decision result, wherein the data node expansion request carries the data node expansion quantity;

[0126] Correspondingly, determining the target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and performing data processing based on the target data node includes:

[0127] Receive a second data node added by the data node resource pool based on the data node expansion quantity carried in the data node expansion request;

[0128] The first data node and the added second data node are determined as target data nodes, and data processing is performed based on the target data nodes.

[0129] For example, if the data node decision result is data node expansion, and the number of data node expansions is 2, then the coordinator will send a data node expansion request to the data node resource pool based on the data node decision result, and the data node expansion request will carry the data node expansion number of 2; then the data node resource pool will receive two second data nodes added based on the data node expansion number carried in the data node expansion request, and then use the first data node and the two added second data nodes as target data nodes for data processing.

[0130] In actual applications, the first data node is the currently running data node, and the second data node is the backup data node in the data node resource pool. When the first data node is not running, it will be recycled by the data node resource pool and wait for re-allocation. Therefore, the data node resource pool includes both the first data node and the second data node.

[0131] In the embodiment of the present specification, when the data node decision result is data node expansion, in order to ensure normal data processing, a second data node corresponding to the data node expansion quantity is extracted from the data node resource pool and used together with the first data node as the target data node for data processing.

[0132] In another embodiment of the present specification, the data node decision result includes data node scaling down and the number of data node scaling down;

[0133] Correspondingly, sending a data node adjustment request to a data node resource pool based on the data node decision result includes:

[0134] Determining a first data node to be shrunk from the first data nodes based on the number of shrunk data nodes;

[0135] issuing a data node scheduling task to the first data node to be shrunk, and stopping data processing of the first data node to be shrunk based on the data node scheduling task;

[0136] When the first data node to be shrunk has written all data into the object storage service platform, sending a data node shrink request to the data node resource pool, wherein the data node shrink request carries the first data node to be shrunk;

[0137] Correspondingly, determining the target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and performing data processing based on the target data node includes:

[0138] Determine a first data node to be stopped in the data node resource pool based on the first data node to be shrunk carried in the data node shrink request;

[0139] A first data node other than the stopped first data node is determined as a target data node, and data processing is performed based on the target data node.

[0140] For example, if the data node decision result is that the data node is scaled down, and the number of data node scale-downs is 2, then the coordinator will determine two first data nodes to be scaled down from the first data nodes based on the number of data node scale-downs 2, and then issue a data node scheduling task to the two first data nodes to be scaled down, and notify the two first data nodes to be scaled down of data processing (such as data writing, etc.) based on the data node scheduling task. At the same time, when the two first data nodes to be scaled down write all the data on the data nodes to the object storage service platform, a data node scale-down request for the two first data nodes to be scaled down is sent to the data node resource pool. After receiving the data node scale-down request, the data node resource pool recycles the two first data nodes to be scaled down, and the coordinator uses the remaining first data nodes as target data nodes for data processing.

[0141] In an embodiment of the present specification, when the data node decision result is that the data node is scaled down, in order to ensure normal data processing, the operation of the first data node to be scaled down is notified, the data node resource pool reclaims the first data node to be scaled down, and the remaining first data node is used as the target data node for data processing.

[0142] In another embodiment of the present specification, after the data processing is performed based on the target data node, the method further includes:

[0143] The target data node is used as the first data node, the operating status parameters of the first data node are continuously acquired, and parameter features of the operating status parameters are extracted based on preset feature extraction rules.

[0144] In actual applications, the first data node is the currently running data node. After the currently running data node is expanded or reduced in capacity to form a target data node, the target data node will be used as the first data node, and the updated running status parameters of the first data node will continue to be obtained to adjust the currently running first data node to expand or shrink based on the running status parameters; thereby, the currently running first data node can always process data normally, ensuring the normal operation of the distributed cloud database.

[0145] In an embodiment of the present specification, the data processing method predicts the decision results of the data nodes in the future time period based on the parameter characteristics of the operating status parameters of the current data nodes and a pre-established data node decision model, and adjusts the number of data nodes in the future time period based on the data node decision results. This can effectively achieve expansion and reduction of data nodes while avoiding frequent adjustments to the number of data nodes.

[0146] Furthermore, the data processing method can predict the expansion and contraction results of data nodes within a preset time period in the future through a data decision model, and has the ability to cope with frequent fluctuations. By storing historical monitoring data, the expansion and contraction has memory, which can reduce the misjudgment of frequent fluctuations. At the same time, in order to reduce the storage of monitoring data (operating status parameters) in the database, the characteristic values ​​of the monitoring data per hour are extracted, which effectively extracts useful information and compresses the cost of monitoring data, reducing the pressure of distributed cloud databases to store large amounts of monitoring data. At the same time, since the operations such as the master-slave switching of the database are very heavy, and in order to effectively achieve elastic expansion and contraction, the cost of expansion and contraction must be reduced as much as possible. In the implementation of this specification, the object storage service platform can be used to effectively save historical data, achieve lightweight expansion and contraction, and ensure data integrity.

[0147] The following combination Figure 4 , taking the application of a data processing method provided in this specification in a distributed cloud database as an example, the data processing method is further described. Specifically, the following steps are included.

[0148] Step 402: The coordinator starts.

[0149] Step 404: Determine that all data nodes currently performing data processing are started.

[0150] Step 406: Obtain the operating status parameters of the currently operating data node.

[0151] Step 408: Summarize the operating status parameters of all currently running data nodes.

[0152] Step 410: Input parameter characteristics corresponding to the operating status parameters of all currently running data nodes into a pre-established data node decision model to determine whether the currently running data nodes need to be expanded or reduced in capacity. If expanded, execute steps 412 to 418; if reduced, execute steps 420 to 426.

[0153] Step 412: Determine the data node expansion quantity output by the data node decision model.

[0154] Step 414: Sending a capacity expansion request to the data node resource pool based on the capacity expansion quantity of the data node.

[0155] Step 416: Receive the expanded data node fed back by the data node resource pool.

[0156] Specifically, after the coordinator sends a capacity expansion request to the data node resource pool, the data node resource pool evaluates currently idle data nodes and allocates a corresponding number of data nodes to the coordinator for scheduling and use.

[0157] Step 418: Process data on the new expanded data node together with the currently running data node.

[0158] Specifically, after the coordinator receives the new data node allocated by the data node resource pool, it registers the metadata of the new data node and allocates the new data node for use, that is, adds the new data node to the service (such as data processing services such as data writing or query).

[0159] Step 420: Determine the number of data node reductions output by the data node decision model.

[0160] Step 422: Send a reduction request to the data node resource pool based on the reduction quantity of the data node.

[0161] Step 424: Stop the currently running data node to be shrunk.

[0162] Step 426: Put the currently running data node to be scaled down back into the data node resource pool.

[0163] Specifically, when it is determined that the currently running data nodes need to be scaled down, the coordinator will evaluate the data nodes that need to be scaled down based on the data processing status of the currently running data nodes, and then send scheduling tasks to these data nodes, stop writing data to these data nodes, etc., and wait for these data nodes to transfer all data to the object storage service platform, then stop the services of these data nodes, and at the same time send a request to scale down these data nodes to the resource pool; after the data nodes to be scaled down stop serving, the resource pool will recycle these data nodes through the scheduling and control tasks.

[0164] The data processing method provided in the embodiments of this specification is designed for elastic expansion and contraction of distributed cloud databases. By extracting characteristic data of monitoring data of data nodes, the storage pressure of monitoring data on the database itself is effectively reduced. At the same time, the data node decision model constructed by extracting historical monitoring data characteristics can effectively determine whether the operating status of the current data node is a slight fluctuation or a persistent fluctuation, thereby making effective expansion and contraction decisions; and through this elastic expansion and contraction, not only the cost of elastic expansion and contraction is lightened and efficiency is improved through object storage, and the completeness of data is guaranteed, but also the storage of historical data of data nodes effectively improves the accuracy of system expansion and contraction decisions, greatly improves system performance, and greatly reduces operation and maintenance costs.

[0165] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 5 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0166] The feature extraction module 502 is configured to obtain the operating state parameter of the first data node and extract parameter features of the operating state parameter based on a preset feature extraction rule;

[0167] The node strategy determination module 504 is configured to input the parameter feature into the data node decision model to obtain the data node decision result corresponding to the parameter feature within a preset time period;

[0168] A request sending module 506 is configured to send a data node adjustment request to a data node resource pool based on the data node decision result, wherein the data node resource pool is composed of a first data node and a second data node;

[0169] The node adjustment module 508 is configured to determine a target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and perform data processing based on the target data node.

[0170] Optionally, the device further includes:

[0171] The model building module is configured to:

[0172] Acquire, at a preset time interval, the first data node, historical parameter characteristics of the historical operating state parameters within the preset time period, and the data node decision result corresponding to the historical parameter characteristics;

[0173] Using the historical parameter features as training data, using the data node decision results as labels, and forming training samples based on the training data and the labels;

[0174] Inputting the training samples into an initial data node decision model to obtain network parameters of the initial data node decision model;

[0175] The initial data node decision model is trained based on the network parameters to obtain a data node decision model.

[0176] Optionally, the device further includes:

[0177] A model storage module is configured to store the data node decision model in an object storage service platform;

[0178] Accordingly, the node strategy determination module 504 is further configured to:

[0179] The data node decision model is called from the object storage service platform, and the parameter characteristics are input into the data node decision model.

[0180] Optionally, the request sending module 506 is further configured to:

[0181] When the data node policy result determined based on the operating status parameters does not match the data node policy result obtained based on the data node decision model, a data node adjustment request is sent to the data node resource pool based on the data node policy result determined based on the operating status parameters.

[0182] Optionally, the device further includes:

[0183] The parameter comparison module is configured to compare the historical operating status parameters with the preset parameter threshold if there is no data node decision model and the historical operating status parameters of the first data node do not exist, and send a data node adjustment request to the data node resource pool based on the comparison result.

[0184] Optionally, the device further includes:

[0185] A parameter storage module, configured to store parameter characteristics of the operating status parameters in the object storage service platform;

[0186] Accordingly, the model building module is further configured as follows:

[0187] Obtain historical parameter characteristics of historical operating status parameters of the first data node within the preset time period from the object storage service platform at preset time intervals.

[0188] Optionally, the data node decision result includes data node expansion and the number of data node expansions;

[0189] Accordingly, the request sending module 506 is further configured to:

[0190] Sending a data node expansion request to a data node resource pool based on the data node decision result, wherein the data node expansion request carries the data node expansion quantity;

[0191] Accordingly, the node adjustment module 508 is further configured to:

[0192] Receive a second data node added by the data node resource pool based on the data node expansion quantity carried in the data node expansion request;

[0193] The first data node and the added second data node are determined as target data nodes, and data processing is performed based on the target data nodes.

[0194] Optionally, the data node decision result includes data node scaling down and the number of data node scaling down;

[0195] Accordingly, the request sending module 506 is further configured to:

[0196] Determining a first data node to be shrunk from the first data nodes based on the number of shrunk data nodes;

[0197] issuing a data node scheduling task to the first data node to be shrunk, and stopping data processing of the first data node to be shrunk based on the data node scheduling task;

[0198] When the first data node to be shrunk has written all data into the object storage service platform, sending a data node shrink request to the data node resource pool, wherein the data node shrink request carries the first data node to be shrunk;

[0199] Accordingly, the node adjustment module 508 is further configured to:

[0200] Determine a first data node to be stopped in the data node resource pool based on the first data node to be shrunk carried in the data node shrink request;

[0201] A first data node other than the stopped first data node is determined as a target data node, and data processing is performed based on the target data node.

[0202] Optionally, the feature extraction module 502 is further configured to:

[0203] A first data node is determined, and when all first data nodes are started, an operating status parameter of the first data node is obtained.

[0204] Optionally, the device further includes:

[0205] The node update module is configured to take the target data node as the first data node, continue to obtain the operating status parameters of the first data node, and extract parameter features of the operating status parameters based on preset feature extraction rules.

[0206] The data processing device provided in the embodiment of the present specification predicts the decision results of the data nodes in the future time period based on the parameter characteristics of the operating status parameters of the current data nodes and a pre-established data node decision model, and adjusts the number of data nodes in the future time period based on the data node decision results. This can effectively achieve the expansion and reduction of data nodes while avoiding frequent adjustments to the number of data nodes.

[0207] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0208] Figure 6 The block diagram of a computing device 600 according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0209] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0210] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0211] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 600 may also be a mobile or stationary server.

[0212] The processor 620 is used to execute the following computer executable instructions, and the processor is used to execute the computer executable instructions, which implement the steps of the data processing method when executed by the processor.

[0213] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data processing method.

[0214] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which implement the steps of the data processing method when executed by a processor.

[0215] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data processing method.

[0216] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0217] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0218] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0219] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0220] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Acquire an operating status parameter of the first data node, and extract parameter features of the operating status parameter based on a preset feature extraction rule; Input the parameter feature into a data node decision model to obtain a data node decision result corresponding to the parameter feature within a preset time period, wherein the preset time period is a future time period, and the data node decision model is obtained by training the historical parameter feature of the first data node and the data node decision result corresponding to the historical parameter feature; Sending a data node adjustment request to a data node resource pool based on the data node decision result, wherein the data node resource pool is composed of a first data node and a second data node; Determine a target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and perform data processing based on the target data node.

2. The data processing method according to claim 1, before obtaining the operating status parameter of the first data node, further comprising: Acquire, at a preset time interval, the first data node, historical parameter characteristics of the historical operating state parameters within the preset time period, and the data node decision result corresponding to the historical parameter characteristics; Using the historical parameter features as training data, using the data node decision results as labels, and forming training samples based on the training data and the labels; Inputting the training samples into an initial data node decision model to obtain network parameters of the initial data node decision model; The initial data node decision model is trained based on the network parameters to obtain a data node decision model.

3. The data processing method according to claim 2, after obtaining the data node decision model, further comprising: Storing the data node decision model in an object storage service platform; Accordingly, the step of inputting the parameter features into the data node decision model includes: The data node decision model is called from the object storage service platform, and the parameter characteristics are input into the data node decision model.

4. The data processing method according to claim 1, wherein sending a data node adjustment request to a data node resource pool based on the data node decision result comprises: When the data node policy result determined based on the operating status parameters does not match the data node policy result obtained based on the data node decision model, a data node adjustment request is sent to the data node resource pool based on the data node policy result determined based on the operating status parameters.

5. The data processing method according to claim 1, after obtaining the operating status parameter of the first data node, further comprising: If there is no data node decision model and no historical operating status parameters of the first data node, the historical operating status parameters are compared with a preset parameter threshold, and a data node adjustment request is sent to the data node resource pool based on the comparison result.

6. The data processing method according to claim 3, after obtaining the operating status parameter of the first data node and extracting the parameter feature of the operating status parameter based on a preset feature extraction rule, further comprising: Storing the parameter characteristics of the operating status parameters in the object storage service platform; Accordingly, the acquisition of the historical parameter characteristics of the historical operating state parameters of the first data node within the preset time interval includes: Obtain historical parameter characteristics of historical operating status parameters of the first data node within the preset time period from the object storage service platform at preset time intervals.

7. The data processing method according to claim 1, wherein the data node decision result includes data node expansion and the number of data node expansions; Correspondingly, sending a data node adjustment request to a data node resource pool based on the data node decision result includes: Sending a data node expansion request to a data node resource pool based on the data node decision result, wherein the data node expansion request carries the data node expansion quantity; Correspondingly, determining the target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and performing data processing based on the target data node includes: Receive a second data node added by the data node resource pool based on the data node expansion quantity carried in the data node expansion request; The first data node and the added second data node are determined as target data nodes, and data processing is performed based on the target data nodes.

8. The data processing method according to claim 3, wherein the data node decision result includes data node shrinkage and the amount of data node shrinkage; Correspondingly, sending a data node adjustment request to a data node resource pool based on the data node decision result includes: Determining a first data node to be shrunk from the first data nodes based on the number of shrunk data nodes; issuing a data node scheduling task to the first data node to be shrunk, and stopping data processing of the first data node to be shrunk based on the data node scheduling task; When the first data node to be shrunk has written all data into the object storage service platform, sending a data node shrink request to the data node resource pool, wherein the data node shrink request carries the first data node to be shrunk; Correspondingly, determining the target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and performing data processing based on the target data node includes: Determine a first data node to be stopped in the data node resource pool based on the first data node to be shrunk carried in the data node shrink request; A first data node other than the stopped first data node is determined as a target data node, and data processing is performed based on the target data node.

9. The data processing method according to claim 1, wherein obtaining the operating status parameter of the first data node comprises: A first data node is determined, and when all first data nodes are started, an operating status parameter of the first data node is obtained.

10. The data processing method according to any one of claims 1 to 9, after performing data processing based on the target data node, further comprising: The target data node is used as the first data node, the operating status parameters of the first data node are continuously acquired, and parameter features of the operating status parameters are extracted based on preset feature extraction rules.

11. A data processing device, comprising: A feature extraction module is configured to obtain an operating state parameter of the first data node and extract parameter features of the operating state parameter based on a preset feature extraction rule; A node strategy determination module is configured to input the parameter feature into a data node decision model to obtain a data node decision result corresponding to the parameter feature within a preset time period, wherein the preset time period is a future time period, and the data node decision model is obtained by training the historical parameter feature of the first data node and the data node decision result corresponding to the historical parameter feature; A request sending module is configured to send a data node adjustment request to a data node resource pool based on the data node decision result, wherein the data node resource pool is composed of a first data node and a second data node; The node adjustment module is configured to determine a target data node after the data node resource pool adjusts the first data node or the second data node based on the data node adjustment request, and perform data processing based on the target data node.

12. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, the steps of the data processing method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 10.

Images