Distributed data processing method, device and equipment

Abstract

The invention relates to a distributed data processing method, device and equipment. According to the method, a space-time state matrix is constructed by collecting hardware loads and data features of heterogeneous nodes, an optimal partition adjustment action is dynamically predicted through a deep reinforcement learning model, non-perceptual data distribution switching is achieved, the deep reinforcement learning model is updated online through feedback driving, and the optimal partition adjustment action is dynamically predicted. The problems of load imbalance, low resource utilization rate and response delay fluctuation caused by a traditional static partitioning strategy in a dynamic heterogeneous environment are effectively solved, and the adaptive scheduling capability, the resource utilization efficiency and the service continuity guarantee level of a distributed system in a complex scene are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a distributed data processing method, device and equipment. Background Art

[0002] In the field of distributed data processing, traditional systems generally adopt static data partitioning strategies, such as data sharding mechanisms based on fixed hash algorithms or predefined ranges. Such methods can meet basic requirements in scenarios with homogeneous hardware and stable loads, but gradually expose significant defects in dynamic heterogeneous environments.

[0003] With the popularization of cloud computing and edge computing, distributed clusters are often composed of heterogeneous nodes with significant differences in computing power. There are both computing nodes equipped with high-performance GPUs and storage nodes with only basic CPU capabilities. The static partitioning strategy cannot perceive the real-time load changes of nodes, resulting in long-term idle high-computing-power nodes and continuous overload of low-performance nodes, with low overall resource utilization and large fluctuations in response latency.

[0004] In addition, existing technologies usually rely on manual expansion or simple threshold-triggered migration when dealing with burst traffic, making it difficult to achieve millisecond-level dynamic response and prone to service degradation due to lagging decisions. The global lock mechanism adopted to ensure consistency during data migration further exacerbates the decline in system throughput, and the risk of service interruption during migration is significant.

[0005] In the face of real-time streaming data processing scenarios, the partition adjustment delay of the traditional batch processing mode can no longer meet the requirements of business continuity. Existing technical solutions have systematic deficiencies in aspects such as load prediction accuracy, smoothness of the migration process, and resource adaptive scheduling, restricting the improvement of the reliability of distributed systems in complex dynamic environments. Summary of the Invention

[0006] Based on this, it is necessary to provide a distributed data processing method, device and equipment for the above technical problems.

[0007] In a first aspect, the present application provides a distributed data processing method, including:

[0008] S1: Collect the hardware load indicators and data processing characteristics of each node in the distributed cluster, and generate a spatio-temporal state matrix based on the hardware load indicators and data processing characteristics;

[0009] S2: Based on the spatio-temporal state matrix, predict the optimal data partition adjustment action through a deep reinforcement learning model;

[0010] S3: According to the optimal data partition adjustment action, perform a partition adjustment operation to obtain a load-balanced data distribution; the partition adjustment operation is to migrate partitions or merge partitions or split partitions or maintain the status quo;

[0011] S4: Perform data processing based on the load balancing data distribution, collect feedback data during the data processing, and dynamically update the parameters of the deep reinforcement learning model based on the feedback data.

[0012] In a second aspect, the present application also provides a distributed data processing device, including:

[0013] A state generation module, configured to collect the hardware load metrics and data processing characteristics of each node in the distributed cluster, and generate a spatio-temporal state matrix based on the hardware load metrics and data processing characteristics;

[0014] An action prediction module, configured to predict an optimal data partitioning adjustment action based on the spatio-temporal state matrix through a deep reinforcement learning model;

[0015] A partitioning adjustment module, configured to perform a partitioning adjustment operation according to the optimal data partitioning adjustment action to obtain a load balancing data distribution; the partitioning adjustment operation is to migrate partitions or merge partitions or split partitions or maintain the status quo;

[0016] A model update module, configured to perform data processing based on the load balancing data distribution, collect feedback data during the data processing, and dynamically update the parameters of the deep reinforcement learning model based on the feedback data.

[0017] In a third aspect, the present application also provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, it implements a distributed data processing method as in the first aspect.

[0018] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements a distributed data processing method as in the first aspect.

[0019] The above-mentioned distributed data processing method, device and equipment construct a spatio-temporal state matrix by collecting the hardware load and data characteristics of heterogeneous nodes, use a deep reinforcement learning model to dynamically predict the optimal partitioning adjustment action, realize seamless data distribution switching, and drive online update of the deep reinforcement learning model through feedback, effectively solving the problems of load imbalance, low resource utilization rate and response delay fluctuation caused by traditional static partitioning strategies in a dynamic heterogeneous environment, and significantly improving the adaptive scheduling ability, resource utilization efficiency and service continuity guarantee level of the distributed system in complex scenarios. Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application or in the related art, the following will briefly introduce the drawings required for use in the description of the embodiments or the related art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0021] Figure 1 It is a schematic flowchart of a distributed data processing method provided by the present invention;

[0022] Figure 2 It is a schematic structural diagram of a distributed data processing device provided by the present invention. Detailed implementation manners

[0023] In order to make the objectives, technical solutions and advantages of the present application clearer, the following further details the present application in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] Referring to Figure 1 , which shows a schematic flowchart of a distributed data processing method provided by the present application. The method includes the following steps:

[0025] S1: Collect the hardware load indicators and data processing characteristics of each node in the distributed cluster, and generate a spatio-temporal state matrix based on the hardware load indicators and data processing characteristics.

[0026] Specifically, when collecting the hardware load indicators, a distributed monitoring tool or the built-in performance monitoring interface of the system can be used to obtain the hardware load indicators such as the CPU usage rate, memory occupancy, network bandwidth consumption, and disk I / O of each node in real time. These indicators are collected periodically at a preset time interval, for example, once per second or every few seconds, to ensure the timeliness and continuity of the data. The collected hardware load indicators can be normalized so that their numerical ranges are unified to [0,1] or other appropriate intervals, which is convenient for subsequent matrix operations and model processing. Common means such as linear scaling and Z-score standardization can be used for the normalization method.

[0027] When extracting the data processing characteristics, the types, scales, processing rates, etc. of the data currently processed by each node can be analyzed. For example, for a node processing streaming data, record the data inflow volume and processing delay per unit time; for a batch processing task node, count the task queue length and average task execution time, etc. Through feature engineering means, these data processing characteristics are quantified and dimensionally reduced to extract the core feature vectors that can effectively characterize the data processing ability and state of the node, while reducing the data dimension and retaining key information.

[0028] When constructing the spatio-temporal state matrix, the collected hardware load metrics and data processing characteristics can be organized according to two dimensions: the time series and the node spatial distribution. The time series dimension covers the data of several past collection cycles to reflect the change trend of the system over time; the node spatial dimension covers all nodes in the distributed cluster, reflecting the spatial heterogeneity of the cluster. The metrics and feature data of each node at different time points are filled into the corresponding positions of the matrix to form a three-dimensional spatio-temporal state matrix. The rows of the matrix represent the time series, the columns represent different nodes, and the elements in the matrix are the multi-dimensional state vectors of the corresponding nodes at a specific time, which are composed of the hardware load metrics and data processing characteristics combined after normalization and feature engineering.

[0029] S2: Based on the spatio-temporal state matrix, predict the optimal data partition adjustment action through a deep reinforcement learning model.

[0030] Specifically, select a deep reinforcement learning model architecture suitable for processing spatio-temporal sequence data and decision-making tasks, such as Deep Q-Network (DQN) and its variants (such as Double DQN, Prioritized Experience Replay DQN, etc.), or policy gradient-based methods (such as the reinforcement learning Actor-Critic algorithm, A3C, etc.). The number of neurons in the input layer of the model matches the dimension of the state vector of each time step and each node in the spatio-temporal state matrix to be able to completely receive and process the input spatio-temporal state information. The hidden layer adopts a multi-layer neural network structure, including a combination of convolutional layers, recurrent neural network layers (such as LSTM, GRU, etc.) or fully connected layers, for extracting spatio-temporal features and performing non-linear transformations to learn the complex system dynamics and optimal policies.

[0031] In the model training stage, use the historical spatio-temporal state matrix data as input samples, and at the same time combine the past actual executed data partition adjustment actions and their corresponding result feedback (such as the degree of load balancing after adjustment, the improvement amplitude of data processing efficiency, etc.) as supervision signals to optimize the model parameters through the reinforcement learning algorithm.

[0032] Define a suitable reward function, which gives positive or negative reward values according to the system performance metrics after data partition adjustment (such as load balancing degree, reduction amplitude of data processing delay, improvement of resource utilization, etc.), guiding the model to learn the optimal policy that can maximize the long-term cumulative reward. For example, when the data partition adjustment action makes the cluster load more balanced and the data processing efficiency is significantly improved, a higher positive reward is given; on the contrary, if the adjustment action leads to the deterioration of system performance, a negative reward is given.

[0033] During the training process, the model continuously tries and corrects mistakes. It predicts possible optimal data partition adjustment actions based on the current spatio-temporal state matrix, observes the rewards and new states feedback by the environment after executing the actions, and then updates the model parameters, adjusting the weights and biases of the policy network to gradually approach the optimal policy.

[0034] During actual operation, the spatio-temporal state matrix generated in real time is input into the trained deep reinforcement learning model. Through forward propagation calculation, the model evaluates the expected reward values of different possible data partition adjustment actions (such as migrating partitions, merging partitions, splitting partitions, maintaining the status quo, etc.) under the current spatio-temporal state.

[0035] Select the data partition adjustment action with the highest expected reward value as the optimal action. This action can theoretically optimize the data processing performance of the distributed cluster to the greatest extent under the current system state, achieving load balancing and efficient resource utilization.

[0036] S3: According to the optimal data partition adjustment action, perform partition adjustment operations to obtain a load-balanced data distribution; the partition adjustment operations are migrating partitions, merging partitions, splitting partitions, or maintaining the status quo.

[0037] Specifically, for the predicted optimal data partition adjustment action, design a corresponding execution mechanism. For the operation of migrating partitions, determine the data partition identifier, source node, and target node that need to be migrated. Through the management interface of the distributed file system or database, migrate the specified data partition from the source node to the target node, and ensure the integrity and consistency during the data migration process.

[0038] In the operation of merging partitions, identify adjacent data partitions that meet the merging conditions, such as similar data types and related processing logics. Integrate the data of these partitions onto one node according to predefined merging rules (such as simple data splicing, merging based on primary keys, etc.), and update the relevant metadata information simultaneously to enable the system to correctly identify and access the merged data.

[0039] For the operation of splitting partitions, based on the characteristics and distribution of the data partitions, such as excessive data volume and uneven access heat, select a suitable splitting strategy (such as splitting by data range, splitting by hash value, etc.), split a data partition into multiple sub-partitions, and reallocate these sub-partitions to different nodes to disperse the load and improve the data processing parallelism.

[0040] When the optimal action is to maintain the status quo, no partition adjustment operation is performed, and the current data distribution and node load status are maintained. Continue to monitor the system operation for subsequent adjustments.

[0041] By performing the above partition adjustment operations, the distribution of data in the distributed cluster becomes more reasonable, and the loads of each node tend to be balanced. For example, the operation of migrating partitions can migrate some data on overloaded nodes to nodes with lighter loads, alleviating the pressure on overloaded nodes; the operation of merging partitions can reduce the difference in the number of data partitions between nodes, avoiding additional management overhead and performance losses caused by some nodes carrying too many small partitions; the operation of splitting partitions can disperse partitions with excessive data volume that cause slow processing to multiple nodes for parallel processing, improving the overall data processing efficiency.

[0042] After the adjustment operations are completed, it is also possible to verify and evaluate the new data distribution situation, check whether the load indicators of each node reach the expected balanced state, whether the data processing process is smooth, etc., to ensure the effectiveness of the partition adjustment operations and the stability of the system.

[0043] S4: Perform data processing based on the load-balanced data distribution, collect feedback data during the data processing process, and dynamically update the parameters of the deep reinforcement learning model based on the feedback data.

[0044] Specifically, on the basis of achieving a load-balanced data distribution, start the distributed data processing task. Each node performs local data processing according to the assigned data partitions, following the preset data processing logic and algorithms, such as performing data calculation, analysis, conversion, etc. Due to the rationality of data distribution and the balance of loads, each node can relatively efficiently complete its respective tasks, reducing processing delays and resource waste caused by overloading of some nodes, and improving the data processing ability and response speed of the entire distributed system.

[0045] During the data processing process, continuously collect feedback data related to system performance and data processing effects. These feedback data include but are not limited to the data processing completion time of each node, the accuracy of processing results, resource consumption situations (such as the peak and average values of CPU, memory, network, etc.), data read and write delays, etc. Clean and preprocess the collected feedback data, remove invalid information such as outliers and missing values, and organize them according to dimensions such as time series and node identifiers to form a feedback data set corresponding to the previous spatio-temporal state matrix, providing a basis for model update.

[0046] Combine the new feedback data with the previous historical data as the incremental training data for the deep reinforcement learning model. Use the experience replay mechanism in the reinforcement learning algorithm to mix these new and old data samples and input them into the model for retraining and learning to update the model's parameters.

[0047] Adjust and optimize the reward function according to the new feedback data, so that the reward function can more accurately reflect the actual effect of the data partitioning adjustment action in the current system environment, further guide the model to learn better strategies, and continuously improve the adaptive ability and performance of the distributed data processing system.

[0048] The above-mentioned distributed data processing method collects the hardware load and data characteristics of heterogeneous nodes to construct a spatio-temporal state matrix, uses a deep reinforcement learning model to dynamically predict the optimal partitioning adjustment action, realizes seamless data distribution switching, and updates the deep reinforcement learning model online through feedback, effectively solving the problems of load imbalance, low resource utilization rate, and response delay fluctuation caused by traditional static partitioning strategies in dynamic heterogeneous environments, and significantly improving the adaptive scheduling ability, resource utilization efficiency, and service continuity guarantee level of the distributed system in complex scenarios.

[0049] In an optional embodiment, S1 includes the following steps:

[0050] S11: Collect the hardware load metrics of each node in the distributed cluster; the hardware load metrics include the CPU utilization rate C at time t i and the GPU video memory occupancy rate G at time t i , where i is the node number.

[0051] Specifically, the hardware load metrics can be the CPU utilization rate and the GPU video memory occupancy rate. Since the CPU and GPU are the main computing resources in the distributed cluster nodes, their utilization rates can directly reflect the computing load of the nodes. For a distributed system with data processing as the core, these two metrics are highly representative and can effectively characterize the usage status of the computing power of the nodes.

[0052] The CPU utilization rate reflects the busyness of the central processing unit, including the usage ratios of the user mode and the kernel mode; the GPU video memory occupancy rate reflects the usage of the graphics processing unit's video memory, which is particularly important for nodes involved in a large number of parallel computing, image processing, or deep learning tasks.

[0053] Use the performance monitoring API provided by the distributed operating system or a third-party hardware monitoring tool to deploy data collection agents on each node. These agent programs obtain the CPU utilization rate C of the nodes in real time at a preset time interval (such as once per second or adjusted according to actual needs) i and the GPU video memory occupancy rate G i , where i is the node number, used to distinguish different nodes in the cluster.

[0054] To ensure the accuracy and integrity of data, the collection process takes into account the compatibility of the operating system and the differences in hardware devices, adapts and optimizes nodes of different architectures and models, and avoids significant impacts on system performance caused by the collection tool itself.

[0055] Cache the collected hardware load metric data locally at the node and store it in a unified data format, such as using a time series database or a lightweight file storage format. Regularly upload this data to the central monitoring node or data processing center of the distributed cluster for subsequent analysis and processing. Ensure the security and integrity of the data during the data transmission process, and encryption transmission protocols and data verification mechanisms can be used.

[0056] S12: Extract the data feature vector of the data processed in the distributed cluster. The data feature vector includes the key-value access frequency f at time t k , where k is the data key.

[0057] Specifically, the key-value access frequency f k is a key element in the data feature vector, where k is the data key. In the distributed data processing scenario, the access frequencies of different data keys vary greatly, and the data keys with high access frequencies often have important impacts on system performance and resource allocation. By statistically analyzing the key-value access frequency, the hot spot distribution of the data can be understood, providing a basis for subsequent data partition adjustment.

[0058] On each node of the distributed cluster, monitor and analyze the data being processed, record the number of accesses to each data key within a unit time, and calculate the key-value access frequency f k . This process can be achieved by setting monitoring points at the data access interface to intercept and count data access requests.

[0059] To improve the efficiency and accuracy of data feature extraction, a distributed computing framework or stream processing technology can be used to perform real-time statistics and analysis on the access conditions of large-scale data, ensuring that changes in data features can be captured in a timely manner.

[0060] Organize the extracted key-value access frequency f k into a vector form to form a data feature vector. The dimension of this vector is determined by the number of data keys of concern, and each element corresponds to the access frequency of a data key. In practical applications, dimension reduction or quantization processing can be performed on the data feature vector to reduce the data scale and computational complexity while retaining key feature information.

[0061] S13: Normalize the hardware load metrics and data feature vectors to generate a spatio-temporal state matrix; the calculation formula for the spatio-temporal state matrix S(t) is:

[0062]

[0063] Among them, μ C represents the mean value of the CPU utilization rate of all nodes at time t, and σ C represents the standard deviation of the CPU utilization rate of all nodes at time t. μ G represents the mean value of the GPU utilization rate of all nodes at time t, and σ G represents the standard deviation of the GPU utilization rate of all nodes at time t. Among them, max(f) represents the maximum value among all key-value access frequencies, φ(·) is a time window encoding function, and T is a preset service cycle.

[0064] Specifically, the purpose of the normalization process is to convert hardware load metrics and data feature vectors with different dimensions and value ranges into the same numerical interval (such as [0, 1]), which is convenient for subsequent matrix operations and model processing. At the same time, it eliminates the influence brought by the dimensional difference between different metrics, and improves the comparability and consistency of the data.

[0065] For the CPU utilization rate C i and the GPU video memory occupancy rate G i , the Z-score standardization method is used for normalization, that is, the mean value μ C and the standard deviation σ C of the CPU utilization rate of all nodes at time t are calculated respectively, as well as the mean value μ G and the standard deviation σ G of the GPU utilization rate. Then, according to the formulas and , the CPU and GPU utilization rates of each node are converted into standardized values.

[0066] For the key-value access frequency f k , the maximum-minimum normalization method is adopted to convert it into a ratio relative to the maximum value max(f), that is which can highlight the relative differences in the access frequencies of different data keys.

[0067] The time window encoding function φ(·) is introduced to encode the current time t and the preset service cycle T to obtain This encoding process can adopt trigonometric functions, polynomial functions or other appropriate encoding methods to convert time information into a numerical form that can be effectively represented in the matrix, so as to reflect the influence of time factors on the system state.

[0068] The normalized hardware load metrics, data feature vectors, and time window encoding values are combined to form a spatio-temporal state matrix S(t). Each row of the matrix represents the state of a node at a specific time, including the normalized CPU utilization, GPU utilization, key-value access frequency, and time window encoding; the columns of the matrix correspond to different nodes. In this way, the overall state of the distributed cluster in the spatio-temporal dimension is comprehensively and systematically reflected.

[0069] In an optional embodiment, S2 includes the following steps:

[0070] S21: Construct a deep Q-network model, input the spatio-temporal state matrix S(t) into the deep Q-network model, and output the action-value function Q(s,a); where the action-value function Q(s,a) represents the expected return of executing action a in state s, the action space A = {migrate partition, merge partition, split partition, maintain the status quo}, s ∈ S(t) and a ∈ A.

[0071] Specifically, the deep Q-network (DQN) model is an algorithm architecture that combines deep learning and reinforcement learning, and is suitable for dealing with decision-making problems in high-dimensional state spaces and complex action spaces. In this method, the constructed deep Q-network model takes the spatio-temporal state matrix S(t) as input, and through the transformation of a multi-layer neural network, outputs the action-value function Q(s,a) of each possible action in different states.

[0072] The network structure can include an input layer, several hidden layers, and an output layer. The number of neurons in the input layer matches the dimension of the spatio-temporal state matrix S(t) to be able to completely receive state information. The hidden layer can adopt combinations such as convolutional layers, fully connected layers, or long short-term memory network (LSTM) layers, etc., for extracting state features and learning complex system dynamics. The number of neurons in the output layer is the same as the size of the action space A, and each neuron corresponds to the action-value function Q(s,a) of an action a.

[0073] Taking the normalized spatio-temporal state matrix S(t) as the model input, through the forward propagation calculation of the neural network, the expected return of each action in the current state is obtained, that is, the action-value function Q(s,a). The action space A includes four actions: migrate partition, merge partition, split partition, and maintain the status quo. Therefore, the output layer will have four neurons, corresponding to the Q-values of these four actions respectively.

[0074] In the model training stage, the deep Q-network is trained using historical data and real-time feedback data. By optimizing algorithms (such as Adam, RMSprop, etc.), the mean square error between the predicted action-value function and the actually obtained reward is minimized, and the network parameters are continuously adjusted to improve the prediction accuracy of the model.

[0075] To improve the stability and convergence speed of the model, an experience replay mechanism can be adopted. Past experiences (states, actions, rewards, next states) are stored in a replay buffer, and small batches of data are randomly sampled for training to avoid overfitting and unstable training caused by consecutive related data.

[0076] S22: Design a reward function R(t) based on the load balance degree, throughput, and amount of migrated data. The expression of the reward function R(t) is:

[0077] R(t) = α·BalanceIndex(t) + β·Throughput(t) - ε·DataVolume(t);

[0078] Among them, BalanceIndex(t) represents the load balance degree at time t, which is calculated through hardware load metrics; Throughput(t) represents the throughput at time t; DataVolume(t) represents the amount of migrated data at time t, and α, β, and ε are preset weight coefficients.

[0079] Specifically, the design of the reward function R(t) aims to comprehensively consider multiple key performance indicators of the distributed system, guiding the deep Q-network model to learn strategies that can optimize the overall system performance. The load balance degree reflects the balance of resource utilization among nodes, the throughput reflects the data processing ability of the system, and the amount of migrated data is related to the cost of data partition adjustment operations.

[0080] The load balance degree BalanceIndex(t) is calculated through hardware load metrics (such as CPU utilization rate, GPU utilization rate, etc.). Methods such as variance, standard deviation, or imbalance index can be used to quantify the balance degree of node loads. For example, calculate the sum of the squared deviations of the CPU utilization rates of all nodes from the average CPU utilization rate, and then take its reciprocal as a measure of the load balance degree. The more balanced the load, the higher this value.

[0081] The throughput Throughput(t) represents the amount of data processed by the system or the number of tasks completed per unit time, directly reflecting the data processing efficiency of the system. It can be obtained by statistically summing up the amount of data processing tasks completed by each node within a time interval.

[0082] The amount of migrated data DataVolume(t) refers to the total amount of data that needs to be migrated during the data partition adjustment operation, which is directly proportional to the cost of the adjustment operation. The smaller the amount of data, the smaller the impact of the adjustment operation on the system performance.

[0083] The above three parts are linearly combined according to the preset weight coefficients α, β, and ε to obtain the reward function R(t) = α·BalanceIndex(t) + β·Throughput(t) - ε·DataVolume(t). The setting of the weight coefficients can be adjusted according to the actual system requirements and performance optimization goals. For example, if more attention is paid to load balancing, the value of α can be appropriately increased.

[0084] The reward function provides a learning supervision signal for the deep Q-network model. By giving positive or negative rewards, it guides the model to select actions that can improve system performance. In practical applications, the weight coefficients of the reward function can be dynamically adjusted according to the dynamic changes and performance feedback of the system to adapt to different workloads and business scenarios.

[0085] S23: Iteratively update the action-value function Q(s,a) through the Bellman equation. The update formula for the action-value function is:

[0086]

[0087] where η is the first preset learning rate, γ is the preset discount factor, s′ is the next-state, a′ is the next-action, s′ ∈ S(t) and a′ ∈ A.

[0088] Specifically, the Bellman equation is an equation used in reinforcement learning to recursively represent the optimal value function. Based on the principle of optimality, it relates the current action-value function to the maximum action-value function at the next time step. By iteratively applying the Bellman equation, the optimal action-value function can be gradually approximated.

[0089] According to the Bellman equation, the update formula for the action-value function is where η is the first preset learning rate, which determines the step size of each update; γ is the preset discount factor, used to balance the weights of the current reward and future rewards; s′ is the state at the next time step after executing action a; a′ is the possible action at the next time step in state s′.

[0090] In the update process, the error between the currently estimated action-value function and the sum of the actual obtained reward and the discounted value of the future maximum possible action-value function is used to adjust the current action-value function in proportion to the learning rate, making it gradually converge to the true value.

[0091] At each time step, based on the current state s and the executed action a, the reward R(t) and the next state s′ are observed, and the action-value function Q(s,a) is updated using the above formula. Repeating this process, as the number of iterations increases, the action-value function will gradually approach the optimal value, and the decision-making performance of the model will continuously improve.

[0092] S24: Select the action corresponding to the maximum Q value according to the updated action-value function According to action a opt Generate the optimal data partition adjustment action

[0093] Specifically, in each state s, according to the updated action-value function Q(s, a), select the action with the maximum Q value This action is considered the optimal action that can bring the maximum expected benefit in the current state

[0094] According to the selected optimal action a opt , generate specific data partition adjustment operation instructions. For example, if a opt is a migration partition, determine the detailed information such as the data partition to be migrated, the source node, and the target node, and generate the corresponding data migration task; if it is a merge partition, determine the partitions to be merged and their nodes, and formulate a merge plan; if it is a split partition, determine the partition to be split, its splitting method, and the target node, etc

[0095] Execute the generated optimal data partition adjustment action in the distributed cluster, observe the performance changes of the system according to the execution results, including the improvement of the load balance degree, the increase in throughput, and the control of the migrated data volume, etc. Use this feedback information as new training data to further optimize the deep Q-network model, forming a closed-loop adaptive optimization process to continuously improve the intelligence level and performance of the system

[0096] In an alternative embodiment, calculating the load balance degree includes the following steps

[0097] S221: Calculate the local load standard deviation of each node at time t according to the hardware load index; the calculation formula for the local load standard deviation is

[0098]

[0099] where is the local load standard deviation of node i in the distributed cluster at time t, L i is the load of node i at time t, and μ L is the average load of all nodes at time t

[0100] Specifically, the local load standard deviation is used to measure the deviation degree of the load of a single node at a specific moment from the average load of its cluster. The larger the standard deviation, the greater the difference between the load of this node and the average load of the cluster, indicating that this node is overloaded or underloaded. By calculating the local load standard deviation of each node, the load distribution of each node in the cluster can be initially understood, providing basic data for the subsequent calculation of the load balance degree

[0101] Hardware metrics that can comprehensively reflect the node load conditions can be selected, such as CPU utilization rate, GPU utilization rate, memory occupancy rate, etc. When calculating the local load standard deviation, these hardware load metrics can be weighted and synthesized to obtain an index L that can comprehensively characterize the node load level. i . The weights can be set according to the degree of influence of different hardware metrics on the system performance. For example, for a cluster mainly composed of compute-intensive tasks, higher weights can be given to the CPU and GPU utilization rates.

[0102] According to the formula where is the local load standard deviation of node i at time t, L i is the comprehensive load value of node i at time t, and μ L is the average load of all nodes at time t. This formula actually calculates the absolute value of the difference between the load of node t and the average load.

[0103] For example, assume that the cluster has 3 nodes, and their comprehensive load values at time t are L1 = 0.7, L2 = 0.5, and L3 = 0.3 (normalized values) respectively. Then the average load The local load standard deviation of node 1 The local load standard deviation of node 2 The local load standard deviation of node 3

[0104] S222: Assign weights w according to the hardware types of each node i ; where w i represents the weight of node i.

[0105] Specifically, nodes of different hardware types play different roles and have different processing capabilities in the distributed cluster. Therefore, corresponding weights w need to be assigned according to the hardware types of the nodes i to reflect their importance in the load balancing degree calculation. For example, a compute node equipped with a high-performance GPU has an advantage in processing specific types of compute tasks and may be assigned more tasks in load distribution, so a higher weight can be given; while a storage node with only basic CPU capabilities has a relatively lower weight.

[0106] The determination of the weights can be achieved through a quantitative evaluation of the hardware configuration. For example, according to hardware parameters such as the CPU performance, GPU performance, memory capacity, and network bandwidth of the nodes, a weighted summation or other appropriate comprehensive evaluation method can be used to calculate the hardware performance scores of each node, and then these scores are normalized to the [0, 1] interval as the weight w of the node. i. It is also possible to manually set the weights according to business requirements and experience to ensure the rationality and fairness of weight allocation.

[0107] To ensure the comparability and stability of the calculation results, the assigned weights are normalized so that the sum of the weights of all nodes is 1. This can avoid numerical instability problems caused by overly large or small weight values in subsequent calculations and is also beneficial for comprehensive calculations with other normalized indicators.

[0108] S223: According to the weight w i Calculate the load balancing degree. The calculation formula for the load balancing degree is:

[0109]

[0110] where N is the total number of nodes in the distributed cluster.

[0111] Specifically, the calculation formula for the load balancing degree is where N is the total number of nodes in the distributed cluster. The numerator part is the weighted sum of the local load standard deviations of each node, reflecting the degree of dispersion of the load distribution across the entire cluster; the denominator part is the sum of the products of the weights of each node and the average load, used to normalize the numerator so that the value range of the load balancing degree is between [0, 1].

[0112] When the local load standard deviation of all nodes is 0, that is, the load of all nodes is equal to the average load, the numerator is 0, and the load balancing degree BalanceIndex(t) = 1, indicating that the load is completely balanced; when the local load standard deviation of the nodes is larger, the numerator is larger, and the load balancing degree is smaller, indicating that the load distribution is more uneven.

[0113] Suppose the cluster has 3 nodes with weights w1 = 0.5, w2 = 0.3, and w3 = 0.2 respectively. At time t, their local load standard deviations are respectively The average load μ L = 0.5. Then the numerator part The denominator part The load balancing degree This result indicates that the load distribution of the current cluster is relatively balanced, but there is still room for improvement.

[0114] In an alternative embodiment, input the spatio-temporal state matrix S(t) into the deep Q-network model to output the action value function Q(s, a), including the following steps:

[0115] S211: Perform principal component analysis on the spatio-temporal state matrix S(t) to extract the first k-dimensional principal components S~(t); where S~(t) ∈ R k , and k is the dimension after preset dimensionality reduction.

[0116] Specifically, principal component analysis (PCA) is a commonly used dimensionality reduction technique that aims to project high-dimensional data into a low-dimensional space while retaining the main features and information in the data as much as possible. By performing PCA on the spatio-temporal state matrix S(t), the complexity of the data can be reduced, the amount of computation can be decreased, and at the same time, the main change trends in the data can be highlighted, improving the training efficiency and generalization ability of the model.

[0117] The steps for principal component extraction can include:

[0118] First, standardize the spatio-temporal state matrix S(t) to ensure that each feature has the same dimension and numerical range.

[0119] Calculate the covariance matrix of the standardized matrix, which reflects the correlation between different features.

[0120] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues and corresponding eigenvectors. The magnitude of the eigenvalues represents the amount of information contained in the corresponding principal components. Arrange them in descending order of eigenvalues and select the first k eigenvectors as the principal components to form a projection matrix.

[0121] Multiply the original spatio-temporal state matrix S(t) by the projection matrix to obtain the first k-dimensional principal components S~(t) after dimensionality reduction, whose dimension is R k , and the value of k is preset according to actual needs, usually determined by the cumulative variance contribution rate to ensure that sufficient information is retained.

[0122] Suppose the dimension of the spatio-temporal state matrix S(t) is 10×100 (10 nodes, 100 time steps). Through PCA analysis, it is found that the cumulative variance contribution rate of the first 10 principal components reaches 95%. Then select k = 10, reduce the data to a 10-dimensional space, and obtain S~(t) ∈ R 10 , by this method, both the main features of the data are retained and the amount of data is greatly reduced, which is beneficial to the subsequent processing of the model.

[0123] S212: Construct a two-layer long short-term memory network, input the first k-dimensional principal components S~(t) into the first LSTM layer of the two-layer long short-term memory network according to time steps to generate the first hidden state sequence and input the first hidden state into the second LSTM layer of the two-layer long short-term memory network to generate the final hidden state where, d is the dimension of the hidden layer.

[0124] Specifically, the Long Short-Term Memory network (LSTM) is a special type of Recurrent Neural Network (RNN) structure that can effectively handle long-term dependencies in sequential data. A two-layer LSTM network is constructed by stacking two LSTM layers together to form a deeper network structure, enhancing the model's ability to extract features and learn from sequential data.

[0125] The first LSTM layer receives the k-dimensional principal component S~(t) input at each time step, encodes and transmits the sequential information through its internal gating mechanism (input gate, forget gate, and output gate), and generates the first hidden state sequence. This hidden state sequence contains the feature extraction results of the first LSTM layer for the input sequence at different time steps.

[0126] The second LSTM layer takes the first hidden state sequence as input, further abstracts and fuses the sequential features, and generates the final hidden state. The final hidden state belongs to the R d space, where d is the dimension of the hidden layer. It synthesizes the information of the entire sequence and can better capture the temporal dependencies and dynamic change features in the spatio-temporal state matrix.

[0127] When constructing the two-layer LSTM network, parameters such as the number of hidden units and activation functions of each layer are reasonably set. For example, the number of hidden units in the first LSTM layer can be selected according to the dimension k of the input principal component and experience, and the number of hidden units d in the second LSTM layer can be balanced in combination with the specific task and model complexity.

[0128] When training the network, appropriate optimization algorithms (such as Adam, RMSprop, etc.) and loss functions (such as mean squared error, cross-entropy, etc.) are used to adjust the network weights and biases according to the training data, enabling the model to accurately learn the mapping relationship between the spatio-temporal state and the action value function.

[0129] Assume k = 10 and the number of time steps is 100. Then the shape of the data input to the first LSTM layer is 100×10. The number of hidden units in the first LSTM layer is set to 50, and the generated first hidden state sequence has a shape of 100×50. The number of hidden units in the second LSTM layer is set to 30. After inputting , the generated final hidden state has a shape of 30×1, that is, it belongs to the R 30 space. This vector will be used as the input to the subsequent fully connected layer for mapping to the action value function.

[0130] S213: Map the final hidden state through the fully connected layer of the double-layer long short-term memory network to the action value function Q(s,a), and the mapping formula is:

[0131]

[0132] where W h and W a are weight matrices, W h ∈R m×d , W a ∈R |A|×m ; h b and b a are bias terms, h b ∈R m , b a ∈R |A| , m is the number of hidden units in the fully connected layer, |A| is the total number of actions included in the action space, and ReLU is the activation function.

[0133] Specifically, the fully connected layer is a common layer type in neural networks. It fully connects each element of the input vector to each element of the output vector, and realizes the linear transformation and non-linear activation from the input space to the output space by learning the weight matrix and bias term.

[0134] In this step, the fully connected layer receives the final hidden state output by the double-layer LSTM network (belonging to the R d space), maps it to the space of the action value function Q(s,a), and the dimension of this space is equal to the total number of actions included in the action space A.

[0135] The mapping formula is where:

[0136] W h and W a are weight matrices, W h ∈R m×d , W a ∈R |A|×m , m is the number of hidden units in the fully connected layer, and |A| is the total number of actions included in the action space. The weight matrix is obtained through training and learning, and is used to adjust the weights of the input signals to distinguish and combine the importance of different features.

[0137] h b and b a are bias terms, h b ∈R m , b a ∈R |A| , and the bias term is used to provide a basic activation value for the neuron, so that the model can better fit the data.

[0138] The ReLU is an activation function, i.e., ReLU(x) = max(0, x), which is used to introduce non - linear factors so that the neural network can learn complex mapping relationships. The fully - connected layer first performs a linear transformation through and then applies the ReLU activation function for non - linear processing. Finally, through the value of the action - value function Q(s, a) is obtained.

[0139] During the model training process, the weight matrices W h and W a and the bias terms h b and b a are updated through the back - propagation algorithm. The error between the predicted action - value function and the actual value is calculated according to the loss function, and then the gradient is calculated according to the chain rule. The optimization algorithm is used to adjust the parameters to minimize the error.

[0140] To prevent overfitting, regularization techniques (such as L1 and L2 regularization) can be used to constrain the weight matrix. At the same time, the number of hidden units in the fully - connected layer and the activation function are reasonably set to balance the complexity and generalization ability of the model.

[0141] In an alternative embodiment, S3 includes the following steps:

[0142] S31: When the Q - value of the migration partition in the action - value function Q(s, a) satisfies Q(s, migration partition)>δ·Q avg where δ is a preset dynamic threshold and Q avg is the historical average Q - value.

[0143] Specifically, the Q - value of the migration partition in the action - value function Q(s, a) reflects the expected benefit of performing the migration - partition action in the current state. By comparing it with δ·Q avg it can be determined whether it is necessary to trigger the data - partition migration operation.

[0144] δ is a preset dynamic threshold, and its value can be determined according to historical experience, system performance requirements, or through experimental tuning. Q avg is the historical average Q - value, representing the average expected benefit of performing each action in historical data. When the Q - value of the migration partition is greater than this dynamic threshold, it means that performing the migration - partition action in the current state is expected to bring higher benefits than the average case, that is, it is more conducive to improving the system's load - balancing degree and overall performance. Therefore, the data - partition migration is triggered.

[0145] Obtain the Q-values of the migration partitions in the real-time action value function Q(s,a), as well as the preset dynamic threshold δ and the calculated historical average Q-value Q avg . Calculate δ·Q avg . Compare the Q-value of the migration partition with it. If Q(s, migration partition) > δ·Q avg , trigger the data partition migration process; otherwise, do not perform the migration partition operation, maintain the current data distribution or consider performing other adjustment actions.

[0146] S32: Execute data partition migration using a double-buffer mechanism to generate an updated data partition layout, including the following steps:

[0147] Create a new partition, write the newly arrived incremental data into the new partition, and at the same time keep the old partition continuing to process the read and write requests of the existing data.

[0148] Extract a full data snapshot from the old partition and synchronize the full data snapshot to the new partition.

[0149] Calculate the differential data ΔP = {d|d ∈ P new ∩ P old , d new .t > d old .t}, where P new is the new partition, P old is the old partition, d new .t and d old .t are the timestamps of the same data item in the new partition and the old partition respectively.

[0150] Compress and encode the differential data to generate a compressed package, and transmit the compressed package to the target storage node.

[0151] Redirect the client requests from the old partition to the new partition, and release the memory and storage resources occupied by the old partition to complete the data partition migration and obtain the updated data distribution.

[0152] Specifically, in a distributed cluster, create a new partition to receive the newly arrived incremental data. The creation of the new partition needs to determine its location, size and other attributes according to the data partition strategy and register them in the metadata of the cluster. Write the newly arrived incremental data into the new partition, and at the same time keep the old partition continuing to process the read and write requests of the existing data. This can ensure that during the data migration process, the data write and read operations of the system are not affected, and avoid service interruption caused by partition migration.

[0153] Extract a full - volume data snapshot from the old partition. The full - volume data snapshot is a complete copy of all data in the old partition. During the extraction process, data consistency and integrity must be ensured. Snapshot technology or data backup tools can be used to achieve this. Synchronize the extracted full - volume data snapshot to the new partition. The synchronization process ensures accurate data transmission. Distributed file systems, data transfer services, or custom data synchronization algorithms can be utilized to ensure that the new partition receives the same full - volume data as the old partition.

[0154] Calculate the differential data ΔP = {d|d ∈ P new ∩P old , d new .t > d old .t}, where P new is the new partition, P old is the old partition, d new .t and d old .t are the timestamps of the same data item in the new partition and the old partition respectively. Compress and encode the differential data to generate a compressed package. Common compression algorithms (such as gzip, zlib, etc.) can be used for compression encoding to reduce the data volume and improve transmission efficiency. Transmit the compressed package to the target storage node. During the transmission process, data security and integrity must be ensured. Encrypted transmission protocols and data verification mechanisms can be adopted to prevent data from being tampered with or damaged during transmission. In distributed data processing, differential data calculation is used to identify incremental changes between the new and old partitions. By only transmitting and processing these differential parts, the system can significantly reduce the transmission volume during the data migration process, improve synchronization efficiency, and reduce the occupancy of network bandwidth. At the same time, differential data calculation can also ensure data consistency and accuracy, enabling the new partition to quickly reach the same state as the old partition after receiving the differential data, reducing errors and performance issues caused by data inconsistency, thereby improving the response speed and resource utilization rate of the entire distributed system and ensuring business continuity and stability.

[0155] Redirect client requests from the old partition to the new partition, which can be achieved by updating the routing table of the distributed system, the load balancer configuration, or the partition mapping information of the client, so that subsequent client requests directly access the new partition. Release the memory and storage resources occupied by the old partition. After ensuring that the new partition has successfully received all data and client requests have been fully redirected, recycle the resources occupied by the old partition, including memory space, disk storage, etc., for other tasks or partitions to use.

[0156] S33: Verify the load - balancing of the updated data partition layout by calculating the variance of the new node loads in the distributed cluster under the updated data distribution; if the variance of the new node loads is less than the variance of the old node loads, confirm that the migration is effective and use the updated data partition layout as the load - balanced data distribution; otherwise, roll back the updated data partition layout to the previous version.

[0157] Specifically, the node load variance is used as an indicator to verify the load balance of the data partition layout after update. The node load variance reflects the degree of dispersion of the loads of each node in the cluster. The smaller the variance, the more balanced the load distribution. To calculate the new node load variance, first obtain the load values of each node under the updated data distribution, and then calculate according to the variance calculation formula. The specific steps are as follows: calculate the sum of the squares of the differences between the load of each node and the average load, and then divide by the total number of nodes.

[0158] Compare the calculated new node load variance with the old node load variance. If the new node load variance is less than the old node load variance, it means that the data partition migration operation is effective and improves the load balance of the system. At this time, confirm that the migration is effective and use the updated data partition layout as the load balancing data distribution.

[0159] If the new node load variance is not less than the old node load variance, it indicates that the migration operation has not achieved the expected effect, and the load imbalance may be aggravated due to unreasonable data allocation during the migration process or other factors. At this time, roll back the updated data partition layout to the previous version, restore it to the state before migration, avoid negative impacts on system performance, and re-evaluate and adjust the data partition strategy, and try migration or other adjustment actions again.

[0160] In an alternative embodiment, S4 includes the following steps:

[0161] S41: Based on the load balancing data distribution, collect the actual load balance degree BalanceIndex real and the actual throughput Throughput real .

[0162] Specifically, in the distributed cluster, according to the implemented load balancing data distribution, the monitoring agents deployed on each node are used to collect the feedback data reflecting the system operation status in real time. The actual load balance degree BalanceIndex real is calculated through the hardware load indicators of the current nodes (such as CPU, GPU utilization, etc.), reflecting the balance of the load distribution among the nodes in the cluster; the actual throughput Throughput real counts the amount of data or tasks successfully processed by the system per unit time, directly measuring the data processing efficiency of the system. These feedback data provide a real and accurate performance evaluation basis for subsequent model parameter updates.

[0163] S42: Calculate the prediction error ∈ according to the actual load balance degree and the actual throughput; the calculation formula of the prediction error is:

[0164] ∈ = |BalanceIndex pred-BalanceIndex real |

[0165] +|Throughput pred -Throughput real |;

[0166] Among them, BalanceIndex pred is the predicted value of the load balancing degree obtained according to the optimal data partition adjustment action; Throughput pred is the predicted value of the throughput obtained according to the optimal data partition adjustment action.

[0167] Specifically, the collected actual load balancing degree BalanceIndex real and the actual throughput Throughput real are respectively compared with the predicted value of the load balancing degree BalanceIndex pred and the predicted value of the throughput Throughput pred predicted according to the optimal data partition adjustment action, and the prediction error ∈ is calculated according to the formula ∈ = |BalanceIndex pred -BalanceIndex real | + |Throughput pred -Throughput real |. This error comprehensively reflects the deviation degree of the deep reinforcement learning model in predicting the load balancing degree and throughput, and is an important indicator to measure the prediction accuracy of the model. A smaller ∈ value indicates that the model prediction fits well with the actual system operation, otherwise it means that there is room for improvement in the model.

[0168] S43: If the prediction error is greater than the preset error threshold, update the parameters θ of the deep Q-network model through the gradient descent algorithm; the update formula of the parameters θ is:

[0169]

[0170] Among them, λ is the second preset learning rate, ρ is the preset regularization coefficient, is the gradient of the parameter θ.

[0171] Specifically, when the prediction error ∈ exceeds the preset error threshold, it indicates that the prediction performance of the model is not ideal, and the parameters θ of the deep Q-network model need to be updated and optimized. Using the gradient descent algorithm, according to the gradient of the parameter θ, according to the formula Adjust the parameter values. Among them, the learning rate λ controls the step size of parameter update, and the regularization coefficient ρ is used to prevent the model from overfitting. By adding a penalty term of the parameter norm to the loss function, the model can maintain the simplicity of the parameters while fitting the data, improving the generalization ability of the model. The purpose of parameter update is to minimize the prediction error ∈, enabling the model to better adapt to the dynamic changes of the distributed cluster and enhancing the prediction accuracy and data processing performance in different scenarios.

[0172] The above-mentioned distributed data processing method constructs a normalized spatio-temporal state matrix by collecting multi-dimensional hardware loads and data features of heterogeneous nodes, combines a deep reinforcement learning model to dynamically predict the optimal partition adjustment actions and designs a reward function driven by the weighted load balance degree, adopts a double-buffer mechanism to gradually execute data migration and solves the differential data conflict based on the timestamp priority strategy. Finally, it uses the migrated real-time feedback data to online optimize the model parameters, comprehensively solving the problems of uneven load, resource waste and response lag of traditional static partition strategies in dynamic heterogeneous environments, realizing the efficient adaptive scheduling of distributed systems, maximizing resource utilization, elastic response to burst traffic and seamless migration of services without perception, and significantly enhancing the system stability and intelligent level in complex scenarios.

[0173] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown sequentially according to the indications of the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps does not have a strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same moment, but can be executed at different moments. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0174] Based on the same inventive concept, the embodiments of the present application also provide an apparatus for implementing the above-mentioned distributed data processing method. The solution provided by this apparatus for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the distributed data processing apparatus provided below can refer to the limitations of a distributed data processing method in the above text, and will not be repeated here.

[0175] In an exemplary embodiment, as Figure 2 shown, a distributed data processing apparatus 20 is provided, including:

[0176] A state generation module 21, configured to collect the hardware load metrics and data processing characteristics of each node in the distributed cluster, and generate a spatio-temporal state matrix based on the hardware load metrics and data processing characteristics.

[0177] An action prediction module 22, configured to predict an optimal data partition adjustment action through a deep reinforcement learning model based on the spatio-temporal state matrix.

[0178] A partition adjustment module 23, configured to perform a partition adjustment operation according to the optimal data partition adjustment action to obtain a load-balanced data distribution; the partition adjustment operation is to migrate partitions or merge partitions or split partitions or maintain the status quo.

[0179] A model update module 24, configured to perform data processing based on the load-balanced data distribution, collect feedback data during the data processing, and dynamically update the parameters of the deep reinforcement learning model based on the feedback data.

[0180] Optionally, the state generation module 21 includes:

[0181] A load collection unit 211, configured to collect the hardware load metrics of each node in the distributed cluster; the hardware load metrics include the CPU utilization rate C at time t i and the GPU video memory occupancy rate G at time t i , where i is the node number.

[0182] A feature extraction unit 212, configured to extract the data feature vectors of the data processed in the distributed cluster, and the data feature vectors include the key-value access frequency f at time t k , where k is the data key.

[0183] A data processing unit 213, configured to perform normalization processing on the hardware load metrics and data feature vectors to generate a spatio-temporal state matrix; the calculation formula of the spatio-temporal state matrix S(t) is:

[0184]

[0185] where, μ C represents the mean value of the CPU utilization rates of all nodes at time t, σ C represents the standard deviation of the CPU utilization rates of all nodes at time t, μ G represents the mean value of the GPU utilization rates of all nodes at time t, σ G represents the standard deviation of the GPU utilization rates of all nodes at time t, where, max(f) represents the maximum value among all key-value access frequencies, φ(·) is a time window encoding function, and T is a preset service cycle.

[0186] Optionally, the action prediction module 22 includes:

[0187] An action value function generation unit 221 is configured to construct a deep Q-network model, input a spatio-temporal state matrix S(t) into the deep Q-network model, and output an action value function Q(s, a); wherein, the action value function Q(s, a) represents the expected reward for executing an action a in a state s, the action space A = {migrate partition, merge partition, split partition, maintain the status quo}, s ∈ S(t) and a ∈ A.

[0188] A reward design unit 222 is configured to design a reward function R(t) based on the load balance degree, throughput, and migrated data volume. The expression of the reward function R(t) is:

[0189] R(t) = α·BalanceIndex(t) + β·Throughput(t) - ε·DataVolume(t);

[0190] wherein, BalanceIndex(t) represents the load balance degree at time t, which is calculated through hardware load metrics; Throughput(t) represents the throughput at time t; DataVolume(t) represents the migrated data volume at time t, and α, β, and ε are preset weight coefficients.

[0191] A function update unit 223 is configured to iteratively update the action value function Q(s, a) through the Bellman equation. The update formula of the action value function is:

[0192]

[0193] wherein, η is a first preset learning rate, γ is a preset discount factor, s′ is the state at the next moment, a′ is the action at the next moment, s′ ∈ S(t) and a′ ∈ A.

[0194] An action selection unit 224 is configured to select the action corresponding to the maximum Q value according to the updated action value function According to the action a opt Generate an optimal data partition adjustment action.

[0195] Optionally, the reward design unit 222 includes:

[0196] A standard deviation calculation sub-unit 2221 is configured to calculate the local load standard deviation of each node at time t according to the hardware load metrics; the calculation formula of the local load standard deviation is:

[0197]

[0198] wherein, is the local load standard deviation of node i in the distributed cluster at time t, L i is the load of node i at time t, μL is the average load of all nodes at time t.

[0199] A weight assignment subunit 2222, configured to assign a weight w according to the hardware type of each node i ; where w i represents the weight of node i.

[0200] An equilibrium degree calculation subunit 2223, configured to calculate a load equilibrium degree according to the weight w i The calculation formula of the load equilibrium degree is:

[0201]

[0202] where N is the total number of nodes in the distributed cluster.

[0203] Optionally, the action value function generation unit 221 includes:

[0204] A dimensionality reduction processing subunit 2211, configured to perform principal component analysis on the spatio-temporal state matrix S(t) to extract the first k-dimensional principal components S~(t); where S~(t) ∈ R k , and k is the preset dimensionality after dimensionality reduction.

[0205] A network construction subunit 2212, configured to construct a two-layer long short-term memory network, input the first k-dimensional principal components S~(t) into the first LSTM layer of the two-layer long short-term memory network according to time steps, and generate a first hidden state sequence and input the first hidden state into the second LSTM layer of the two-layer long short-term memory network to generate a final hidden state where d is the dimension of the hidden layer.

[0206] A value mapping subunit 2213, configured to map the final hidden state to the action value function Q(s,a) through the fully connected layer of the two-layer long short-term memory network, and the mapping formula is:

[0207]

[0208] where W h and W a are weight matrices, W h ∈ R m×d , W a ∈ R |A|×m ; h b and b a are bias terms, h b ∈ R m ,b a ∈ R |A|, where m is the number of hidden units in the fully connected layer, |A| is the total number of actions in the action space, and ReLU is the activation function.

[0209] Optionally, the partition adjustment module 23 includes:

[0210] A migration trigger unit 231, configured to trigger data partition migration when the Q value of the migration partition in the action value function Q(s,a) satisfies Q(s, migration partition) > δ·Q avg where δ is a preset dynamic threshold, and Q avg is the historical average Q value.

[0211] A migration execution unit 232, configured to execute data partition migration using a double-buffer mechanism to generate an updated data partition layout, including:

[0212] Create a new partition, write the newly arrived incremental data into the new partition, and at the same time keep the old partition to continue processing the read and write requests of the existing data.

[0213] Extract a full data snapshot from the old partition and synchronize the full data snapshot to the new partition.

[0214] Calculate the differential data ΔP = {d|d∈P new ∩P old ,d new .t > d old .t}, where P new is the new partition, P old is the old partition, d new .t and d old .t are the timestamps of the same data item in the new partition and the old partition respectively.

[0215] Compress and encode the differential data to generate a compressed package, and transmit the compressed package to the target storage node.

[0216] Redirect the client request from the old partition to the new partition, and release the memory and storage resources occupied by the old partition to complete the data partition migration and obtain the updated data distribution.

[0217] A verification unit 233, configured to verify the load balance of the updated data partition layout by calculating the new node load variance of the distributed cluster under the updated data distribution; if the new node load variance is less than the old node load variance, confirm that the migration is effective and use the updated data partition layout as the load-balanced data distribution; otherwise, roll back the updated data partition layout to the previous version.

[0218] Optionally, the model update module 24 includes:

[0219] A feedback data acquisition unit 241, configured to collect an actual load balancing degree BalanceIndex based on the load balancing data distribution real and an actual throughput Throughput real .

[0220] An error calculation unit 242, configured to calculate a prediction error ∈ according to the actual load balancing degree and the actual throughput; the calculation formula of the prediction error is:

[0221] ∈ = |BalanceIndex pred - BalanceIndex real |

[0222] + |Throughput pred - Throughput real |;

[0223] wherein, BalanceIndex pred is a predicted value of the load balancing degree obtained according to the optimal data partition adjustment action; Throughput pred is a predicted value of the throughput obtained according to the optimal data partition adjustment action.

[0224] A model parameter update unit 243, configured to update the parameters θ of the deep Q - network model by using a gradient descent algorithm when the prediction error is greater than a preset error threshold; the update formula of the parameters θ is:

[0225]

[0226] wherein, λ is a second preset learning rate, ρ is a preset regularization coefficient, is the gradient of the parameter θ.

[0227] An embodiment of the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps in the foregoing method embodiments are implemented.

[0228] An embodiment of the present application further provides a computer - readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

[0229] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0230] The above embodiments only represent several implementation manners of the embodiments of the present application. The descriptions are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent for the embodiments of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the embodiments of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A distributed data processing method, characterized in that, The method includes: S1: Collect the hardware load metrics and data processing characteristics of each node in the distributed cluster, and generate a spatio-temporal state matrix based on the hardware load metrics and the data processing characteristics; S2: Based on the spatio-temporal state matrix, predict the optimal data partition adjustment action through a deep reinforcement learning model; S3: According to the optimal data partition adjustment action, perform a partition adjustment operation to obtain a load-balanced data distribution; the partition adjustment operation is to migrate partitions, merge partitions, split partitions, or maintain the status quo; S4: Perform data processing based on the load-balanced data distribution, collect feedback data during the data processing, and dynamically update the parameters of the deep reinforcement learning model based on the feedback data.

2. The method according to claim 1, wherein The S1 includes: S11: Collect the hardware load metrics of each node in the distributed cluster; the hardware load metrics include the CPU utilization rate C at time t i and the GPU video memory occupancy rate G at time t i , where i is the node number; S12: Extract the data feature vector of the data processed in the distributed cluster, where the data feature vector includes the key-value access frequency f at time t k , where k is the data key; S13: Normalize the hardware load metrics and the data feature vectors to generate the spatio-temporal state matrix; the calculation formula of the spatio-temporal state matrix S(t) is: Among them, μ C represents the mean value of the CPU utilization rates of all the nodes at time t, and σ C represents the standard deviation of the CPU utilization rates of all the nodes at time t, μ G represents the mean value of the GPU utilization rates of all the nodes at time t, and σ G represents the standard deviation of the GPU utilization rates of all the nodes at time t, where max(f) represents the maximum value among all the key-value access frequencies, φ(·) is a time-window encoding function, and T is a preset service cycle.

3. The method according to claim 2, characterized in that The S2 includes: S21: Construct a deep Q-network model, input the spatio-temporal state matrix S(t) into the deep Q-network model, and output the action value function Q(s,a); where, the action value function Q(s,a) represents the expected return of executing action a in state s, the action space A = {migrate partitions, merge partitions, split partitions, maintain the status quo}, s ∈ S(t) and a ∈ A; S22: Design a reward function R(t) based on the load balance degree, throughput, and the amount of migrated data. The expression of the reward function R(t) is: R(t) = α·BalanceIndex(t) + β·Throughput(t) - ε·DataVolume(t); where, BalanceIndex(t) represents the load balance degree at time t, calculated from the hardware load metrics; Throughput(t) represents the throughput at time t; DataVolume(t) represents the amount of migrated data at time t, and α, β, ε are preset weight coefficients; S23: Iteratively update the action value function Q(s,a) through the Bellman equation. The update formula of the action value function is: where, η is the first preset learning rate, γ is the preset discount factor, s′ is the next moment state, a′ is the next moment action, s′ ∈ S(t) and a′ ∈ a; S24: Select the action corresponding to the maximum Q value according to the updated action value function According to the action a opt Generate the optimal data partition adjustment action.

4. The method according to claim 3, wherein Calculating the load balance degree includes: S221: Calculate the local load standard deviation of each node at time t according to the hardware load metrics. The calculation formula of the local load standard deviation is: Among them, is the standard deviation of the local load of node i in the distributed cluster at time t, L i is the load of node i at time t, μ L is the average load of all nodes at time t; S222: Assign weights w according to the hardware types of the respective nodes i ; where w i represents the weight of node i; S223: According to the weight w i calculate the load balancing degree, and the calculation formula of the load balancing degree is: where, N is the total number of nodes in the distributed cluster.

5. The method according to claim 3, characterized in that, The inputting the spatio-temporal state matrix S(t) into the deep Q-network model and outputting the action value function Q(s,a) includes: S211: Perform principal component analysis on the spatio-temporal state matrix S(t) to extract the first k-dimensional principal components S~(t); where S~(t) ∈ R k , and k is the dimension after preset dimensionality reduction; S212: Construct a double-layer long short-term memory network, input the first k-dimensional principal components S~(t) into the first LSTM layer of the double-layer long short-term memory network according to time steps, and generate a first hidden state sequence and input the first hidden state into the second LSTM layer of the double-layer long short-term memory network to generate a final hidden state where d is the dimension of the hidden layer; S213: Map the final hidden state to the action value function Q(s,a) through the fully connected layer of the double-layer long short-term memory network, and the mapping formula is: ​ Among them, W h and W a are weight matrices, W h ∈R m×d , W a ∈R |A|×m ; h b and b a are bias terms, h b ∈R m , b a ∈R |A| , m is the number of hidden units in the fully connected layer, |A| is the total number of actions included in the action space, and ReLU is the activation function.

6. The method according to claim 3, characterized in that The S3 includes: S31: When the Q value of the migration partition in the action value function Q(s,a) satisfies Q(s, migration partition) > δ·Q avg a data partition migration is triggered; where δ is a preset dynamic threshold, and Q avg is the historical average Q value; S32: Use a double-buffer mechanism to perform the data partition migration and generate an updated data partition layout, including: Create a new partition, write the newly arrived incremental data into the new partition, and at the same time keep the old partition continuing to process the read and write requests of the existing data. Extract a full - volume data snapshot from the old partition and synchronize the full - volume data snapshot to the new partition; Calculate the difference data ΔP = {d|d ∈ P new ∩P old , d new .t > d old .t}, where P new is the new partition, P old is the old partition, d new .t and d old .t are the timestamps of the same data item in the new partition and the old partition respectively; Compress and encode the differential data to generate a compressed package, and transfer the compressed package to the target storage node; Redirect the client request from the old partition to the new partition, and release the memory and storage resources occupied by the old partition to complete the data partition migration, obtaining the updated data distribution; S33: Verify the load balance of the updated data partition layout by calculating the variance of the new node loads in the updated data distribution of the distributed cluster; if the variance of the new node loads is less than the variance of the old node loads, confirm that the migration is effective and use the updated data partition layout as the load - balanced data distribution; otherwise, roll back the updated data partition layout to the previous version.

7. The method according to any one of claims 3 to 6, characterized in that, The S4 includes: S41: Based on the load balancing data distribution, collect the actual load balancing degree BalanceIndex real and the actual throughput Throughput real ; S42: Calculate the prediction error ∈ according to the actual load balance degree and the actual throughput; the calculation formula of the prediction error is: ∈ = |BalanceIndex pred -BalanceIndex real | +|Throughput pred -Throughput real |; Among them, BalanceIndex pred is the predicted value of the load balancing degree obtained according to the optimal data partition adjustment action; Throughput pred is the predicted value of the throughput obtained according to the optimal data partition adjustment action; S43: If the prediction error is greater than the preset error threshold, update the parameters θ of the deep Q - network model through the gradient descent algorithm; the update formula of the parameters θ is: where λ is the second preset learning rate and ρ is the preset regularization coefficient, is the gradient of the parameter θ.

8. A distributed data processing device, characterized in that, The device includes: A state generation module, configured to collect the hardware load indicators and data processing characteristics of each node in the distributed cluster, and generate a spatio - temporal state matrix based on the hardware load indicators and the data processing characteristics; An action prediction module, configured to predict the optimal data partition adjustment action through a deep reinforcement learning model based on the spatio - temporal state matrix; A partition adjustment module, configured to perform a partition adjustment operation according to the optimal data partition adjustment action to obtain a load - balanced data distribution; the partition adjustment operation is to migrate partitions or merge partitions or split partitions or maintain the status quo; A model update module, configured to perform data processing based on the load - balanced data distribution, collect feedback data during the data processing, and dynamically update the parameters of the deep reinforcement learning model based on the feedback data.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 7.

Images