An Adaptive Adjustment Method for HBase Region-Level Migration Rate Based on Private AI

By adopting a region-level migration rate adaptive adjustment method based on private AI, the problems of coarse-grained rate limiting and resource waste in HBase data migration are solved. This method achieves region-level migration rate optimization and resource utilization, reducing the impact on business performance and IO shock.

CN123115972APending Publication Date: 2026-11-17YUEZHI ZHONGSHENG (SHANGHAI) INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611566859.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-10
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing HBase data migration tools have coarse-grained rate limiting, making it difficult to adjust the migration rate according to the real-time load of the Region, resulting in wasted resources or degraded business performance. Furthermore, migration tasks can easily cause IO shocks when combined with Major Compaction.

Method used

A region-level migration rate adaptive adjustment method based on private AI is adopted. By collecting and preprocessing the operating metrics of RegionServer and its Region, a health scoring model and a compaction prediction model are constructed to dynamically adjust the migration rate to avoid resource waste and IO impact.

Benefits of technology

It enables region-level migration rate adjustment, optimizes resource utilization, reduces the impact on business performance, and predicts and adjusts the migration rate before Major Compaction to reduce IO impact.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115972A_ABST
    Figure CN123115972A_ABST
Patent Text Reader

Abstract

This invention discloses a method, system, electronic device, and storage medium for adaptive adjustment of HBase Region-level migration rate based on private AI, relating to the field of distributed database data migration technology. The method collects operational metrics of RegionServers and their respective Regions according to a preset sampling period, preprocesses them to form standardized feature vectors, calculates health scores using a Region health scoring model, and predicts the probability of triggering Major Compaction within a future time window based on historical Compaction logs and the current StoreFile status. An initial migration rate is determined based on the health score, and then adjusted a second time based on the predicted probability. A smoothing constraint is applied to the rate during continuous migration, and migration is executed through an independent rate limiter corresponding to each Region. Business latency and actual migration throughput are collected to update model weights. This invention can adjust the migration rate for different Regions separately and slow down or pause it before Major Compaction occurs, balancing migration efficiency and online business stability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of distributed database data migration and migration flow control technology, and in particular to a method, system, electronic device and computer-readable storage medium for adaptive adjustment of HBase Region-level migration rate based on private AI. Background Technology

[0002] HBase is a distributed column-oriented database. HBase tables divide data into multiple Regions based on key-value ranges, and each Region is managed by a different RegionServer. A Region is the basic unit for data distribution and load management in HBase; different Regions typically exhibit significant differences in access frequency, data write volume, number of StoreFiles, and background task status.

[0003] When migrating data from a source HBase cluster to a target HBase cluster, methods such as Export / Import, CopyTable, Snapshot, or HBase Replication are commonly used. To avoid impacting production operations on migration traffic, existing migration tools typically set global bandwidth limits, restrict the number of migration threads per table, or pause migrations during peak business periods. These control parameters are generally preset by operations personnel and remain unchanged for an extended period.

[0004] When the fixed migration rate is set low, migration is still performed at a low rate during off-peak periods, and the available CPU, disk I / O and network bandwidth of the cluster cannot be fully utilized, thus prolonging the migration cycle. When the migration rate is set high, migration read and write traffic will compete with online requests for the CPU, memory, disk I / O and network resources of the RegionServer, which can easily cause increased P99 latency or even request timeouts.

[0005] Furthermore, existing migration tools often use tables or clusters as the uniform rate limiting granularity, making it difficult to adjust the migration rate according to the actual load of individual regions. When hot regions and low-load regions are subjected to the same rate, it may degrade the business performance of hot regions and prevent the remaining resources of low-load regions from being utilized.

[0006] HBase's Major Compaction merges multiple HFiles within a single Region, consuming significant disk I / O and CPU resources. If data migration and Major Compaction occur concurrently on the same Region, it can cause excessive instantaneous I / O pressure. Manually reducing the migration rate only after an anomaly occurs results in a delayed response. Therefore, a technical solution is needed that can sense the Region-level operational status, predict Major Compaction risks, and automatically adjust the migration rate of each Region. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system, electronic device, and computer-readable storage medium for adaptive adjustment of HBase Region-level migration rate based on private AI. This addresses the problems of existing migration tools having coarse-grained rate limiting, failing to fully utilize available resources based on real-time load, and easily affecting online services when migration tasks are superimposed with Major Compaction.

[0008] To achieve the above objectives, this invention provides a method for adaptive adjustment of HBase Region-level migration rate based on private AI, applied to data migration from a source HBase cluster to a target HBase cluster, comprising: The operation metrics of each RegionServer and its Region are collected according to a preset sampling period. The operation metrics of the RegionServer hosting the corresponding Region are associated with the Region-level operation metrics of the corresponding Region. The operation metrics are preprocessed to construct a standardized feature vector corresponding to each Region. The standardized feature vectors are input into the Region health score model to calculate the health score of each Region, and the health status of each Region is determined based on the health score. Based on historical compaction logs and the current StoreFile status, the probability of each Region triggering Major compaction within a preset future time window is predicted using a compaction prediction model. The initial migration rate is calculated based on the health score of each Region, and the initial migration rate is adjusted a second time according to the corresponding probability to obtain the target migration rate of each Region. Applying a rate smoothing constraint of adjacent adjustment periods to the target migration rate of a Region that does not meet the suspension condition to obtain a final migration rate, and issuing the final migration rate to an independent rate limiter corresponding to the respective Region; for a Region that meets the suspension condition, setting its final migration rate to zero; performing data migration respectively at the corresponding final migration rate by using each independent rate limiter with a Region as the minimum control granularity; collecting service delay and actual migration throughput during the migration process to form feedback data, and updating weight parameters of the Region health degree scoring model according to the feedback data.

[0009] Further, the preset sampling period is 10 seconds; the operation metrics include: RegionServer process CPU utilization, user mode and kernel mode proportion, heap memory usage, old generation usage, Full GC frequency, read QPS, write QPS, P99 read latency, P99 write latency, number of StoreFiles, total number of HFiles, compression ratio, Compaction queue length, number of ongoing Compaction tasks and estimated completion time, inbound bandwidth, outbound bandwidth, network card packet loss rate, disk IO utilization, IOPS and average waiting time; the preprocessing includes data cleaning, normalization and feature extraction, and the normalization maps each operation metric to the interval [0,1].

[0010] Further, the Region health degree scoring model adopts a weighted scoring method, and calculates the health degree score H according to the following formula: H=w1×(100%-CPU utilization)+w2×(100%-memory usage)+w3×(100%-GC pressure index)+w4×(100%-QPS pressure index)+w5×(100%-Compaction pressure index)+w6×(100%-disk IO pressure index); where w1 to w6 are weight parameters obtained by training based on historical data, and the sum of w1 to w6 is 1; the health states include a healthy state, a sub-healthy state and a dangerous state, wherein H≥70 is determined as the healthy state, 30<H<70 is determined as the sub-healthy state, and H≤30 is determined as the dangerous state.

[0011] Further, the input features of the Compaction prediction model include the current number of StoreFiles, the average size of a single StoreFile, the write traffic in the latest 1 hour, and the Compaction interval rule of similar Regions in history, and the preset future time window is 30 minutes; when the predicted probability is greater than 80% and less than or equal to 95%, the migration rate of the corresponding Region is reduced in advance; when the predicted probability is greater than 95%, the data migration of the corresponding Region is suspended, and the adaptive adjustment of the migration rate for this Region is resumed after the Major Compaction is completed.

[0012] Further, a base rate R_base is set in advance according to the total bandwidth of the cluster, and the target migration rate R_target of each Region is determined according to the following rules: When H≤30 or the probability P>95%, R_target=0; When H>30 and 80%<P≤95%, R_target=R_base×0.5; When P≤80% and H≥70, R_target=R_base×min(2.0,1+(H-70) / 30); When P≤80% and 30<H<70, R_target=R_base×(H / 70).

[0013] Further, when the Region is in a continuous migration state, the adjustment range of the migration rate between two adjacent adjustment cycles is limited to no more than ±20%, and the final migration rate R_current of the current adjustment cycle is determined according to the following formula: R_current=max(R_prev×0.8,min(R_prev×1.2,R_target)); Wherein, R_prev is the final migration rate of the previous adjustment cycle, and R_target is the target migration rate of the current adjustment cycle.

[0014] Further, the feedback data includes the service P99 delay variation curve during the migration, the actual migration throughput, and the correspondence between the health score and the actual service performance; the feedback data is added to the training set, and the Region health scoring model is retrained according to the preset update cycle to update the weight parameters.

[0015] The present invention further provides a private AI-based HBase Region-level migration rate adaptive adjustment system, which is configured to perform data migration from a source HBase cluster to a target HBase cluster, comprising: The data acquisition module is used to collect the operating metrics of each RegionServer and its respective Region according to a preset sampling period; The feature engineering and vectorization module is used to associate the RegionServer-level operation metrics of the corresponding Region with the Region-level operation metrics of the corresponding Region, and to preprocess the operation metrics to construct standardized feature vectors corresponding to each Region. The AI ​​intelligent analysis engine includes a Region health scoring model and a Compaction prediction model. The Region health scoring model is used to calculate the health score of each Region based on the standardized feature vectors. The Compaction prediction model is used to predict the probability of each Region triggering Major Compaction within a preset future time window based on historical Compaction logs and the current StoreFile status. The migration rate decision engine is used to calculate the corresponding initial migration rate based on the health score of each Region, adjust the initial migration rate a second time according to the corresponding probability, and apply a rate smoothing constraint of adjacent adjustment cycles to the migration rate of Regions that do not meet the pause condition, so as to obtain the final migration rate of each Region. The Region-level migration execution controller includes an independent rate limiter corresponding to each Region, which is used to perform data migration according to the corresponding final migration rate with the Region as the smallest control granularity. The feedback optimization module is used to collect business latency and actual migration throughput during the migration process, form feedback data, and update the weight parameters of the Region health scoring model based on the feedback data.

[0016] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the above-described adaptive adjustment method for HBase Region-level migration rate based on private AI.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the above-described adaptive adjustment method for HBase Region-level migration rate based on private AI is implemented.

[0018] Compared with the prior art, the present invention has at least the following beneficial effects: First, the Region is used as the smallest control granularity for migration rate, and an independent rate limiter is set for each Region. Different migration rates can be executed for hot Regions, unhealthy Regions, and low-load Regions, avoiding resource waste or local business performance degradation caused by table-level or cluster-level uniform rate limiting.

[0019] Secondly, the Region health scoring model integrates multiple indicators such as CPU, memory, GC, requests, compaction, and disk I / O to form a quantitative health score, enabling the migration rate to dynamically change with the real-time running status of the Region and its RegionServer.

[0020] Third, the compaction prediction model outputs the trigger probability before a major compaction occurs and links this probability with the migration rate decision. This can reduce or pause the migration of the corresponding region before resource contention intensifies, thereby reducing the IO impact caused by the superposition of migration traffic and background merging tasks.

[0021] Fourth, applying a ±20% smoothing constraint to adjacent rate adjustments during continuous migration can suppress network and storage load fluctuations caused by drastic changes in migration rate.

[0022] Fifth, by updating the model weights using the correlation between business P99 latency, actual migration throughput, and health score and business performance, the health scoring model can be gradually adapted to the current cluster's operating characteristics. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the HBase Region-level migration rate adaptive adjustment method provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the architecture of the HBase Region-level migration rate adaptive adjustment system provided by the present invention.

[0025] Figure 3 This is a schematic diagram of the Region health score and migration rate decision-making process provided by the present invention.

[0026] Figure 4 This is a timing diagram illustrating the adaptive adjustment of migration rate provided by the present invention.

[0027] The main labels in the attached figures are: 101. Operational index collection and preprocessing; 102. Standardized feature vector construction; 103. Region health score; 104. Major compaction probability prediction; 105. Migration rate decision and smoothing; 106. Region-independent rate limiting execution; 107. Feedback data collection and model update; 201. Data acquisition module; 202. Feature engineering and vectorization processing module; 203. AI intelligent analysis engine; 2031. Region health score model; 2032. Compaction prediction model; 204. Migration rate decision engine; 205. Region-level migration execution controller; 206. Feedback optimization module. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0029] The migration rate in this specification refers to the throughput of data replication from the source HBase cluster to the target HBase cluster per unit time, which can be expressed in MB / s or records / s. RegionServer-level performance metrics reflect the resource status of the service processes or nodes hosting the Region, while Region-level performance metrics reflect the status of requests, storage files, and background tasks for the corresponding Region.

[0030] On-Premises AI refers to deploying artificial intelligence models, inference services, training or fine-tuning tools, and related data processing systems in an enterprise's own data center, private cloud, or other proprietary computing environment, which is built, managed, and maintained by the enterprise itself. Its core characteristic is that model operation and data processing are completed within a secure boundary controlled by the enterprise, and business data is generally not transmitted to external public cloud platforms, thereby improving data security, privacy protection, and system autonomy and controllability. On-Premises AI allows for customization, fine-tuning, and continuous optimization of models based on the enterprise's business scenarios and internal data, making it suitable for application scenarios with high requirements for data confidentiality, compliance management, network isolation, and system stability.

[0031] This implementation can be achieved using Java combined with HBase's native API. The migration control node collects operational metrics through the RegionServer JMX interface, obtains Region distribution information and compaction status through the HBase Admin API, and determines the relationship between each Region and its corresponding RegionServer. The machine learning model can be implemented using Apache Spark MLlib or Deeplearning4j and deployed on the migration control node to minimize modifications to the internal service processes of the production cluster.

[0032] like Figure 1 As shown, the method of this embodiment includes steps 101 to 107.

[0033] Step 101: Operational Metrics Collection and Preprocessing. During the data migration from the source HBase cluster to the target HBase cluster, operational metrics for each RegionServer and its respective Region are collected at a sampling period of 10 seconds. Data collection can be performed by a JMX metrics collector, a RegionServer log collector, a compaction status monitor, an audit log collector, and a network I / O collector.

[0034] Performance metrics include CPU-related metrics, memory-related metrics, request-related metrics, storage-related metrics, background task-related metrics, network-related metrics, and disk-related metrics. CPU-related metrics include RegionServer process CPU utilization and the ratio of user-mode to kernel-mode CPU utilization; memory-related metrics include heap memory utilization, old generation utilization, and Full GC frequency; request-related metrics include read QPS, write QPS, P99 read latency, and P99 write latency.

[0035] Storage-related metrics include the number of StoreFiles, the total number of HFiles, and the compression ratio; background task-related metrics include the compaction queue length, the number of compaction tasks currently being executed, and the estimated completion time; network-related metrics include inbound bandwidth, outbound bandwidth, and network interface card packet loss rate; disk-related metrics include disk I / O utilization, IOPS, and average latency.

[0036] Based on the Region distribution information obtained through the HBase Admin API, each Region is associated with the RegionServer currently hosting that Region. This ensures that the feature data of the corresponding Region simultaneously includes the resource status of its respective RegionServer and the business, storage, and background task status of the Region itself. The collected raw data is cleaned, normalized, and features are extracted, mapping indicators of different dimensions to the [0,1] interval.

[0037] Step 102: Construction of Standardized Feature Vectors. The normalized results are combined according to a predetermined index order to form a standardized feature vector V=[v1,v2,...,vn] corresponding to each Region. The standardized feature vectors serve as input data for the Region health scoring model and the Compaction prediction model.

[0038] Step 103: Region health score calculation. The Region health scoring model calculates a health score H ranging from 0 to 100 based on CPU utilization, memory usage, GC pressure index, QPS pressure index, Compaction pressure index and disk I / O pressure index. Each pressure index is represented by a normalized indicator of the corresponding category, and a larger value indicates higher corresponding resource or service pressure.

[0039] In this embodiment, a weighted scoring method is used to calculate the health score H. The weights w1 to w6 are obtained by training on historical data, and the sum thereof is 1. The initial weights can be set according to expert experience, for example: w1=0.20, w2=0.20, w3=0.15, w4=0.15, w5=0.15, w6=0.15.

[0040] Regions are divided into three levels according to the health score H: when H ≥ 70, the Region is in a healthy state, indicating that the Region has sufficient service capacity; when 30 < H < 70, the Region is in a sub-healthy state, and the migration rate needs to be reduced; when H ≤ 30, the Region is in a dangerous state, and data migration of this Region is suspended to prioritize ensuring online services.

[0041] Step 104: Major Compaction probability prediction. Historical Compaction logs and current StoreFile status are read, with the current number of StoreFiles, the average size of a single StoreFile, the write traffic in the last hour, and the Compaction interval pattern of similar Regions in history as input features, and a Compaction prediction model is used to output the probability P that a Major Compaction will be triggered in the corresponding Region within the next 30 minutes.

[0042] The Compaction prediction model can form training data from historical Compaction logs. In addition to the prediction model obtained by training, a rule-based prediction method can also be adopted; when there is continuous time series data, an LSTM time series model can also be used to predict the next trigger time according to the historical Compaction interval sequence.

[0043] Step 105: Migration rate decision and smoothing processing. The administrator presets a base rate R_base according to the total cluster bandwidth. The base rate can be initially allocated among Regions to be migrated according to the total bandwidth, and then the migration rate decision engine calculates the target migration rate R_target respectively according to the health score H and the Major Compaction trigger probability P of each Region.

[0044] As Figure 3As shown, the initial migration rate is first determined according to the health score: the migration rate is increased in the healthy state, with a maximum not exceeding 2.0 times the base rate; the migration rate is reduced at a ratio of H / 70 in the sub-healthy state; the migration rate is set to zero in the dangerous state. Then, secondary adjustment is performed by superimposing the Major Compaction trigger probability.

[0045] To avoid overlapping of judgment intervals, the migration rate decision engine performs judgment in the order of priority of the suspension condition: when H≤30 or P>95%, R_target=0; when H>30 and 80%<P≤95%, R_target=R_base×0.5; when P≤80% and H≥70, R_target=R_base×min(2.0,1+(H-70) / 30); when P≤80% and 30<H<70, R_target=R_base×(H / 70).

[0046] When the prediction probability P is greater than 80% and the suspension condition is not met, the migration rate can be reduced to 50% of the base rate within a preset advance period, which is, for example, 30 seconds. When the prediction probability P is greater than 95%, the data migration of the corresponding Region is suspended until the Major Compaction is completed.

[0047] For Regions in a continuous migration state with a target migration rate not equal to zero, the adjustment range of the migration rate between two adjacent adjustment cycles is limited to no more than ±20%. When the suspension condition is met as H≤30 or P>95%, the final migration rate is directly set to zero. After the suspension condition is lifted, the migration rate in the recovery phase is re-determined according to the current health score and the Major Compaction trigger probability, and then the smooth constraint under the continuous migration state is continued to be executed.

[0048] Step 106: Independent speed limit execution for Regions. The Region-level migration execution controller maintains an independent speed limiter instance for each Region to be migrated, and issues the final migration rate of each Region to the corresponding independent speed limiter. The independent speed limiter can make hot Regions, sub-healthy Regions and low-load Regions migrate in parallel at different rates by limiting the maximum read and write bandwidth of the corresponding Region or controlling the sending rhythm of the migration thread.

[0049] Step 107: Feedback data collection and model update. During the migration execution period and after each migration is completed, the business P99 delay change curve, the actual migration throughput, and the correspondence between the health score and the actual business performance are collected, and the above feedback data is added to the training set. The weight parameters in the Region health scoring model are retrained according to a preset update cycle, for example, every week.

[0050] The feedback optimization module can compare the correlation between health scores, actual business latency, and migration throughput, and adjust the weight parameters accordingly. Feedback optimization can also employ online learning for real-time incremental updates. After the updated model is put into use, subsequent sampling cycles continue to execute steps 101 to 107, thus forming a closed-loop control.

[0051] like Figure 2 As shown, the system in this embodiment includes a data acquisition module 201, a feature engineering and vectorization processing module 202, an AI intelligent analysis engine 203, a migration rate decision engine 204, a region-level migration execution controller 205, and a feedback optimization module 206.

[0052] The data acquisition module 201 collects runtime data from each node of the HBase cluster. The feature engineering and vectorization processing module 202 completes the association between Region and RegionServer, raw indicator cleaning, normalization, and feature extraction. The AI ​​intelligent analysis engine 203 includes a Region health scoring model 2031 and a Compaction prediction model 2032, which output the health score H and the Major Compaction trigger probability P, respectively.

[0053] The migration rate decision engine 204 determines the final migration rate for each Region based on H and P. The Region-level migration execution controller 205 includes multiple independent rate limiters, each corresponding to a Region. The feedback optimization module 206 feeds the migration results back to the AI ​​intelligent analysis engine 203 to update the weight parameters of the Region health scoring model 2031.

[0054] Figure 4 The state changes of three representative Regions during a continuous adjustment period are shown. Region A maintains a health level above 70, and its migration rate remains at a high level under smoothing constraints; Region B's health level gradually decreases from 65 to below 30, its migration rate decreases accordingly, and it pauses after entering a dangerous state; Region C pauses migration after entering a dangerous state from a sub-healthy state, and then re-enters adaptive speed regulation after its health level recovers and Major Compaction ends.

[0055] exist Figure 4In the example shown, the prediction probability of Major Compaction for Region C gradually increases from 40% and 70% to 85% and 95%, and continues to increase during Major Compaction execution. Early avoidance is triggered when the probability exceeds 80%; the system remains paused when the probability exceeds 95% or when Major Compaction is in progress. After Major Compaction completes and health is restored, the system recalculates the migration rate for the recovery phase. This timing is only used to illustrate the relationship between health, prediction probability, and migration rate.

[0056] In other implementations, the Region health scoring model can use random forests, gradient boosting trees, or neural network models instead of weighted scoring methods. The input remains features reflecting the operational status of the Region and its associated RegionServer, and the output remains a health score or corresponding health status. The migration rate mapping relationship can also use a sigmoid function or a piecewise function.

[0057] Data acquisition can be implemented using the JMX interface, the HBase Metrics framework, or a custom RegionObserver coprocessor. Besides Java, Scala or Python can also be used. These alternative methods do not change the technical concept of using the Region as the smallest unit of control and jointly adjusting the migration rate using health scores and Major Compaction trigger probabilities.

[0058] This embodiment also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, steps 101 to 107 described above are completed. The electronic device can be configured as a migration control node outside the HBase cluster.

[0059] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the above-described HBase Region-level migration rate adaptive adjustment method. The computer-readable storage medium may be a read-only memory, random access memory, disk, optical disk, or other medium capable of storing program code.

[0060] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for adaptive adjustment of HBase Region-level migration rate based on private AI, applied to data migration from a source HBase cluster to a target HBase cluster, characterized in that, include: The operation metrics of each RegionServer and its Region are collected according to a preset sampling period. The operation metrics of the RegionServer hosting the corresponding Region are associated with the Region-level operation metrics of the corresponding Region. The operation metrics are preprocessed to construct a standardized feature vector corresponding to each Region. The standardized feature vectors are input into the Region health score model to calculate the health score of each Region, and the health status of each Region is determined based on the health score. Based on historical compaction logs and the current StoreFile status, the probability of each Region triggering Major compaction within a preset future time window is predicted using a compaction prediction model. The initial migration rate is calculated based on the health score of each Region, and the initial migration rate is adjusted a second time according to the corresponding probability to obtain the target migration rate of each Region. Apply rate smoothing constraints of adjacent adjustment cycles to the target migration rate of Regions that do not meet the pause conditions to obtain the final migration rate, and send the final migration rate to the independent speed limiter corresponding to the corresponding Region; for Regions that meet the pause conditions, set their final migration rate to zero. Each of the independent speed limiters performs data migration according to the corresponding final migration rate, with the Region as the smallest control granularity. Collect business latency and actual migration throughput during the migration process to form feedback data, and update the weight parameters of the Region health scoring model based on the feedback data.

2. The HBase Region-level migration rate adaptive adjustment method based on private AI according to claim 1, characterized in that, The preset sampling period is 10 seconds; the operating metrics include: RegionServer process CPU utilization, user mode and kernel mode ratio, heap memory utilization, old generation utilization, Full GC frequency, read QPS, write QPS, P99 read latency, P99 write latency, number of StoreFiles, total number of HFiles, compression ratio, compaction queue length, number of currently executing compaction tasks and estimated completion time, inbound bandwidth, outbound bandwidth, network card packet loss rate, disk IO utilization, IOPS and average waiting time; the preprocessing includes data cleaning, normalization and feature extraction, and the normalization maps each operating metric to the [0,1] interval.

3. The HBase Region-level migration rate adaptive adjustment method based on private AI according to claim 1, characterized in that, The Region health score model uses a weighted scoring method and calculates the health score H according to the following formula: H = w1 × (100% - CPU utilization) + w2 × (100% - memory utilization) + w3 × (100% - GC pressure index) + w4 × (100% - QPS pressure index) + w5 × (100% - Compaction pressure index) + w6 × (100% - disk I / O pressure index); where w1 to w6 are weight parameters trained from historical data, and the sum of w1 to w6 is 1; the health states include a healthy state, a sub-healthy state and a dangerous state, the healthy state is determined when H≥70, the sub-healthy state is determined when 30<H<70, and the dangerous state is determined when H≤30.

4. The HBase Region-level migration rate adaptive adjustment method based on private AI according to claim 1, characterized in that, The input features of the Compaction prediction model include the current number of StoreFiles, the average size of a single StoreFile, the write traffic in the most recent hour, and the Compaction interval pattern of similar Regions in history, and the preset future time window is 30 minutes; when the predicted probability is greater than 80% and less than or equal to 95%, the migration rate of the corresponding Region is reduced in advance; when the predicted probability is greater than 95%, the data migration of the corresponding Region is paused, and the adaptive adjustment of the migration rate for this Region is resumed after the Major Compaction is completed.

5. The HBase Region-level migration rate adaptive adjustment method based on private AI according to claim 3, characterized in that, A base rate R_base is set in advance according to the total cluster bandwidth, and the target migration rate R_target of each Region is determined according to the following rules: When H≤30 or the probability P>95%, R_target=0; When H>30 and 80%<P≤95%, R_target=R_base×0.5; When P≤80% and H≥70, R_target=R_base×min(2.0,1+(H-70) / 30); When P≤80% and 30<H<70, R_target=R_base×(H / 70).

6. The HBase Region-level migration rate adaptive adjustment method based on private AI according to claim 1, characterized in that, When the Region is in a continuous migration state, the adjustment range of the migration rate between two adjacent adjustment cycles is limited to no more than ±20%, and the final migration rate R_current of the current adjustment cycle is determined according to the following formula: R_current=max(R_prev×0.8,min(R_prev×1.2,R_target)); where R_prev is the final migration rate of the previous adjustment cycle, and R_target is the target migration rate of the current adjustment cycle.

7. The HBase Region-level migration rate adaptive adjustment method based on private AI according to claim 1, characterized in that, The feedback data includes the service P99 delay variation curve during the migration, the actual migration throughput, and the correspondence between the health score and the actual service performance; The feedback data is added to the training set, and the Region health scoring model is retrained according to a preset update period to update the weight parameters.

8. A proprietary AI-based HBase Region-level migration rate adaptive adjustment system for performing data migration from a source HBase cluster to a target HBase cluster, characterized in that... comprising: a data acquisition module, configured to acquire operation indicators of each RegionServer and Regions belonging thereto according to a preset sampling period; a feature engineering and vectorization processing module, configured to associate the RegionServer-level operation indicators of the RegionServer hosting the corresponding Region with the Region-level operation indicators of the corresponding Region, preprocess the operation indicators, and construct standardized feature vectors respectively corresponding to each Region; The AI ​​intelligent analysis engine includes a Region health scoring model and a Compaction prediction model. The Region health scoring model is used to calculate the health score of each Region based on the standardized feature vectors. The Compaction prediction model is used to predict the probability of each Region triggering Major Compaction within a preset future time window based on historical Compaction logs and the current StoreFile status. The migration rate decision engine is used to calculate the corresponding initial migration rate based on the health score of each Region, adjust the initial migration rate a second time according to the corresponding probability, and apply a rate smoothing constraint of adjacent adjustment cycles to the migration rate of Regions that do not meet the pause condition, so as to obtain the final migration rate of each Region. The Region-level migration execution controller includes an independent rate limiter corresponding to each Region, which is used to perform data migration according to the corresponding final migration rate with the Region as the smallest control granularity. The feedback optimization module is used to collect business latency and actual migration throughput during the migration process, form feedback data, and update the weight parameters of the Region health scoring model based on the feedback data.

9. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the HBase Region-level migration rate adaptive adjustment method based on private AI as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the HBase Region-level migration rate adaptive adjustment method based on private AI as described in any one of claims 1 to 7.