A cloud migration diffusion prediction and mirror split cooperative control method

Abstract

This invention discloses a collaborative control method for diffusion prediction and mirroring offloading in cloud migration, relating to the field of big data analytics. The invention collects end-to-end evidence through an observation window and saves it as a snapshot; it divides the cloud into service, database, and storage tiers and configures budget and SLO constraints; it constructs a basic matrix by statistically counting inter-tier transitions, obtains an inter-tier transition matrix based on multi-factor correction, constructs a dissipation matrix with budget margins and applies constraints, constructs a migration length matrix, and calculates a diffusion coefficient matrix to identify propagation hotspots; it performs mirroring offloading layered control and probabilistic landing point control for newly written data on the storage side, writes relevant data to snapshots, and updates each matrix in a closed loop. The system includes modules for evidence collection and tiered management, and also corresponds to computer-readable storage media. This invention accurately locates inefficient paths, reduces migration jitter, ensures stable prediction calculations, and achieves a comprehensive goal of controllable cost, predictable migration, and stable performance.

Description

Technical Field

[0001] This invention relates to the field of big data analytics, specifically a collaborative control method for cloud migration diffusion prediction and mirroring offloading. Background Technology

[0002] In engineering practice, cloud migration is often simplified to "moving the application deployment location to the cloud." However, what truly affects the results is often not the deployment location itself, but rather the cost structure and performance bottlenecks exposed by the combined effects of billing methods, resource specification constraints, and performance chains after migration. Under an elastic billing system, problems such as inefficient calls, resource misconfiguration, and mismatched specification selections, which might have been "masked" by fixed investments, are amplified in the form of continuous billing. Especially when business load fluctuates between peaks and valleys, the mismatch between resource supply and demand is more likely to manifest as long-term cost redundancy and periodic performance degradation.

[0003] In public cloud environments, computing and storage resources are typically provided in fixed and discrete tiers, and many commercial components are coupled with the number / tier of vCPUs in common models. When relational databases are deployed in the cloud as virtual machines, any increase in throughput or concurrency on the business side often triggers "scaling up across tiers," resulting in a tiered increase in costs. At the same time, redundant replicas, backups, cross-availability zone traffic, and storage performance reservations introduced to meet reliability and SLO requirements may further increase long-term operating expenses. On the other hand, issues such as redundant orchestration, repeated serialization / deserialization, excessive log collection, and retry strategies introduced at the system integration interface layer to pursue functional interoperability will bring additional CPU consumption, memory usage, and network round trips, which will translate into considerable ongoing costs under pay-as-you-go billing conditions, while also increasing the probability of triggering failures and congestion.

[0004] Furthermore, post-migration performance issues are not limited to the computing side. The behavior of the storage layer and data path under dynamic loads often determines the system's tail latency and stable throughput: when access patterns experience short-term bursts, hotspot shifts, or changes in read / write ratios, queue backlogs in the storage backend, IOPS / throughput limits, and multi-tenant interference effects are more easily amplified, leading to increased tail latency, exacerbated jitter, and even cascading timeouts. Traditional tiering and migration strategies often rely on empirical rules or static thresholds such as "hot data moving up, cold data moving down." However, in scenarios where cloud-side resource tiering, cross-tier data migration costs, and business fluctuations coexist, problems such as strategy lag, frequent oscillations, or local optima can easily arise, making it difficult to simultaneously achieve the comprehensive goals of cost control, predictable migration, and stable performance. Summary of the Invention

[0005] The purpose of this invention is to provide a method for collaborative control of cloud migration diffusion prediction and mirroring diversion, so as to solve the problems raised in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a collaborative control method for cloud migration diffusion prediction and mirroring, the collaborative control method comprising the following steps: Within a preset observation time window, end-to-end usage evidence is collected and solidified into an evidence snapshot. The end-to-end usage evidence includes: call chain sequence, service processing latency and call count, database statement statistics, end-to-end access latency of storage performance layer and capacity layer, and cost mapping information corresponding to cloud billing items. The evidence snapshots include: Link sequence evidence: the call sequence from the entry request to the service, downstream service, database statement, and storage layer; Service evidence: average processing latency, number of calls, error rate, retry rate, request size; Database evidence: statement signature, relative statement weight, CPU / IO ratio, number of connections; Storage evidence: end-to-end access latency at the performance layer, end-to-end access latency at the capacity layer, tail latency, read / write ratio, and queue metrics; Cost evidence: computing power, storage, log analysis, key services, database license / specification tiers, etc.

[0007] Evidence snapshots are used for subsequent matrix construction and review playback; Based on evidence snapshots, the system state is divided into multiple groups, including service groups, database groups, and storage layer groups, and each group is pre-configured. The preset configuration includes a budget limit or quota limit and service level target constraints; The service group is divided by service name / tenant / use case; the database group is divided by statement signature / tablespace / instance; the storage layer group is divided by performance layer / capacity layer, and further divided by image category / layer category. The group division follows two hard constraints: first, it can be stably statistically analyzed from evidence snapshots; second, it can correspond to specific control actions. Based on the call chain sequence, a basic transition matrix is ​​constructed. The basic transition matrix is ​​then weighted and row-normalized to obtain the inter-group transition matrix. The weight is defined as follows: ; Where W[S] is the weight, t i [S] represents the internal processing latency of service S, indicating the average processing time consumed by the service itself in a single request chain; this latency excludes external dependency waiting time. n i[s] represents the number of internal calls to service S within the observation time window; t0[S] represents the average waiting latency of service S for external dependencies; n0[s] represents the number of external dependency calls to service S within the observation window; γ is the reduction factor for external dependency wait items, used to reduce the blocking effect caused by external waits to a scale comparable to internal consumption. It can be set to 1 without changing the original structure; it can be greater than 1 when external dependencies are significantly billing-sensitive or external slowness is more sensitive to thread / connection pool usage; it can be between 0 and 1 when there is a greater emphasis on identifying manageable internal consumption.

[0008] The cost sensitivity factor is determined based on cost mapping information, and higher weights are assigned to transfers to database groups, database specification / license sensitive resource groups, or high-cost platform service groups, in order to improve the ability of the inter-group transfer matrix to characterize the propagation of cost risks. The cost sensitivity factor C[S] is expressed as: ; Where R represents the set of cost-sensitive resources, which may include computing power, storage capacity / bandwidth, network outbound traffic, logs / observations, key / security services, database specifications / licenses, etc.; Cost r [s] represents the r-th type of billing cost or cost increment attributed to service S within the observation window; Norm() is the normalization function; β r These are the weighting coefficients for resource categories.

[0009] Construct a budget dissipation diagonal matrix and apply dissipation constraints to the inter-group transition matrix to generate the dissipated inter-group transition matrix; The budget dissipation diagonal matrix is ​​a diagonal matrix: ; Where a i The diagonal element must be set with a lower bound to prevent uninterpretable propagation; diagonal element a i The calculation needs to combine budget margin, risk consumption forecast, and SLO penalty factor. When the group shows an SLO default trend or an abnormal increase in costs, the dissipation constraint is an additional penalty term. ,make The system needs to monitor the upper bound or equivalent convergence indicator of |P|. Once an increased convergence risk is detected, the system should prioritize reducing a. i The model was pulled back to the stable region.

[0010] The expression for the budget surplus is: ; BudgetLimit i CostAccum is the maximum allowable cost or quota for group i within the budget period. iAll accumulated consumption data can be obtained from billing attribution or internal billing statistics; The inter-group transition matrix and migration length matrix are updated smoothly using an exponentially weighted moving average to suppress the impact of short-term fluctuations on the stability of diffusion prediction.

[0011] Construct a migration length matrix, and calculate the diffusion coefficient moment based on the migration length matrix and the dissipated inter-group transition matrix; The migration length matrix is ​​used to characterize the average impact length caused by unit propagation in each group, and the impact length is one or more of the delay increment, congestion increment, and cost increment. Based on the diffusion coefficient matrix, the propagation hotspots and risk sources are determined, and mirror-based hierarchical control is performed on the storage side. Probabilistic write landing point control is performed on newly written data, and evidence snapshots are written. Furthermore, the inter-group transition matrix, migration length matrix, and budget dissipation diagonal matrix are updated. Write the intermediate values ​​of the diversion ratio, mirror category coverage, write landing point statistics and diffusion prediction calculation into the evidence snapshot, and update the inter-group transition matrix, migration length matrix and budget dissipation diagonal matrix accordingly. Closed-loop updates need to set limits on the magnitude of actions, such as the maximum change in the diversion ratio within a single cycle, and a cooldown time for adjusting the mirror size to avoid closed-loop oscillations. After the traffic splitting action changes the actual access path, the link count will be updated accordingly. When constructing the inter-group transfer matrix in the next cycle, the behavior after the traffic splitting should be reflected naturally to achieve mutual correction between the model and the execution.

[0012] Furthermore, the basic transition matrix is ​​constructed as follows: Extract the group sequence of each call link sequence within the observation time window. Count the number of group sequence pairs at adjacent positions in the call link sequence to obtain the inter-group adjacent transition count. Divide the adjacent group transition count from group i to group j into the numerator and the sum of the transition counts from group i to all groups plus the smoothing term ε into the denominator. The result is the element value at that position. The expression for the element value at that position is: ; Among them The fundamental transition matrix, N i→j and N i→u The transition count is the number of transitions between adjacent groups, and ε is the smoothing term; The call intensity factor includes one or more of the following: call count, request size, retry rate, and error rate; the latency factor and congestion factor include one or more of the following: average latency, tail latency, and queue metrics. The inter-group transition matrix is ​​obtained by combining the call intensity factor, delay factor, congestion factor, and cost sensitivity factor to perform weight correction and row normalization on the basic transition matrix.

[0013] Furthermore, the diagonal elements of the budget dissipation diagonal matrix are determined as follows: With the budget limit B of group i i Let C be the numerator, representing the risk consumption prediction quantity for group i. i Add the stable term to the denominator and perform division. Take the minimum value of the result and 1 to obtain the intermediate value. This intermediate value is compared with the penalty factor p triggered by the service level target risk for group i. i When performing multiplication, the final result is the diagonal element 'a'. i , and a i The value range is (0,1]; The calculation formula is: ; Among them, a i For the diagonal elements of the budget dissipation diagonal matrix, B i This represents the budget or quota cap for group i. The predicted risk consumption of group i is obtained from the evidence snapshot and diffusion prediction. As a stable term, p i This is a penalty factor triggered by service level target risks.

[0014] After introducing budget dissipation, the corresponding summation S A for: ; Since the inter-group transition matrix is ​​a row-normalized matrix, its infinite norm satisfies The budget dissipation diagonal matrix is ​​a diagonal matrix and ,therefore .thereby: ; Therefore S A It is a Neumann series and converges, and can be equivalently represented by a matrix inversion: ; This demonstrates that even with the introduction of the budget dissipation matrix, diffusion prediction can still be calculated with arbitrary precision, and that a remains accurate even when the budget becomes tight or the risk increases. i As the convergence range decreases, the system will be actively pulled back to a stronger convergence region, improving numerical stability and engineering controllability.

[0015] Furthermore, the migration length matrix is ​​constructed as follows: The row index of the migration length matrix corresponds to the source group, the column index corresponds to the target group, and the matrix elements are the normalized increments of the key indicators of the target group when 1 unit load is injected into the source group within the observation time window.

[0016] Furthermore, the diffusion coefficient matrix is ​​constructed as follows: The total migration length matrix is ​​obtained by multiplying the migration length matrix by the sum of powers of the dissipated inter-group transition matrix from order 0 to the truncation order N. The dissipated inter-group transition matrix is ​​the product of the budgeted dissipation diagonal matrix and the inter-group transition matrix. Dividing the total migration length matrix by 3 gives the diffusion coefficient matrix. The formula for calculating the diffusion coefficient matrix D is as follows: ; Among them Total migration length matrix, is the migration length matrix, P′ is the dissipated inter-group transition matrix, N is the truncation order, and D is the diffusion coefficient matrix.

[0017] The truncation order is also constrained by the maximum order; when the maximum order is reached, the accumulation stops and the current diffusion coefficient matrix is ​​output.

[0018] Furthermore, mirror-based hierarchical control includes: The target data object is divided into mirror categories and hierarchical categories, and a copy of the mirror category data is kept between the performance layer storage device and the capacity layer storage device. Set the traffic splitting ratio β so that mirror category read and write requests access the capacity layer replica with probability β and access the performance layer replica with probability 1-β. The maximum value of the shunting ratio needs to be adaptively tightened based on the risk of capacity layer tail delay or end-to-end tail delay, in order to avoid overall performance degradation due to capacity layer tail delay deterioration. The traffic splitting ratio is adjusted based on the end-to-end access latency of the performance layer and the end-to-end access latency of the capacity layer, so that the absolute value of the difference between the end-to-end access latency of the performance layer and the end-to-end access latency of the capacity layer falls within a preset tolerance range, and the adjustment of β and the mirror migration or copying operation are stopped when the tolerance range is met. The division between mirror categories and hierarchical categories is determined jointly based on the propagation heat zone output by heat statistics and diffusion coefficient matrix; Identifying propagation hotspots requires combining the propagation intensity of the diffusion coefficient matrix with cost-sensitive factors, prioritizing high-cost / high-risk clusters such as database clusters and storage performance layer clusters; The division between mirror categories and hierarchical categories needs to be combined with the propagation hot zones output by the storage side heat statistics and diffusion coefficient matrix. Priority should be given to creating mirror copies of data objects corresponding to the propagation hot zones. Mirror categories should only cover a small number of hot objects to ensure storage efficiency.

[0019] Furthermore, the execution process of the probabilistic write landing point control: Newly written data is written to the capacity layer with probability β and to the performance layer with probability 1-β; and a single-copy writing method is used for mirror category data, with metadata validity markers indicating the layer where the valid copy is located. The metadata validity tag includes a validity bit and a location bit, which are used to indicate the storage layer where the valid copy of the mirror category data is located, and to route read and write requests accordingly.

[0020] Furthermore, the written evidence snapshot includes: The cutoff order, error threshold stopping reason, change in D between two consecutive tests, and change step size of the shunting ratio are used for verification.

[0021] Furthermore, a cloud migration diffusion prediction and mirroring offloading collaborative control system includes: The module includes: evidence collection module, group management module, matrix construction module, dissipation constraint module, diffusion prediction module, mirroring and splitting execution module, and closed-loop update module. Each module is configured to execute a cloud migration diffusion prediction and mirroring offloading collaborative control method.

[0022] Furthermore, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements a cloud migration diffusion prediction and mirroring diversion collaborative control method.

[0023] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention uses evidence snapshots and service resource weights to pinpoint inefficient calls and misconfigured paths to specific services / statements / links, and links them to budgets / quotas. Migration studies indicate that end-to-end usage evidence can effectively track overuse and inefficiency, thereby supporting budget governance.

[0024] 2. This invention adjusts the traffic offloading ratio by adjusting the latency alignment. When the performance layer slows down due to high load, the request is offloaded to the capacity layer and the operation stops after alignment, thereby reducing write amplification and jitter caused by pure migration. The idea of ​​image optimization layering has shown that a small number of hot spots combined with traffic offloading can help improve throughput and reduce migration overhead under high load.

[0025] 3. This invention uses power-order explicit summation for diffusion prediction, which can converge exponentially to arbitrary precision. More importantly, this invention uses the budget dissipation matrix to pull the propagation matrix back to the convergence domain, avoiding the invalidation of diffusion meaning when the norm of the inter-group transition matrix does not meet the conditions, thus ensuring the stability and usability of the prediction calculation in engineering. Attached Figure Description

[0026] Figure 1 This is a general block diagram of the collaborative control method for cloud migration diffusion prediction and mirroring diversion according to the present invention. Figure 2 This is a schematic diagram of multi-cluster modeling for a cloud migration diffusion prediction and mirroring diversion collaborative control method according to the present invention. Figure 3 This is a schematic diagram illustrating the calculation of the diffusion coefficient matrix in a cloud migration diffusion prediction and mirroring diversion collaborative control method according to the present invention. Figure 4 This is a flowchart of the mirror-based traffic splitting control method for cloud migration diffusion prediction and mirror-based traffic splitting collaborative control according to the present invention. Figure 5 This is a schematic diagram illustrating the probabilistic new write landing point and mirror category promotion of the cloud migration diffusion prediction and mirror diversion collaborative control method of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Example: This invention provides a technical solution, a collaborative control method for cloud migration diffusion prediction and mirroring offloading, the collaborative control method including the following steps: Within a preset observation time window, end-to-end usage evidence is collected and solidified into an evidence snapshot. The end-to-end usage evidence includes: call chain sequence, service processing latency and call count, database statement statistics, end-to-end access latency of storage performance layer and capacity layer, and cost mapping information corresponding to cloud billing items. For example: In one embodiment, the present invention is deployed in a production environment or gray-scale environment after cloud migration, and consists of two parts: a control plane and a data plane.

[0029] The control plane is used for evidence collection, model calculation, policy generation, and closed-loop updates; the data plane is used for performing mirroring and routing, writing endpoints, storing metadata maintenance, and providing read and write access services to the outside world.

[0030] The control plane can run on a set of containerized services, including: evidence collection service, evidence snapshot repository, diffusion prediction service, cost-sensitive partitioning service, image distribution controller, policy delivery service, and audit playback service. The data plane can be deployed on the same machine as the storage system, or connected to block storage, object storage, or distributed file system via a proxy / bypass mechanism.

[0031] To ensure verifiability, the control plane freezes evidence snapshots at the end of each time window and generates summary fingerprints for the input and output calculation results. In the event of subsequent disputes, the evidence snapshot can be used to replay the matrix construction, series truncation, and control actions, thereby avoiding the problem of "uninterpretable strategy".

[0032] In one embodiment, the system generates evidence snapshots with a fixed observation time window. The duration can be an engineering-feasible value such as 30 seconds, 60 seconds, or 5 minutes; if the load fluctuates drastically, the duration can be adaptively shortened to reduce statistical lag.

[0033] The evidence snapshot includes the following fields (which can be added or removed as needed): TraceSeq, a collection of link sequences, records the entry request identifier, service call sequence, downstream dependency type (service / database / external dependency / storage layer), timestamp, and time consumption for each link sequence.

[0034] Service statistics table SvcStat[S]: Records the number of times service S is called within the window, average processing latency, P95 / P99 latency, error rate, retry rate, and request volume.

[0035] External dependency statistics ExtStat[S]: Records the number of external calls to service S, the average external latency, and the failure rate (used to identify "apparent load caused by external slowness").

[0036] Database statement statistics DBStat[q]: Records metrics such as the number of times statement signature q is called, average time consumption, CPU / IO contribution, lock wait, etc., and maintains the attribution mapping from statement to service path.

[0037] Storage layer statistics StoStat: Records end-to-end access latency, queue length or equivalent congestion metrics, tail latency, and read / write ratio at the performance / capacity layer.

[0038] CostMap: Maps cloud billing items to service groups / database groups / storage groups, forming the input of "cost-sensitive factors"; mapping can be completed by resource tags, instance identifiers, or billing item rules.

[0039] Controller State (CtrlState): Records the traffic splitting ratio, maximum traffic splitting limit, mirror category size, optimizer smoothing parameters, etc. of the previous window.

[0040] Evidence collection can employ sampling strategies to reduce overhead; the sampling ratio is written to CtrlState, and normalization correction is performed at the same ratio during subsequent playback to avoid "sampling-induced caliber drift".

[0041] Based on evidence snapshots, the system state is divided into multiple groups, including service groups, database groups, and storage layer groups, and each group is pre-configured. For example: In one embodiment, the system divides the state space into m groups and assigns a unique number to each group. The group division follows the principle of "scalability and controllability": Service groups: Divided by service name, tenant, or use case, suitable for carrying out actions such as frequency limiting, quotas, and caching suggestions; Database clusters: Divided by statement signature, tablespace, or instance, suitable for handling operations such as index / parameter correction, connection pool constraints, and avoiding scaling. Storage tiers are divided into performance / capacity tiers and image / layer categories, suitable for handling traffic splitting ratios and image scale operations.

[0042] After group partitioning is completed, the system generates a group dictionary, GroupDict, which includes: group type, group member set, cost-sensitive tags, budget cap, SLO objectives, and a list of actionable actions. This dictionary is persisted along with the evidence snapshot to prevent difficulties in verification caused by changes in group definitions during operation.

[0043] Based on the call chain sequence, a basic transition matrix is ​​constructed. The basic transition matrix is ​​then weighted and row-normalized to obtain the inter-group transition matrix. The basic transition matrix is ​​constructed as follows: Extract the group sequence of each call link sequence within the observation time window. Count the number of group sequence pairs at adjacent positions in the call link sequence to obtain the inter-group adjacent transition count. Divide the adjacent group transition count from group i to group j into the numerator and the sum of the transition counts from group i to all groups plus the smoothing term ε into the denominator. The result is the element value at that position. The expression for the element value at that position is: ; Among them The fundamental transition matrix, N i→j and N i→u The transition count is the number of transitions between adjacent groups, and ε is the smoothing term; The call intensity factor includes one or more of the following: call count, request size, retry rate, and error rate; the latency factor and congestion factor include one or more of the following: average latency, tail latency, and queue metrics. The inter-group transition matrix is ​​obtained by combining the call intensity factor, delay factor, congestion factor, and cost sensitivity factor to perform weight correction and row normalization on the basic transition matrix; For example: In one embodiment, a service resource weight (W[S]) is calculated for each service (S) to identify “high-consumption and manageable” paths. This can take the following form: ; Where t i [S] and n i [s] comes from SvcStat[S], and t0[S] and n0[s] come from ExtStat[S]. If the business side is more concerned about tail latency, t can also be used. i Replace [S] with the P95 or P99 indicator to make the weighting more closely reflect risk.

[0044] Cost sensitivity factors are generated based on CostMap. A typical approach is to assign weight coefficients to labels such as "database specification / license sensitivity", "expensive platform service sensitivity", and "storage bandwidth sensitivity", and then degrade these weight coefficients to the links between groups as part of the subsequent propagation matrix correction terms.

[0045] In one embodiment, neighbor group transition counts are extracted from TraceSeq. For example, the link segment "ingress → service A → service B → database statement q → performance layer access" will generate transition counts. , , ; The basic transition matrix is ​​obtained by row normalizing the counts: ; To smooth out terms, avoid division by zero, and reduce spikes in sparse counting; The basic matrix only reflects the "probability of propagation". To allow the propagation matrix to simultaneously express the "propagation strength and cost", the system constructs evidence weights ω. i→j And obtain the correction matrix: ; ω i→j It can be composed of the following parts: Invocation intensity: Related to the number of calls and retry rate; Delay factor and congestion term: related to downstream P95 / P99 and queue indicators; Cost-sensitive items: Increase weight when downstream groups are database-sensitive or expensive service-sensitive; Traffic splitting feedback item: When mirroring causes changes in access path probability, use CtrlState to perform consistency correction on the relevant link weights to prevent statistical lag.

[0046] To suppress short-window jitter, the system uses an exponentially weighted moving average smoothing for P: ; in Commonly used engineering values ​​such as 0.1–0.3 can be used. In the matrix implementation, sparse storage (only storing non-zero items) is preferred to reduce computational and storage overhead.

[0047] Construct a budget dissipation diagonal matrix and apply dissipation constraints to the inter-group transition matrix to generate the dissipated inter-group transition matrix; The diagonal elements of the budget dissipation diagonal matrix are determined as follows: With the budget limit B of group i i Let C be the numerator, representing the risk consumption prediction quantity for group i. i Add the stable term to the denominator and perform division. Take the minimum value of the result and 1 to obtain the intermediate value. This intermediate value is compared with the penalty factor p triggered by the service level target risk for group i. i When performing multiplication, the final result is the diagonal element 'a'. i , and a i The value range is (0,1]; The calculation formula is: ; Among them, a i For the diagonal elements of the budget dissipation diagonal matrix, B i This represents the budget or quota cap for group i. The predicted risk consumption of group i is obtained from the evidence snapshot and diffusion prediction. As a stable term, p i A penalty factor triggered by service level target risks; For example: The diffusion order must be usable, and propagation must not diverge. Therefore, in one embodiment, the system constructs a budgeted dissipation diagonal matrix: ; Where a i Reflection group g i The "propagation attenuation intensity" is determined by the budget margin, SLO risk, and cost ceiling. A directly achievable calculation method is provided: Let the group budget cap be B. i Group risk prediction costs ,but ; Additional penalties will be imposed when a group exhibits a trend of SLO default or an abnormal increase in costs. ,make This achieves the goal of "the higher the risk, the greater the attenuation of transmission".

[0048] Then dissipation is applied to the propagation matrix: ; In implementation, the system monitors the upper bound or equivalent convergence indicator of |P|; once an increased convergence risk is detected, it prioritizes tightening the budget (reducing a). i Pull the model back to the stable range, and then output actions such as frequency limiting / downsampling / buffering to avoid the model being unable to calculate.

[0049] Construct a migration length matrix, and calculate the diffusion coefficient matrix based on the migration length matrix and the dissipated inter-group transition matrix; The migration length matrix is ​​used to characterize the average impact length caused by unit propagation in each group, and the impact length is one or more of the delay increment, congestion increment, and cost increment. The migration length matrix is ​​constructed as follows: The row index of the migration length matrix corresponds to the source group, the column index corresponds to the target group, and the matrix elements are the normalized increments of the key indicators of the target group when 1 unit load is injected into the source group within the observation time window. For example: In one embodiment, the migration length matrix is ​​used to characterize the "average length of the effect per unit propagation on the target swarm". To enhance implementability, this invention provides three optional definitions, which can be selected according to swarm type during deployment: Delayed Length : Take the delay increment of P95 / P99 in this group or the end-to-end time contribution of a unit request; Congestion Length : Retrieve queue length increment, waiting time increment, or equivalent congestion metric; Cost-type length : Take the sensitivity of the unit request cost increment or "triggering expansion risk" (e.g., weighted by the specification level transition risk coefficient).

[0050] When the same group is simultaneously affected by both performance and cost, the above lengths can be weighted and combined: ; Weight The definition can be determined through linear regression or empirical settings during the grayscale stage, and then solidified into GroupDict to ensure consistency.

[0051] The migration length matrix is ​​also updated smoothly to avoid misjudgment caused by single-window anomalies. The diffusion coefficient matrix is ​​constructed as follows: The total migration length matrix is ​​obtained by multiplying the migration length matrix by the sum of powers of the dissipated inter-group transition matrix from order 0 to the truncation order N. The dissipated inter-group transition matrix is ​​the product of the budgeted dissipation diagonal matrix and the inter-group transition matrix. Dividing the total migration length matrix by 3 gives the diffusion coefficient matrix. The formula for calculating the diffusion coefficient matrix D is as follows: ; Among them Total migration length matrix, Let P' be the migration length matrix, P' be the inter-group transition matrix after dissipation, N be the truncation order, and D be the diffusion coefficient matrix. For example: In one embodiment, the system calculates the total migration length matrix and the diffusion coefficient matrix according to the following relationship: ; The engineering implementation uses a truncation order N, and stops are determined by an error threshold: Initialize S=I, T=I; Loop through n=1 to N: Let , and then ; If |T| is less than the threshold or the change in D between two consecutive intervals is less than the threshold, then stop early; get And thus obtain .

[0052] The system will write the N value of each truncation, the reason for stopping (reaching the threshold or the upper limit), and the approximate value of the matrix norm into the evidence snapshot warehouse to form a verifiable record.

[0053] Based on the diffusion coefficient matrix, the propagation hotspots and risk sources are determined, and mirror-based hierarchical control is performed on the storage side. Probabilistic write landing point control is performed on newly written data, and evidence snapshots are written. Furthermore, the inter-group transition matrix, migration length matrix, and budget dissipation diagonal matrix are updated. Mirror-based hierarchical control includes: The target data object is divided into mirror categories and hierarchical categories, and a copy of the mirror category data is kept between the performance layer storage device and the capacity layer storage device. Set the traffic splitting ratio β so that mirror category read and write requests access the capacity layer replica with probability β and access the performance layer replica with probability 1-β. The traffic splitting ratio is adjusted based on the end-to-end access latency of the performance layer and the end-to-end access latency of the capacity layer, so that the absolute value of the difference between the end-to-end access latency of the performance layer and the end-to-end access latency of the capacity layer falls within a preset tolerance range, and the adjustment of β and the mirror migration or copying operation are stopped when the tolerance range is met. The division between mirror categories and hierarchical categories is determined jointly based on the propagation heat zone output by heat statistics and diffusion coefficient matrix; The execution process of the probabilistic write landing point control: Newly written data is written to the capacity layer with probability β and to the performance layer with probability 1-β; and a single-copy writing method is used for mirror category data, with metadata validity markers indicating the layer where the valid copy is located. For example: In one embodiment, data objects are divided into mirror categories and hierarchical categories: Mirror categories: Only cover a small number of frequently accessed objects, and retain copies across the performance and capacity layers; Hierarchical categories: Maintain a single copy and place them according to the cold / hot migration strategy.

[0054] The size of a mirror category can be determined by two types of signals: one is from storage-side heat statistics, and the other is from the propagation hotspots predicted by diffusion (for example, when the propagation coefficient of certain database groups to the storage layer is significantly higher, the mirror category will prioritize covering the data set corresponding to that path), thus converting "prediction" into "specific covered objects".

[0055] In one embodiment, a traffic splitting ratio β is defined: mirror category requests are routed to the capacity layer replica with probability β, and to the performance layer replica with probability 1-β. The control objective is not simply to offload traffic, but to allow the end-to-end access latency of both layers to enter a tolerance range. ; Where is the tolerance threshold.

[0056] The adjustment logic can be implemented according to the following rules: when If the performance layer is slower, the system improves β; if β reaches its limit and still cannot be aligned, the mirror category is expanded or the mirror hotspot set is updated. when The capacity layer is slower, the system reduces β, and limits further migration / replication to the capacity layer; When the system enters the tolerance range, it stops migrating and parameter tuning to avoid repeated jumps.

[0057] The control cycle can be set from 200ms to 1s, and L... P ,L c Perform exponentially weighted smoothing; write the period and smoothing coefficient into CtrlState to avoid caliber drift.

[0058] In one embodiment, new writes do not always land on the performance layer, but rather on a probability-based decision: the capacity layer is chosen as the primary write location with probability β, and the performance layer is chosen with probability 1-β. Once the primary write location is determined, only one replica is updated, avoiding write amplification caused by dual writes; replica consistency is guaranteed by metadata validity marking.

[0059] When the capacity layer tail delay deteriorates significantly, the system tightens β. maxIt also imposes an upper limit constraint on the write probability to prevent the mirror category from being dragged down by the capacity layer in extreme scenarios.

[0060] The metadata validity tag includes a validity bit and a location bit, which are used to indicate the storage layer where the valid copy of the mirror category data is located, and to route read and write requests accordingly. For example: In one embodiment, storage is managed in fixed segments (e.g., 2MB). Each segment maintains segment-level metadata, with typical fields including: segment identifier, performance tier address, capacity tier address, hotness count, read / write count, validity bitmap pointer, rewrite distance count, etc.

[0061] To reduce metadata overhead, bitmaps can maintain "valid bit / position bit" at the subpage (e.g., 4KB) granularity. Read and write routes select valid copies based on the bitmap, thereby maintaining data consistency without double-writing.

[0062] The optimizer runs in a single thread with a fixed cycle; within the cycle, it reads the block layer counter to estimate L. P ,L c And update β, mirror size and hotspot set.

[0063] The written evidence snapshot includes: The cutoff order, error threshold stopping reason, change in D between two consecutive tests, and change step size of the shunting ratio are used for verification. For example: The diffusion prediction output D can be understood as a "propagation heatmap". In one embodiment, the controller extracts several pairs (i, j) with the highest propagation intensity from D and generates actions based on cost-sensitive factors: When a hotspot group points to a database group and is highly cost-sensitive, the controller prioritizes outputting "pressure reduction actions," such as: caching suggestions (for static read paths), interface budget limits / frequency limits (for inefficient high-frequency calls), and misconfiguration corrections (for abnormal statements and abnormal links), to suppress passive specification expansion; When a hotspot group points to a storage performance layer group, the controller increases the priority of image category coverage and adjusts the beta to enter the offload interval. When the risk of model convergence increases, the controller first tightens the budget dissipation, and then adds frequency limiting or downsampling to ensure the stability of the series calculation.

[0064] Closed-loop updates occur after each evidence snapshot period ends: The traffic splitting action changes the actual access path, and the link count is updated accordingly. The behavior after the traffic splitting is naturally reflected when constructing the inter-group transfer matrix in the next cycle. The diffusion prediction output inverse constraints and mirror scale enable the "model-execution" to correct each other, rather than each speaking its own language.

[0065] To avoid closed-loop oscillation, the controller sets a "step size limit" for the action amplitude, such as limiting the maximum change of β within a single cycle, and sets a cooling time for the adjustment of the mirror size to ensure that the system is smoother near the load boundary.

[0066] In a simplified system, we divide the data into 6 groups: Entry / Gateway Group; Core Service Group; Non-Core Service Group; Database Group; Performance Layer Group; Capacity Layer Group.

[0067] After collecting link sequences within a preset observation time window, the system obtains transfer counts and forms a basic transfer matrix. Subsequently, higher cost-sensitive weights are introduced for links pointing to the database group, and congestion weights are introduced for links pointing to the performance layer group, resulting in a corrected inter-group transfer matrix. Simultaneously, if the database group's budget margin is insufficient, it is lowered in the budget dissipation diagonal matrix, causing the propagation links through the database group to decay faster, and the truncation of the series to reach the error threshold earlier. The diffusion coefficient matrix obtained from the diffusion prediction shows that the propagation intensity from the core service group to the database group and from the database group to the performance layer group is too high. The controller outputs buffering and frequency limiting suggestions and increases the storage-side β to relieve pressure on the performance layer. As β is adjusted, the access path distribution changes, entering the link count of the next cycle. After the inter-group transfer matrix is ​​updated, the propagation hotspot decreases, and the closed loop converges.

[0068] This embodiment embodies three key features of the present invention: the propagation matrix is ​​derived from evidence, convergence is guaranteed by budget dissipation, and execution is achieved by mirroring and routing to feed back into the model.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for collaborative control of cloud migration diffusion prediction and mirroring offloading, characterized in that: The collaborative control method includes the following steps: Within a preset observation time window, end-to-end usage evidence is collected and solidified into an evidence snapshot. The end-to-end usage evidence includes: call chain sequence, service processing latency and call count, database statement statistics, end-to-end access latency of storage performance layer and capacity layer, and cost mapping information corresponding to cloud billing items. Based on evidence snapshots, the system state is divided into multiple groups, including service groups, database groups, and storage layer groups, and each group is pre-configured. Based on the relationship between the call chain sequence and the group, a basic transition matrix is ​​constructed. The basic transition matrix is ​​then weighted and row-normalized to obtain the inter-group transition matrix. Construct a budget dissipation diagonal matrix and apply dissipation constraints to the inter-group transition matrix to generate the dissipated inter-group transition matrix; Construct a migration length matrix, and calculate the diffusion coefficient matrix based on the migration length matrix and the dissipated inter-group transition matrix; Based on the diffusion coefficient matrix, the propagation hotspots and risk sources are determined, and mirror-based hierarchical control is performed on the storage side. Probabilistic write landing point control is performed on newly written data, and evidence snapshots are written. The inter-group transition matrix, migration length matrix, and budget dissipation diagonal matrix are updated.

2. The cloud migration diffusion prediction and mirroring offloading collaborative control method according to claim 1, characterized in that: The basic transition matrix is ​​constructed as follows: Extract the group sequence of each call link sequence within the observation time window. Count the number of group sequence pairs at adjacent positions in the call link sequence to obtain the inter-group adjacent transition count. Divide the adjacent group transition count from group i to group j into the numerator and the sum of the transition counts from group i to all groups plus the smoothing term ε into the denominator. The result is the element value at that position. The expression for the element value at that position is: ； Among them The fundamental transition matrix, N i→j and N i→u The transition count is the number of transitions between adjacent groups, and ε is the smoothing term; The call intensity factor includes one or more of the following: call count, request size, retry rate, and error rate; the latency factor and congestion factor include one or more of the following: average latency, tail latency, and queue metrics. The inter-group transition matrix is ​​obtained by combining the call intensity factor, delay factor, congestion factor, and cost sensitivity factor to perform weight correction and row normalization on the basic transition matrix.

3. The cloud migration diffusion prediction and mirroring offloading collaborative control method according to claim 1, characterized in that: The diagonal elements of the budget dissipation diagonal matrix are determined as follows: With the budget limit B of group i i Let C be the numerator, representing the risk consumption prediction quantity for group i. i Add the stable term to the denominator and perform division. Take the minimum value of the result and 1 to obtain the intermediate value. This intermediate value is compared with the penalty factor p triggered by the service level target risk for group i. i When performing multiplication, the final result is the diagonal element 'a'. i , and a i The value range is (0,1]; The calculation formula is: ； Among them, a i For the diagonal elements of the budget dissipation diagonal matrix, B i This represents the budget or quota cap for group i. The predicted risk consumption of group i is obtained from the evidence snapshot and diffusion prediction. As a stable term, p i This is a penalty factor triggered by service level target risks.

4. The cloud migration diffusion prediction and mirroring offloading collaborative control method according to claim 1, characterized in that: The migration length matrix is ​​constructed as follows: The row index of the migration length matrix corresponds to the source group, the column index corresponds to the target group, and the matrix elements are the normalized increments of the key indicators of the target group when 1 unit load is injected into the source group within the observation time window.

5. The cloud migration diffusion prediction and mirroring offloading collaborative control method according to claim 1, characterized in that: The diffusion coefficient matrix is ​​constructed as follows: The total migration length matrix is ​​obtained by multiplying the migration length matrix by the sum of powers of the dissipated inter-group transition matrix from order 0 to the truncation order N. The dissipated inter-group transition matrix is ​​the product of the budgeted dissipation diagonal matrix and the inter-group transition matrix. Dividing the total migration length matrix by 3 gives the diffusion coefficient matrix. The formula for calculating the diffusion coefficient matrix D is as follows: ； Among them Total migration length matrix, is the migration length matrix, P′ is the dissipated inter-group transition matrix, N is the truncation order, and D is the diffusion coefficient matrix.

6. The cloud migration diffusion prediction and mirroring offloading collaborative control method according to claim 1, characterized in that: Mirror-based hierarchical control includes: The target data object is divided into mirror categories and hierarchical categories, and a copy of the mirror category data is kept between the performance layer storage device and the capacity layer storage device. Set the traffic splitting ratio β so that mirror category read and write requests access the capacity layer replica with probability β and access the performance layer replica with probability 1-β. The traffic splitting ratio is adjusted based on the end-to-end access latency of the performance layer and the end-to-end access latency of the capacity layer, so that the absolute value of the difference between the end-to-end access latency of the performance layer and the end-to-end access latency of the capacity layer falls within a preset tolerance range, and the adjustment of β and the mirror migration or copying operation are stopped when the tolerance range is met. The division between mirror categories and hierarchical categories is determined jointly based on the propagation heat zones output by heat statistics and diffusion coefficient matrix.

7. The cloud migration diffusion prediction and mirroring offloading collaborative control method according to claim 1, characterized in that: The execution process of the probabilistic write landing point control: Newly written data is written to the capacity layer with probability β and to the performance layer with probability 1-β; and a single-copy writing method is used for mirror category data, with metadata validity markers indicating the layer where the valid copy is located. The metadata validity tag includes a validity bit and a location bit, which are used to indicate the storage layer where the valid copy of the mirror category data is located, and to route read and write requests accordingly.

8. The cloud migration diffusion prediction and mirroring offloading collaborative control method according to claim 1, characterized in that: The written evidence snapshot includes: The cutoff order, error threshold stopping reason, change in D between two consecutive tests, and change step size of the shunting ratio are used for verification.

9. A cloud migration diffusion prediction and mirroring diversion collaborative control system, characterized in that, include: The module includes: evidence collection module, group management module, matrix construction module, dissipation constraint module, diffusion prediction module, mirroring and splitting execution module, and closed-loop update module. Each module is configured to perform the method described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 8.

Images