Data migration method and apparatus

Abstract

The application provides a data migration method and device; the method comprises the following steps: determining a risk score of a target computing task executed on a current execution node facing an execution interruption, and a pre-warning interruption duration of the current execution node; determining a migration duration required for migrating state data of the target computing task from the current execution node to a target node; in the case of determining that the target computing task needs to be migrated according to the risk score, the pre-warning interruption duration and the migration duration, migrating the state data from the current execution node to the target node.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a data migration method and apparatus. Background Technology

[0002] In a hybrid cloud environment, utilizing preemptible computing instances provided by the public cloud to perform high-performance computing (HPC) tasks can effectively expand local computing resources. However, these instances may be reclaimed by the cloud service provider at any time. Although the cloud service provider provides brief outage warnings, fluctuations in cross-cloud network bandwidth often prevent data from being migrated within the outage warning window, resulting in the loss of computing results and waste of computing power. Summary of the Invention

[0003] This application provides a data migration method and apparatus.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a data migration method, the method comprising: Determine the risk score of the target computation task being executed on the current execution node facing execution interruption, and the warning interruption duration of the current execution node; Determine the migration time required to migrate the status data of the target computation task from the current execution node to the target node; If, based on the risk score, the warning interruption duration, and the migration duration, it is determined that the target computing task needs to undergo state data migration, the state data is migrated from the current execution node to the target node.

[0005] In some embodiments, determining the risk score of execution interruption faced by the target computation task being executed on the current execution node includes: The first probability that the current execution node will be reclaimed, the second probability that the status data of the target computing task will fail to be transmitted from the current execution node to the target node within the warning interruption period, and the extent of loss of computing resources already occupied by the target computing task due to execution interruption are determined. The risk score is obtained by weighted summation of the first probability, the second probability, and the degree of loss.

[0006] In some embodiments, determining a first probability that the current execution node will be reclaimed includes: Obtain the price fluctuation information and historical interruption information of the currently executing node; Based on the price fluctuation information and the historical interruption information, a first probability is determined that the current execution node will be recycled.

[0007] In some embodiments, the method further includes: After migrating the state data from the current execution node to the target node, the weights of the first probability, the second probability, and the degree of loss in the weighted summation are adjusted based on whether the current execution node is reclaimed.

[0008] In some embodiments, determining the migration time required to migrate the state data of the target computing task from the current execution node to the target node includes: Determine the amount of the status data, and the network bandwidth information between the current execution node and the target node; Based on the data volume and the network bandwidth information, the migration time required to migrate the status data of the target computing task from the current execution node to the target node is determined.

[0009] In some embodiments, when it is determined, based on the risk score, the warning interruption duration, and the migration duration, that the target computing task requires state data migration, migrating the state data from the current execution node to the target node includes: Based on the risk score and the migration duration, a safety threshold is determined; If it is determined that the warning interruption duration is less than the safety threshold, and it is determined that the target computing task needs to perform state data migration, then the state data is migrated from the current execution node to the target node.

[0010] In some embodiments, the method further includes: If the warning interruption duration is determined to be greater than or equal to the safety threshold, and the deviation of the warning interruption duration from the safety threshold is less than a specific threshold, the time interval for storing the status data of the target computing task locally on the current execution node is shortened based on a specific time interval.

[0011] In some embodiments, the method further includes: If it is determined that the warning interruption duration is greater than the safety threshold, and the deviation of the warning interruption duration from the safety threshold is greater than or equal to the specific threshold, the status data of the target computing task is saved locally on the current execution node at the specific time interval.

[0012] In some embodiments, migrating the state data from the current execution node to the target node includes: The status data is divided into multiple data blocks; The multiple data blocks are transmitted to the target node in parallel using multiple parallel transmission threads.

[0013] This application provides a data migration apparatus, the apparatus comprising: The first determining module is used to determine the risk score of the target computing task being executed on the current execution node facing execution interruption, and the warning interruption duration of the current execution node; The second determining module is used to determine the migration time required to migrate the state data of the target computing task from the current execution node to the target node; The data migration module is used to migrate the state data from the current execution node to the target node when it is determined that the target computing task needs to perform state data migration based on the risk score, the warning interruption duration, and the migration duration. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0015] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0016] Figure 1 This is one of the flowcharts illustrating a data migration method provided in an embodiment of this application; Figure 2 This is a second schematic flowchart of a data migration method provided in an embodiment of this application; Figure 3 This is the third flowchart illustrating a data migration method provided in this application embodiment; Figure 4 This is a fourth flowchart illustrating a data migration method provided in an embodiment of this application; Figure 5 This is a schematic diagram of a risk perception and migration system architecture provided in an embodiment of this application; Figure 6 This is a schematic diagram of a risk perception and migration decision-making process provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a data migration device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit the scope of this application.

[0019] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0020] The terms "first," "second," and "third" appearing in the embodiments of this application do not have a specific meaning (e.g., there is no order, nor does it indicate a special limitation on the number of devices in the embodiments of this application), but are merely for the purpose of clearly describing the embodiments of this application and do not constitute any limitation on the embodiments of this application. The term "multiple" appearing in the embodiments of this application refers to two or more integers.

[0021] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies or terms of the embodiments of this application are described below. The following related technologies or related terms are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0022] Public cloud spot instances are subject to repossession at any time. Although cloud service providers offer brief outage warning windows (e.g., Amazon Web Services Elastic Compute Cloud (AWS EC2) offers 120-second warnings, and Microsoft Azure and Google Cloud Platform (GCP) offer 30-second warnings), in cross-cloud scenarios, data needs to be transmitted back to private cloud storage via a wide area network (WAN). Real-time fluctuations in network bandwidth often prevent data from being transmitted within the warning window, resulting in the loss of computing results and waste of computing power.

[0023] The inventors of this application discovered during their research and analysis that the related technology has at least the following problems: (1) Passive response: Ignoring cross-cloud network bandwidth fluctuations and relying solely on the interruption warning signals from cloud service providers to trigger checkpoints, resulting in the inability to migrate task status data in a timely manner when the network is congested, leading to task failure.

[0024] (2) Fixed frequency checkpoints: lack of risk awareness, the timing of saving does not match the risk status, resulting in frequent execution of input / output (I / O) operations when the risk is low, wasting computing resources, and potentially losing a large amount of task computing progress when the risk is high.

[0025] (3) Price threshold-based forecasting: Focusing only on market price changes and ignoring non-price disruption risks makes it difficult to cope with mission interruptions caused by sudden resource recovery.

[0026] To address at least one of the aforementioned problems in the related technologies, this application provides a data migration method that can proactively determine and execute cross-node migration of state data based on the risk score of the target computing task being executed on the current execution node and the warning interruption duration of the current execution node, combined with the estimated migration time required for state data migration. This ensures that the target computing task completes the safe migration of state data before the current execution node is reclaimed, avoiding the loss of computing results and waste of computing power due to interruption.

[0027] Figure 1 This is one of the flowcharts illustrating a data migration method provided in an embodiment of this application; as shown below. Figure 1 As shown, the method may include the following steps 101 to 103: Step 101: Determine the risk score of the target computing task being executed on the current execution node facing execution interruption, and the warning interruption duration of the current execution node.

[0028] It should be noted that the current execution node refers to the computing node that is currently executing the target computing task. This type of computing node is a preemptive resource node, and in this embodiment, it can be a public cloud-provided auction instance. This execution node is characterized by low cost but may be reclaimed at any time. The target computing task refers to the high-performance computing task running on the current execution node. The current execution node is the carrier of the target computing task, and the risk of interruption of the target computing task stems from the possibility that the current execution node may be reclaimed by the cloud service provider. The risk score of the target computing task facing execution interruption can be a comprehensive quantitative indicator used to characterize the probability and potential loss of the target computing task currently facing interruption. This score is a value between 0 and 1, with a higher value indicating a greater risk.

[0029] It's important to note that the warning outage duration is a fixed constant pre-published by the cloud service provider, representing the time between issuing the outage notification and actually reclaiming the spot instance. For example, the warning outage duration for AWS EC2 spot instances is 120 seconds, while for Azure and GCP it's 30 seconds. This duration is a known constant and can be known without waiting for an actual outage notification.

[0030] In some embodiments, a risk score can be determined by comprehensively considering multiple risk factors that affect the risk of execution interruption of the target computing task.

[0031] In some embodiments, a risk score can be obtained by weighted summation of multiple risk factors.

[0032] In some embodiments, the corresponding warning interruption duration can be obtained from the local configuration or the cloud service provider's public documentation, based on the instance type of the current execution node (such as AWS EC2 Spot).

[0033] Step 102: Determine the migration time required to migrate the state data of the target computing task from the current execution node to the target node.

[0034] It should be noted that status data refers to the necessary information generated and stored in memory during the execution of the target computation task, including variable values, computation progress, and model parameters. This data is the basis for resuming execution after the target computation task is interrupted.

[0035] It should be noted that migration time refers to the estimated time required to completely transfer state data from the current execution node to the target node. This time is directly proportional to the amount of state data and inversely proportional to network bandwidth.

[0036] It should be noted that the target node is the recipient of the state data, which can be a remote storage system (such as Network Attached Storage (NAS) or object storage) or a standby compute node (such as on-demand instance).

[0037] In some embodiments, the data volume information of the current state data of the target computing task and the network transmission capability information between the current execution node and the target node can be obtained, and then the migration time can be determined based on the data volume information and the network transmission capability information.

[0038] In some embodiments, system fixed overhead and / or network jitter factor may also be considered when calculating the migration duration. System fixed overhead may include: system response time (the time delay between the system determining that state data migration is needed and the start of migration command execution), signal processing time (the time from receiving or generating a migration trigger signal to the signal being recognized and responded to by the corresponding processing module), and I / O initialization time (the preparation time from the start of data transmission preparation to the actual data transmission via the network). The network jitter factor is a safety redundancy coefficient introduced to cope with sudden network fluctuations. For example, if the pure transmission time is 100 seconds and the jitter factor is 0.2, then the actual reserved time is 120 seconds (100 × 1.2), with the extra 20 seconds used to cope with sudden network fluctuations.

[0039] Step 103: If, based on the risk score, the warning interruption duration, and the migration duration, it is determined that the target computing task needs to undergo state data migration, the state data is migrated from the current execution node to the target node.

[0040] In some embodiments, the warning interruption duration (i.e., the fixed interruption warning window provided by the cloud service provider) can be compared with the dynamically estimated migration duration, while the risk score is used as a safety redundancy to comprehensively determine whether it is necessary to proactively migrate state data before the interruption. When the comparison indicates that the warning interruption duration cannot meet the data transmission requirements under the current network conditions, the system determines that the state data needs to be migrated immediately. This determination does not depend on the actual receipt of the warning interruption signal, but is proactively triggered based on real-time calculated parameters. After determining that migration is necessary, the system transmits the state data from the current execution node to the target node, ensuring that data is saved before the interruption occurs.

[0041] In some embodiments, after the migration is complete, status data can be obtained from the target node, and backup computing resources can be started to continue executing the target computing task; or, the target computing task can be put into a dormant state, waiting for subsequent resumption of execution triggered by the system or user.

[0042] In this embodiment, the risk score of the target computing task being executed on the current execution node is first determined to quantify the interruption risk and obtain the warning interruption duration of the current execution node. Then, the migration duration required to migrate the state data of the target computing task from the current execution node to the target node is determined to dynamically estimate the actual time required to complete the state data migration. Next, based on the risk score, warning interruption duration, and migration duration, it is proactively determined whether the state data of the target computing task needs to be migrated immediately. If migration is deemed necessary, the state data is immediately migrated from the current execution node to the target node. This proactively triggers state data migration without relying on warning interruption signals when the warning interruption duration is insufficient or network conditions deteriorate, ensuring the safe migration of state data for the target computing task before the current execution node is reclaimed, and avoiding the loss of computing results and waste of computing power due to interruption.

[0043] In some embodiments, such as Figure 2 As shown, step 101 may further include steps 1011 to 1012: Step 1011: Determine the first probability that the current execution node will be reclaimed, the second probability that the state data of the target computing task will fail to be transmitted from the current execution node to the target node within the warning interruption period, and the extent of loss of computing resources already occupied by the target computing task due to execution interruption.

[0044] It's important to clarify that the first probability refers to the likelihood that the current execution node will be reclaimed by the cloud service provider within a certain period. This probability is an indicator for assessing market risk and can be determined based on a combination of the current execution node's price information and historical outage information. The price information can include price fluctuations reflecting the scarcity of available computing resources for the current execution node per unit of time. The second probability refers to the likelihood that, under current network conditions, state data cannot be transmitted from the current execution node to the target node within the warning outage duration provided by the cloud service provider. This second probability reflects network transmission risk and is related to real-time network bandwidth, the amount of state data, and the warning outage duration. The degree of loss refers to the extent to which invested computing resources (such as running time and completed computations) will be wasted if the current execution node experiences an outage at the current moment. This degree of loss is positively correlated with the task's progress; the closer the task is to completion, the greater the loss caused by the outage.

[0045] In some embodiments, the first probability can be calculated based on the price information and historical interruption information of the current execution node.

[0046] In some embodiments, the critical bandwidth can be obtained by dividing the amount of state data by the warning interruption duration, and then a second probability can be calculated based on the critical bandwidth and the real-time network bandwidth.

[0047] In some embodiments, the loss degree can be calculated by dividing the running time of the target computing task by the estimated total running time.

[0048] At step 1012, the first probability, the second probability, and the loss degree are weighted and summed to obtain the risk score.

[0049] In the embodiments of the present application, the first probability, the second probability, and the loss degree determined at step 1011 can be weighted and summed to obtain the risk score. The weight coefficients of the weighting can be preset with initial values according to historical experience, and can be dynamically adjusted through a feedback correction mechanism subsequently. For example, initially, the first probability, the second probability, and the loss degree can be set to have similar weights; as the running experience accumulates, the ratio of each weight can be dynamically optimized according to actual interruption conditions, so that the risk judgment is more accurate.

[0050] It can be understood that the embodiments of the present application quantify the risk from three dimensions respectively: the first probability of the current execution node being recycled (market risk), the second probability of the state data failing to be transmitted within the pre-warning interruption time (network risk), and the loss degree of the occupied computing resources possibly caused by the interruption (progress risk). The three dimensions correspond to the "possibility", "feasibility", and "cost" of interruption respectively. By weighted sum of the three, a comprehensive risk score is obtained. In this way, the overall interruption risk faced by the target computing task can be comprehensively and balancedly evaluated, and the one-dimensional judgment is avoided, which provides an accurate risk quantification basis for subsequent active migration decision.

[0051] In some embodiments, at step 1011, the first probability of the current execution node being recycled is determined by: obtaining price fluctuation information and historical interruption information of the current execution node; based on the price fluctuation information and the historical interruption information, determining the first probability of the current execution node being recycled.

[0052] It should be noted that the price fluctuation information refers to dynamic data of the market price of the current execution node (bidding instance) changing over time. The price fluctuation information can include a price fluctuation rate. The price fluctuation rate can be obtained by taking the derivative of the historical price in the time dimension, represents the change rate of the historical price in the time dimension, and can reflect the scarcity of available computing resources of the current execution node per unit time. When the price fluctuation rate sharply rises (i.e., the scarcity of available resources sharply rises), the corresponding price sharply rises, indicating that the supply and demand of the instance type are tight, and the risk of recycling the instance increases. The historical interruption information refers to historical interruption statistical data of the current execution node instance type disclosed by the cloud service provider. For example, AWS SpotAdvisor provides historical interruption rates of different instance types, and the interruption rate of popular GPU instances (such as p4 and g5) can be more than 20%. The higher the historical interruption rate, the worse the stability of the instance type, and the higher the probability of recycling.

[0053] In some embodiments, the historical price data of the current execution node can be monitored in real time, the change rate of the historical price in the time dimension can be calculated, and the change rate can be taken as the price fluctuation information. The historical interruption information can be obtained from the interface or document disclosed by the cloud service provider. For instance types with high interruption rates, a higher historical interruption risk value can be preset.

[0054] In some embodiments, the price fluctuation information and the historical interruption information can be weighted and summed to determine the first probability, or the larger value in the price fluctuation information and the historical interruption information can be taken as the first probability. For example, when the scarcity of available resources sharply rises, the corresponding price sharply rises, and the first probability tends to 1; when the scarcity of available resources is at a lower level and changes gently, the corresponding price is relatively stable, and the first probability is mainly determined by the historical interruption information.

[0055] It can be understood that the embodiments of the present application can more accurately and stably quantify the recycling risk of the current execution node by fusing the price fluctuation information and the historical interruption information, thereby providing a reliable market risk basis for subsequent risk assessment.

[0056] In some embodiments, the method further comprises: After migrating the state data from the current execution node to the target node, adjusting the weights of the first probability, the second probability, and the loss degree in the weighted sum based on the result of whether the current execution node is recycled.

[0057] It should be noted that whether the current execution node has been recycled refers to whether the system has actually recycled after the state data migration is completed, by polling the cloud service provider's metadata interface or monitoring the instance status. Adjusting the weights refers to dynamically modifying the weights of the first probability, the second probability, and the degree of loss in the weighted sum based on whether the current execution node has been recycled, to form feedback correction. This allows the system to learn from each decision and optimize its parameters, for example, through positive reinforcement or negative punishment, making subsequent risk assessments more accurate.

[0058] In some embodiments, it is possible to monitor whether the currently executing node is actually recycled after the migration is completed. Based on the monitoring results, the weights in the weighted summation are dynamically adjusted as follows: if state data migration is triggered but recycling does not actually occur (false alarm), the weight of the risk factor with the highest weight value in this risk score calculation (such as the first probability, the second probability, and the degree of loss) can be appropriately reduced to reduce the occurrence of similar false alarms; if state data migration is triggered and recycling actually occurs (correct warning), the current weights can be kept unchanged; if state data migration is not triggered but recycling actually occurs (missed alarm), the weights of each risk factor are reallocated, and the weights of the risk factors that are underestimated in this risk score calculation can be increased to improve the system's sensitivity to these risk factors.

[0059] It is understood that the embodiments of this application can make risk judgment more and more accurate by dynamically adjusting the weights of the first probability, the second probability and the degree of loss in the weighted summation, i.e., feedback correction, and gradually optimize the weight ratio of each risk factor in the weighted summation.

[0060] In some embodiments, such as Figure 3 As shown, step 102 may further include steps 1021 to 1022: Step 1021: Determine the amount of the status data and the network bandwidth information between the current execution node and the target node.

[0061] It should be noted that the state data volume refers to the total size (usually in GB) of data generated during the execution of the target computing task that needs to be persistently stored. This data volume is a fundamental input parameter for calculating migration time, and its size directly affects the transmission duration. Network bandwidth information characterizes the real-time network transmission capability between the current executing node and the target node (usually in Mbps or Gbps). This information reflects the channel capacity for data transmission and is a key factor in determining transmission speed. Network bandwidth has dynamic fluctuations and can be obtained in real time through active probing or passive monitoring.

[0062] In some embodiments, the amount of state data can be obtained by querying the task's runtime memory usage or by calling relevant interfaces. For example, the system can periodically obtain the memory snapshot size of the target computing task as the amount of state data to be migrated.

[0063] In some embodiments, network bandwidth information can be obtained by actively sending probe packets or monitoring historical transmission rates. For example, the system can send several small probe packets per second to the target node and estimate the currently available bandwidth based on the round-trip time and packet loss rate; alternatively, it can monitor the actual data transmission rate over a recent period as a bandwidth estimate. The obtained network bandwidth information can be an instantaneous value or a statistical value over a period of time (such as an average or a minimum value within a sliding window) to ensure the reliability of the estimate.

[0064] Step 1022: Based on the data volume and the network bandwidth information, determine the migration time required to migrate the status data of the target computing task from the current execution node to the target node.

[0065] In some embodiments, the migration time can be obtained by dividing the data volume by the network bandwidth. For example, if the state data volume is 10GB and the current network bandwidth is 500Mbps, then the migration time is approximately 10GB / 500Mbps 160 seconds.

[0066] In some embodiments, the migration time can be obtained by dividing the data volume by the network bandwidth and then adding the system fixed overhead (such as I / O initialization time, signal processing time, etc., typically 10 to 15 seconds).

[0067] In some embodiments, the migration time can be obtained by dividing the data volume by the network bandwidth, adding the system fixed overhead, and the product of the system fixed overhead and the jitter factor. That is, migration time = data volume ÷ network bandwidth + system fixed overhead × (1 + jitter factor). The jitter factor can be between 0.1 and 0.3 to reserve time redundancy for system response and I / O initialization.

[0068] It is understood that this embodiment first determines the amount of state data and the network bandwidth information between the current execution node and the target node, and then dynamically calculates the migration time required to complete the state data migration based on the data amount and network bandwidth information. This migration time can truly reflect the transmission requirements under the current network conditions.

[0069] In some embodiments, such as Figure 4 As shown, step 103 may further include steps 1031 to 1032: Step 1031: Determine the safety threshold based on the risk score and the migration duration.

[0070] It should be noted that the safety threshold is a reference value obtained by combining the risk score and the migration time length, and is used for comparison with the warning interruption time length provided by the cloud service provider. The safety threshold can reflect the time requirement after considering the risk. The higher the risk score, the larger the safety threshold, which means a longer safety window is required; otherwise, a shorter safety window is required.

[0071] In the embodiments of the present application, the safety threshold can be calculated based on the migration time length and the risk score. For example, the safety threshold can be obtained by adding the migration time length and the product of the migration time length and the risk score. When the risk score is low, the safety threshold is close to the migration time length; when the risk score is high, the safety threshold is significantly greater than the migration time length, thereby reserving more time redundancy for high-risk scenarios. The safety threshold can also be calculated in other ways as needed, as long as it can reflect the amplification effect of the risk score on the time requirement.

[0072] Step 1032, in a case where it is determined that the warning interruption time length is less than the safety threshold, it is determined that the target computing task needs to perform state data migration, and then the state data is migrated from the current execution node to the target node.

[0073] In the embodiments of the present application, the safety threshold determined in step 1031 can be compared with the warning interruption time length provided by the cloud service provider. When the warning interruption time length is less than the safety threshold, it is determined that state data migration is needed, and the migration operation is immediately performed. Specifically, the system obtains the warning interruption time length of the instance type corresponding to the current execution node (for example, 120 seconds for AWS EC2 bid instance), and compares it with the safety threshold determined in step 1031. If the warning interruption time length is less than the safety threshold, it indicates that the warning interruption window is insufficient to safely complete data migration under the current risk condition and network condition, and the system actively triggers state data migration without waiting for the cloud service provider to issue a warning interruption signal. During migration, the system completely transmits the state data of the target computing task from the current execution node to the target node.

[0074] It can be understood that, in the embodiments of the present application, the safety threshold is determined based on the risk score and the migration time length, and the safety threshold comprehensively reflects the actual time requirement after considering the risk. When the warning interruption time length provided by the cloud service provider is less than the safety threshold, it indicates that the warning interruption window is insufficient to safely complete data migration under the current risk condition and network condition. At this time, instead of waiting for the cloud service provider to issue a warning interruption signal, state data migration is actively triggered immediately. In this way, state data migration can be performed in advance when the warning interruption window is insufficient, thereby avoiding state data loss due to exhaustion of the warning interruption window.

[0075] In some embodiments, the method further comprises: in a case where it is determined that the pre-warning interruption duration is greater than or equal to the safety threshold and a deviation of the pre-warning interruption duration relative to the safety threshold is less than a specific threshold, shortening, on a basis of a specific time interval, a time interval of local saving of state data of the target computing task at the current execution node.

[0076] It can be understood that the pre-warning interruption duration is greater than or equal to the safety threshold, and the deviation of the pre-warning interruption duration relative to the safety threshold is less than the specific threshold, which means that the pre-warning interruption duration provided by the cloud service provider is very close to the safety threshold, that is, the two are exactly equal or the pre-warning interruption duration is slightly excessive, and belongs to a risk critical state.

[0077] It should be noted that the specific time interval refers to a preset regular checkpoint execution interval (for example, an optimal interval calculated according to the Young / Daly formula). When in a low-risk state, the system can perform checkpoint saving according to the normal interval.

[0078] In the embodiment of the present application, when the pre-warning interruption duration is greater than or equal to the safety threshold, and the deviation of the pre-warning interruption duration relative to the safety threshold is less than the specific threshold, it is determined that the current state is a risk critical state. At this time, although the pre-warning interruption window provided by the cloud service provider is sufficient, the margin is insufficient, and if there is a slight network fluctuation or system response delay, it may still cause migration failure. Therefore, the system performs a defensive processing strategy: on the basis of the preset regular checkpoint interval, the saving time interval of the state data at the current execution node SSD is shortened. For example, based on the idea of Shannon's theorem (i.e., the Nyquist sampling theorem), the checkpoint interval can be shortened to half of the normal interval, or the adjustment range is dynamically adjusted according to the risk score. By increasing the saving frequency, the system can reduce the calculation progress loss caused by the interruption as much as possible in the case of insufficient window margin.

[0079] It can be understood that the present application determines that the current state is a risk critical state when it is determined that the pre-warning interruption duration is greater than or equal to the safety threshold and the deviation of the pre-warning interruption duration relative to the safety threshold is less than the specific threshold. At this time, the state data is not immediately migrated, but the local saving time interval is shortened on the basis of a specific time interval, and a defensive write is performed. In this way, the saving frequency of the state data can be improved without interrupting the target computing task, and a buffer is reserved for network fluctuation or system delay; at the same time, the calculation interruption and network resource consumption caused by directly triggering migration are avoided, and a balance is achieved between risk control and resource efficiency.

[0080] In some embodiments, the method further comprises: If it is determined that the warning interruption duration is greater than the safety threshold, and the deviation of the warning interruption duration from the safety threshold is greater than or equal to the specific threshold, the status data of the target computing task is saved locally on the current execution node at the specific time interval.

[0081] It is understandable that if the warning interruption duration is greater than the safety threshold, and the deviation of the warning interruption duration from the safety threshold is greater than or equal to a specific threshold, it means that the warning interruption duration provided by the cloud service provider is significantly greater than the currently calculated safety threshold, the warning interruption window time is sufficient, and network fluctuations or system delays are unlikely to cause migration timeouts.

[0082] In this embodiment, when the warning interruption duration exceeds a safety threshold, and the deviation of the warning interruption duration from the safety threshold is greater than or equal to a specific threshold, the system determines that it is currently in a low-risk state. At this time, the warning interruption window is ample, the risk is low, and no additional defensive measures or emergency migration are required. The system saves the status data of the target computation task locally on the current execution node according to a preset specific time interval (such as the optimal checkpoint interval determined by the Young / Daly formula). For example, if the optimal checkpoint interval is 45 minutes, the system performs a regular checkpoint operation every 45 minutes, writing the status data to the local SSD.

[0083] It is understood that, in this embodiment of the application, a low-risk state is determined when the warning interruption duration exceeds a safety threshold and the deviation of the warning interruption duration from the safety threshold is greater than or equal to a specific threshold. In this case, the state data is not immediately migrated; instead, the state data of the target computation task is saved locally on the current execution node at specific time intervals. That is, by maintaining a normal checkpoint frequency, the system can minimize I / O overhead while ensuring data security, thereby improving computational efficiency.

[0084] In some embodiments, step 103, migrating the state data from the current execution node to the target node, includes: The status data is divided into multiple data blocks; The multiple data blocks are transmitted to the target node in parallel using multiple parallel transmission threads.

[0085] In some embodiments, the state data to be migrated can be divided into multiple independent data segments, i.e., data blocks, according to a certain size (such as 5MB, 10MB, or 100MB). Each data block can be transmitted and acknowledged independently, and if the transmission of a data block fails, only that data block needs to be retransmitted, without retransmitting the entire state data.

[0086] It should be noted that multiple parallel transmission threads can run multiple transmission tasks simultaneously, with each thread responsible for transmitting one or more data blocks. By executing in parallel, network bandwidth can be fully utilized, the total transmission time can be shortened, and transmission efficiency can be improved.

[0087] In some embodiments, when the system determines that state data migration is necessary, a parallel block transmission method can be used to transmit the state data to the target node. Specifically, the state data to be migrated is first divided into multiple data blocks according to a preset data block size (e.g., 10MB). After division, each data block has an independent identifier, facilitating subsequent transmission confirmation and breakpoint resumption. Then, multiple parallel transmission threads (e.g., 4 or 8 threads) are started, with each thread responsible for transmitting one or more data blocks. Each thread works independently, transmitting its assigned data blocks to the target node in parallel. After receiving all data blocks, the target node reassembles them into complete state data based on the data block identifiers.

[0088] It is understood that the embodiments of this application, by dividing the state data into multiple data blocks and initiating multiple parallel transmission threads to transmit these data blocks to the target node in parallel, can fully utilize the bandwidth resources of the cross-cloud network and significantly shorten the total time of state data migration. Compared with the serial transmission method, the parallel block transmission method improves transmission efficiency and increases the success rate of completing state data migration before interruption.

[0089] The following examples illustrate possible implementation schemes of the data migration method described in one or more of the above embodiments.

[0090] Figure 5 This is a schematic diagram of a risk perception and migration system architecture provided in an embodiment of this application, such as... Figure 5 As shown, the solution provided in this application embodiment is applied in a hybrid cloud environment and mainly includes the following components: Public Cloud (Spot Instances): The Compute Nodes adopt the spot instance type and serve as the main execution carrier for the target computing tasks. This public cloud can also deploy risk sentinel agents to monitor and determine the probability of spot instance outages and WAN bandwidth in real time.

[0091] Local HPC Cluster: When local computing power is insufficient, tasks can be elastically extended (burst) to public cloud spot instances for execution; moreover, as a stable computing resource in a hybrid cloud architecture, it can serve as a migration target for state data when public cloud spot instances face the risk of interruption.

[0092] Remote Network Attached Storage (NAS): Used for persistent storage of state data (i.e., checkpoints).

[0093] Local solid state drive (SSD): Used to store state data in low-risk situations, enabling fast checkpoint operations.

[0094] Risk Perception and Time Window Calculator: This includes a "network + computing power" risk model and a dynamic security time window setting. The "network + computing power" risk model is used to jointly assess network and computing power risks. The dynamic security time window setting is used to calculate the security boundary (i.e., the security threshold in the above embodiments) and compare it with the warning window provided by the cloud service provider (i.e., the warning interruption duration in the above embodiments).

[0095] It should be noted that in this solution, a risk sentinel agent is deployed on the Spot instance in the public cloud. This risk sentinel agent can monitor the status of the spot instance, cross-cloud bandwidth, and the amount of status data of the task in real time, and work with the private cloud controller to execute save and migration commands.

[0096] exist Figure 5 In the system architecture shown, under normal circumstances, the target computing task is executed on the local high-performance computing cluster. When local computing power is insufficient, the target computing task is elastically scaled to a Spot instance on the public cloud through a "Burst to Cloud" mechanism. When it is determined through risk perception and time window calculation and judgment that Spot faces the risk of interruption, the status data of the target computing task is migrated from the public cloud spot instance to a remote NAS or local SSD according to the risk level.

[0097] Understandably, this solution transforms passive disaster recovery into proactive risk management, encompassing at least the following three aspects: 1. Construct a "network + computing power" risk model. Mathematically model network transmission risk and computing power interruption risk, and accurately predict the shrinkage of the security boundary before physical interruption occurs by combining market price signals, real-time network detection, and application layer state awareness.

[0098] 2. Dynamic security time window setting. Dynamically calculates the time required for data migration under the current network conditions, and dynamically calculates the security boundary in combination with the risk score. It is then compared with the fixed warning window provided by the cloud service provider to achieve proactive risk assessment, balancing computational efficiency and data security.

[0099] 3. Multi-level I / O persistence strategy based on risk and security windows. Automatically switch storage media (such as local SSD and remote NAS) according to risk level to optimize write efficiency. Proactively trigger checkpoints and task migration when the warning interruption duration is less than the security threshold, without relying on interruption notifications from cloud service providers.

[0100] Through the aforementioned risk management mechanism, this plan can achieve at least the following beneficial effects: (1) Task completion rate improved: from 60-70% to over 95%; (2) Cost advantage remains: the overall cost is still 50%-80% lower than that of pure on-demand instances; (3) Reduced computational waste: Compared with the traditional bidding instance scheme, it reduces invalid computation waste by about 30%.

[0101] Furthermore, this solution employs proactive risk management and a multi-level storage strategy to prioritize avoiding "missed reports" (i.e., actual interruptions that do not trigger migration) and tolerate controllable "false reports" (i.e., migrations that are triggered without interruption). Even under deteriorating network conditions, it effectively ensures the survivability of long-cycle HPC tasks and effectively prevents the loss of computational results.

[0102] Figure 6 This is a schematic diagram of a risk perception and migration decision-making process provided in an embodiment of this application, such as... Figure 6 As shown, when the task starts, the initialization and monitoring module 601 first initializes parameters, such as loading historical interruption rate data of the bidding instance, establishing an initial bandwidth benchmark, and configuring the Spot metadata listener; then it performs real-time monitoring loops, such as polling the market price of the bidding instance every 5 seconds, actively probing the network bandwidth (BandwidthProbe) between the current execution node and the target storage node, and tracking the runtime of the target computing task to calculate the task progress. Subsequently, the risk perception module 602 calculates the market risk probability (i.e., the first probability in the above embodiment) based on the market price volatility and historical interruption rate of the bidding instance; the risk perception module 602 calculates the network risk probability (i.e., the second probability in the above embodiment) based on the network bandwidth; the risk perception module 602 calculates the progress risk probability (i.e., the degree of loss in the above embodiment) based on the task progress; then, the risk perception module 602 performs a weighted summation of the three risk probabilities to obtain a comprehensive risk score; next, the security window module 603 calculates the security boundary based on the risk score, and then judges the risk zone (including the green zone (low risk), the yellow zone (critical risk), and the red zone (high risk)) based on the security boundary. Finally, the execution feedback module 604 immediately migrates the status data for red zone risks. After the migration is completed, it updates the parameters and adjusts the weights based on whether an interruption actually occurs.

[0103] It's important to note that Spot instance reclamation is not a random event but follows specific supply and demand market rules. Different cloud service providers offer different warning and interruption mechanisms, which determine the upper limit of the safe time window. For example, AWS EC2 Spot instances provide a 120-second (2-minute) warning window, notified via the Instance Metadata Service (IMDS) or EventBridge. A typical handling strategy is to poll the IMDS or trigger a Lambda function to execute a script. In contrast, Google Cloud's Preemptible VMs or Spot virtual machines only provide a 30-second warning window, notified via the Metadata Server or Advanced Configuration and Power Interface (ACPI) signals. Due to the extremely short time, this requires frequent checkpointing or hot migration strategies. Azure's Spot virtual machines also only provide a 30-second warning window, notified via the Scheduled Events API. This duration is only sufficient to store a small amount of metadata and is not suitable for applications with large memory requirements.

[0104] In this solution, the risk perception module 602 can calculate the probabilities of the following three types of risks: (1) Market Risk Probability. This is calculated based on the market price volatility and historical outage rate of the current execution node (bid instance). For example, a weighted sum of the two can be used, with the weights dynamically adjusted according to the actual scenario; alternatively, the larger of the two values ​​can be taken as the market risk probability. The specific combination method can be flexibly configured according to application needs. Market price volatility can be obtained by analyzing the derivative of the historical price of the current execution node; when the price rises sharply, the market risk probability approaches 1. Historical outage rate can be obtained by analyzing publicly available data from cloud service providers (such as AWS Spot Advisor); when the historical outage rate of popular instance types (such as GPU instances p4 / g5) exceeds a certain threshold (such as 20%), the market risk probability increases accordingly. It can be understood that the market risk probability... It represents the probability that the auctioned instance will be recycled.

[0105] (2) Network Risk Probability. Based on the real-time network bandwidth between the current executing node and the target node, calculate the probability that the state data transmission will fail within the warning interruption duration. Specifically, when the real-time bandwidth is lower than the critical bandwidth (i.e., the minimum bandwidth required to complete transmission, equal to the amount of state data divided by the warning interruption duration), the network risk probability approaches 1; when the real-time bandwidth is much higher than the critical bandwidth, the network risk probability approaches 0. It can be calculated using the following formula (1): (1) in, Indicates the critical bandwidth; Indicates real-time bandwidth; This is a sensitivity parameter that can be calibrated.

[0106] (3) Schedule Risk Probability. Calculated as the ratio of the completed time of the target calculation task to the estimated total time, it represents the cost or loss of completed work due to interruption. The closer the target calculation task is to completion, the greater the loss caused by interruption, and the higher the schedule risk probability accordingly. Schedule Risk Probability It can be calculated using the following formula (2): (2) in, This is the runtime; For the estimated total duration; This is a schedule risk index used to adjust the rate at which risk increases with the schedule.

[0107] In this solution, the risk perception module 602 can perform a weighted summation of the above-mentioned market risk probability, network risk probability, and schedule risk probability to obtain a comprehensive risk score. As shown in equation (3): (3) in, Let be the weight coefficient, and satisfy... The initial weights can be set based on experience (e.g.) (This can be dynamically adjusted subsequently through the closed-loop learning mechanism of the feedback module 604.)

[0108] In this solution, the security window module 603 can dynamically calculate the migration time required to complete the state data migration based on the amount of state data that needs to be persisted and the real-time network bandwidth. As shown in equation (4): (4) in, This indicates the amount of state data (in GB) that needs to be persisted in the current application's memory. This represents the lower limit of effective bandwidth (Gbps) within a future set duration (e.g., 2 minutes) predicted based on a sliding window. This represents the fixed overhead for system response, signal processing, and I / O initialization (the value can be 10-15 seconds). This represents the Safety Factor, used to address sudden network jitter, and can be calibrated.

[0109] In this solution, the security window module 603 can also adjust the migration duration. and risk score Calculate the security boundary (i.e., the security threshold in the above embodiments), and the security boundary is equal to Furthermore, it can also adjust the downtime of alerts provided by cloud service providers. By comparing it with the safety boundary, the current risk zone can be determined. Specifically, when the following conditions are met... When the current risk zone is determined to be a green zone; when the conditions are met... When the current risk zone is determined to be a yellow zone; when the conditions are met... At that time, the current risk zone is determined to be a red zone.

[0110] In this solution, upon triggering a red zone, the execution feedback module 604 immediately suspends the execution of the target computation task (this can be done by calling a Message Passing Interface (MPI) command), forcibly migrating state data from the current execution node to the target node. For example, it initiates multipart upload to write the data to Simple Storage Service (S3) or Elastic Block Store (EBS), and then releases the current execution node or switches to an on-demand instance to continue execution. After the migration is complete, the weighting coefficients of market risk probability, network risk probability, and schedule risk probability in the weighted summation are dynamically adjusted based on whether the current execution node has actually been reclaimed, forming a closed-loop learning mechanism. Simultaneously, for yellow zone risks, the interval for storing state data on the local SSD can be shortened (e.g., based on the Nyquist sampling theorem in signal processing, the interval is shortened to half its original value), and defensive writes are performed. For green zone risks, regular checkpointing operations can be performed according to the optimal Young / Daly interval.

[0111] For example, suppose at a certain moment (e.g., T=30 minutes), network bandwidth is detected to drop sharply from 2Gbps to 500Mbps. Based on the current state data volume and real-time bandwidth, the migration time is calculated to be approximately 1024 seconds. Assuming the current risk score is 0.1, the security boundary is approximately 1126.4 seconds, while the cloud service provider's warning interruption duration is only 120 seconds. Since the warning interruption duration is less than the security boundary, the system determines that it has entered the red zone and, without waiting for the cloud service provider to issue a warning interruption signal, immediately triggers a checkpoint operation. Because the spot instance has not yet been forcibly reclaimed, the program has sufficient time to complete the writing of state data. After writing, the current spot instance is actively released, and a new spot instance is requested or switched to an on-demand instance to continue running. Ultimately, the task's state data is successfully saved. Although some computation time is lost due to data transmission, task rollback and loss of computation results due to spot instance interruption are avoided.

[0112] Understandably, this solution is like equipping a supercomputing task with an intelligent evacuation assistant before a typhoon. Traditional solutions rely on a fixed "2-hour typhoon warning," while this solution can sense network bandwidth (like traffic congestion) and computing power interruption risks (like the dynamic changes of a typhoon) in real time, dynamically calculate the safe "latest evacuation time" (i.e., the safety boundary), and trigger task migration and checkpoint saving in advance. This ensures that the computing results (like valuable luggage) safely arrive at the private cloud or secure storage (like an inland castle) before the bidding instance is interrupted (like a typhoon hitting), thereby avoiding the loss of computing results for cross-cloud Spot supercomputing tasks.

[0113] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution.

[0114] Based on the same inventive concept as the foregoing embodiments, this application provides a data migration device.

[0115] Figure 7 This is a schematic diagram of the structure of a data migration device provided in an embodiment of this application, as shown below. Figure 7 As shown, the data migration device 700 includes: a first determining module 710, a second determining module 720, and a data migration module 730; wherein: The first determining module 710 is used to determine the risk score of the target computing task being executed on the current execution node facing execution interruption, and the warning interruption duration of the current execution node; The second determining module 720 is used to determine the migration time required to migrate the state data of the target computing task from the current execution node to the target node; The data migration module 730 is used to migrate the state data from the current execution node to the target node when it is determined that the target computing task needs to perform state data migration based on the risk score, the warning interruption duration, and the migration duration.

[0116] In some embodiments, the first determining module 710 is further configured to: determine a first probability that the current execution node is recycled, a second probability that the state data of the target computing task fails to be transmitted from the current execution node to the target node within the warning interruption duration, and the degree of loss of the computing resources occupied by the target computing task due to the execution interruption; and perform a weighted summation of the first probability, the second probability and the degree of loss to obtain the risk score.

[0117] In some embodiments, the first determining module 710 is further configured to: obtain price fluctuation information and historical interruption information of the current execution node; and determine a first probability that the current execution node will be recycled based on the price fluctuation information and the historical interruption information.

[0118] In some embodiments, the data migration apparatus 700 further includes: The adjustment module is used to adjust the weights of the first probability, the second probability, and the degree of loss in the weighted summation based on whether the current execution node is recycled after the state data is migrated from the current execution node to the target node.

[0119] In some embodiments, the second determining module 720 is further configured to: determine the amount of data in the status data and the network bandwidth information between the current execution node and the target node; and based on the amount of data and the network bandwidth information, determine the migration time required to migrate the status data of the target computing task from the current execution node to the target node.

[0120] In some embodiments, the data migration module 730 is further configured to: determine a safety threshold based on the risk score and the migration duration; if it is determined that the warning interruption duration is less than the safety threshold, and the target computing task needs to undergo state data migration, then the state data is migrated from the current execution node to the target node.

[0121] In some embodiments, the data migration module 730 is further configured to: when it is determined that the warning interruption duration is greater than or equal to the safety threshold, and the deviation of the warning interruption duration from the safety threshold is less than a specific threshold, shorten the time interval for storing the state data of the target computing task locally on the current execution node based on a specific time interval.

[0122] In some embodiments, the data migration module 730 is further configured to: when it is determined that the warning interruption duration is greater than the safety threshold, and the deviation of the warning interruption duration from the safety threshold is greater than or equal to the specific threshold, save the status data of the target computing task locally on the current execution node at the specific time interval.

[0123] In some embodiments, the data migration module 730 is further configured to: divide the state data into multiple data blocks; and transmit the multiple data blocks to the target node in parallel through multiple parallel transmission threads.

[0124] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0125] Based on the same inventive concept as the foregoing embodiments, this application provides an electronic device.

[0126] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 The illustrated electronic device 800 includes a processor 801, which can call and run computer programs from memory to implement the methods in the embodiments of this application.

[0127] Optionally, such as Figure 8 As shown, the electronic device 800 may further include a memory 802. The processor 801 can retrieve and run computer programs from the memory 802 to implement the methods described in the embodiments of this application.

[0128] The memory 802 can be a separate device independent of the processor 801, or it can be integrated into the processor 801.

[0129] Optionally, such as Figure 8 As shown, the electronic device 800 may also include a transceiver 803, which the processor 801 can control to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices.

[0130] The transceiver 803 may include a transmitter and a receiver. The transceiver 803 may further include an antenna, and the number of antennas may be one or more.

[0131] This application also provides a computer-readable storage medium for storing computer programs.

[0132] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of this application, and the computer program causes the processor or electronic device to perform the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0133] This application also provides a computer program product, including computer program instructions.

[0134] Optionally, the computer program product can be applied to the electronic device in the embodiments of this application, and the computer program instructions cause the processor or electronic device to execute the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0135] This application also provides a computer program.

[0136] Optionally, the computer program can be applied to the electronic device in the embodiments of this application. When the computer program runs on the processor or electronic device, it causes the processor or electronic device to execute the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0137] It should be noted that the descriptions of the devices, storage media, computer program products, and computer program embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the device, storage media, computer program products, and computer program embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0138] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0139] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0140] It should be understood that the above-described memory is exemplary and not a limiting description. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0146] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0147] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data migration method, the method comprising: Determine the risk score of the target computation task being executed on the current execution node facing execution interruption, and the warning interruption duration of the current execution node; Determine the migration time required to migrate the status data of the target computation task from the current execution node to the target node; If, based on the risk score, the warning interruption duration, and the migration duration, it is determined that the target computing task needs to undergo state data migration, the state data is migrated from the current execution node to the target node.

2. The method according to claim 1, wherein determining the risk score of execution interruption faced by the target computing task being executed on the current execution node includes: The first probability that the current execution node will be reclaimed, the second probability that the status data of the target computing task will fail to be transmitted from the current execution node to the target node within the warning interruption period, and the extent of loss of computing resources already occupied by the target computing task due to execution interruption are determined. The risk score is obtained by weighted summation of the first probability, the second probability, and the degree of loss.

3. The method according to claim 2, wherein determining the first probability that the currently executing node is reclaimed includes: Obtain the price fluctuation information and historical interruption information of the currently executing node; Based on the price fluctuation information and the historical interruption information, a first probability is determined that the current execution node will be recycled.

4. The method according to claim 2, further comprising: After migrating the state data from the current execution node to the target node, the weights of the first probability, the second probability, and the degree of loss in the weighted summation are adjusted based on whether the current execution node is reclaimed.

5. The method according to claim 1, wherein determining the migration time required to migrate the state data of the target computing task from the current execution node to the target node includes: Determine the amount of the status data, and the network bandwidth information between the current execution node and the target node; Based on the data volume and the network bandwidth information, the migration time required to migrate the status data of the target computing task from the current execution node to the target node is determined.

6. The method according to any one of claims 1 to 5, wherein migrating the state data from the current execution node to the target node when it is determined, based on the risk score, the warning interruption duration, and the migration duration, that the target computing task requires state data migration, comprises: Based on the risk score and the migration duration, a safety threshold is determined; If it is determined that the warning interruption duration is less than the safety threshold, and it is determined that the target computing task needs to perform state data migration, then the state data is migrated from the current execution node to the target node.

7. The method according to claim 6, further comprising: If the warning interruption duration is determined to be greater than or equal to the safety threshold, and the deviation of the warning interruption duration from the safety threshold is less than a specific threshold, the time interval for storing the status data of the target computing task locally on the current execution node is shortened based on a specific time interval.

8. The method according to claim 7, further comprising: If it is determined that the warning interruption duration is greater than the safety threshold, and the deviation of the warning interruption duration from the safety threshold is greater than or equal to the specific threshold, the status data of the target computing task is saved locally on the current execution node at the specific time interval.

9. The method according to any one of claims 1 to 5, wherein migrating the state data from the current execution node to the target node comprises: The status data is divided into multiple data blocks; The multiple data blocks are transmitted to the target node in parallel using multiple parallel transmission threads.

10. A data migration apparatus, the apparatus comprising: The first determining module is used to determine the risk score of the target computing task being executed on the current execution node facing execution interruption, and the warning interruption duration of the current execution node; The second determining module is used to determine the migration time required to migrate the state data of the target computing task from the current execution node to the target node; The migration module is used to migrate the state data from the current execution node to the target node when it is determined that the target computing task needs to perform state data migration based on the risk score, the warning interruption duration, and the migration duration.

Images