Data Migration Scheduling Method Based on Device Abnormality Detection

Abstract

The present invention relates to the field of information technology and discloses a data migration scheduling method based on device anomaly detection. The method includes the following steps: S1. Through the health state modeling of the device, monitor and real-time estimate the health state of the device, and judge whether the device is in an abnormal state based on the estimated value; S2. When the device is in an abnormal state, trigger data migration scheduling; S3. Design an optimized scheduling objective to minimize data migration latency, device load fluctuation, and bandwidth resource consumption, and calculate the data migration path and scheduling strategy based on the device health state and bandwidth constraints; S4. Use the Lagrange multiplier method to solve the optimal scheduling problem with constraints. By modeling and real-time estimating the device health state through stochastic differential equations and Kalman filtering algorithms, accurate anomaly state detection is achieved, effectively solving the problems of untimely response and insufficient accuracy to device state changes in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and specifically to a data migration scheduling method based on device anomaly detection. Background Art

[0002] With the development of information technology, large-scale device networks and complex systems (such as data centers, Internet of Things device networks, etc.) are increasingly widely used. The reliability of the operating state of devices in these systems is directly related to the overall performance and stability of the system. However, the operating environment and load conditions of devices are complex and variable, resulting in significant randomness in the dynamic changes of the device health state, and increasing the risk of device failures or performance degradation. To reduce the system performance degradation or data loss caused by failures, data migration scheduling technology has gradually become a key means in modern device management.

[0003] Existing device data migration scheduling methods usually rely on the following technical means: Anomaly detection mechanism with fixed thresholds. The detection of device abnormal states is usually based on simple fixed-threshold judgment methods. For example, when parameters such as temperature and current exceed the preset range, an alarm is triggered. However, this method has significant deficiencies in scenarios where the device health state fluctuates greatly. Static migration path and strategy design. In the prior art, the design of data migration paths and strategies is usually based on static analysis or historical experience, lacking the ability to adapt to the dynamic changes of device health states and system resources in real time. This fixed strategy often has low efficiency when facing complex and dynamically changing device operating environments. The singularity of optimization methods. Many existing methods adopt simple optimization objectives, such as minimizing migration time or reducing resource consumption, without comprehensively considering the balance of multiple key performance indicators (such as migration delay, device health state fluctuations, and bandwidth resource consumption). Lack of closed-loop optimization for dynamic scheduling. Traditional migration scheduling processes are often executed statically once, lacking real-time monitoring, feedback, and dynamic adjustment capabilities. Once the device operating environment changes (such as bandwidth resource tension or further deterioration of the device health state), the scheduling strategy cannot effectively adapt, resulting in a decline in migration efficiency or even system operation interruption.

[0004] In summary, existing device data migration scheduling methods have obvious deficiencies in aspects such as the accuracy of anomaly detection, the adaptability of dynamic scheduling, the comprehensiveness of optimization objectives, and the closed-loop nature of the scheduling process. These problems are particularly prominent in complex device operating environments, directly affecting the efficiency of data migration and the reliability of the system. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention provides a data migration scheduling method based on device anomaly detection, which solves the problems of obvious deficiencies in aspects such as the accuracy of anomaly detection, the adaptability of dynamic scheduling, the comprehensiveness of optimization objectives, and the closed-loop nature of the scheduling process in existing device data migration scheduling methods.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions: A data migration scheduling method based on device anomaly detection, comprising the following steps:

[0007] S1. Through the health state modeling of the device, monitor and real-time estimate the health state of the device, and based on this estimated value, determine whether the device is in an abnormal state;

[0008] S2. When the device is in an abnormal state, trigger data migration scheduling;

[0009] S3. Design an optimized scheduling objective to minimize data migration latency, device load fluctuations, and bandwidth resource consumption, and based on the device health state and bandwidth constraints, calculate the data migration path and scheduling strategy;

[0010] S4. Use the Lagrange multiplier method to solve the optimal scheduling problem with constraints to determine the data migration volume between each device;

[0011] S5. Use the stochastic optimal control method to model the randomness of the device health state, thereby dynamically adjusting the scheduling strategy to ensure the optimal data migration path of the device in normal or abnormal states;

[0012] S6. Apply the obtained optimal migration scheduling strategy to the system in real time.

[0013] Preferably, the steps of the health state modeling of the device include:

[0014] Obtain various monitoring data of the device and convert it into the initial value of the device health state;

[0015] Use a stochastic differential equation to model the change of the device health state, expressed as:

[0016]

[0017] where, is the health state of the device at time , is the drift term, is the diffusion term, is the random perturbation;

[0018] Compare with the observed data and the health state model to obtain the real-time estimated value of the device health state;

[0019] Compare the estimated value with the preset threshold to determine whether the device is in a normal state or an abnormal state, and provide a basis for subsequent anomaly detection and data migration decisions.

[0020] Preferably, the steps for estimating the health state of the device include:

[0021] Obtain the observed data of the device and assume that there is a linear relationship between it and the health state of the device ;

[0022] Use the Kalman filtering algorithm to estimate the health state of the device, and obtain the estimated health state of the device at each moment through recursive calculation ; ;

[0023] Compare the estimated health state with the preset normal health state threshold to determine whether the device is in an abnormal state;

[0024] When the health state of the device exceeds the threshold , trigger data migration scheduling.

[0025] Preferably, the steps for optimizing the scheduling objective include:

[0026] Determine the objective function of data migration , where it includes minimizing data migration latency, device health state fluctuations, and system resource consumption;

[0027] Set weight coefficients , which are used to balance the relative importance of migration latency, health state fluctuations, and bandwidth resource consumption;

[0028] By calculating the amount of data migration between each device , combined with the maximum carrying capacity of the device and the remaining available resources , optimize the objective function;

[0029] Under bandwidth constraints, calculate the optimal data migration path between devices to meet the system performance optimization objective.

[0030] Preferably, the steps for optimizing the scheduling objective function include:

[0031] By minimizing data migration latency , fluctuations in changes in the device health state and bandwidth resource consumption , construct the objective function;

[0032] Assign weight coefficients to each index in the objective function , so that migration latency, health state fluctuations, and resource consumption are optimized in a predetermined ratio;

[0033] According to the remaining available resources of the device Adjust the data migration volume between devices according to the bandwidth constraint, and ensure that the migration process does not exceed the bandwidth limit;

[0034] Solve the optimization problem to obtain the optimal data migration scheduling strategy.

[0035] Preferably, the steps of solving the optimal scheduling problem with constraints by using the Lagrange multiplier method include:

[0036] Construct a Lagrangian function by combining the objective function with the bandwidth constraint and the device health status constraint, which is expressed as:

[0037]

[0038] Where, represents the Lagrangian function, represents the optimization objective function, represents the device Lagrange multiplier of the health status constraint, represents the device at time rate of change of the health status, represents the device drift term of the change in the health status, represents the Lagrange multiplier of the bandwidth constraint between devices, represents the device to the device at time data migration volume, represents the device and the device bandwidth limit between, represents the summation operation for all devices in the system, is the total number of devices in the system;

[0039] Take the partial derivatives of the Lagrangian function with respect to the device health status and the data migration volume respectively;

[0040] Obtain the optimal data migration volume and the device health status evolution path according to the Lagrange equation to meet the system performance and resource constraints;

[0041] Determine the optimal data migration path and scheduling strategy between each device according to the solution results.

[0042] Preferably, the steps of using the stochastic optimal control method include:

[0043] Model the health state of the device by establishing the Hamilton-Jacobi-Bellman equation, considering the stochastic changes in the device health state;

[0044] Define the state variable for the device 's health state, and the control variable is the data migration strategy, with the goal of minimizing the risk of device failure and migration delay;

[0045] Solve the Hamilton-Jacobi-Bellman equation to obtain the optimal control strategy , which is the optimal strategy for data migration between devices;

[0046] Based on the optimal control strategy , dynamically adjust the data migration path between devices to ensure that the system always maintains optimal scheduling during the change of device states.

[0047] Preferably, the step of applying the obtained optimal migration scheduling strategy to the system in real time includes:

[0048] Transmit the optimal migration scheduling strategy to the scheduling execution module to ensure that each device migrates data according to the optimal path;

[0049] Dynamically adjust the scheduling strategy according to the real-time monitored device health state and system load changes;

[0050] During the data migration process, continuously monitor the bandwidth and device resource utilization to ensure that the scheduling strategy always meets the bandwidth limit and device health state constraints during actual operation;

[0051] According to the change of device state, update the scheduling strategy in a timely manner to ensure that the system always operates in an optimal state.

[0052] Preferably, the steps of solving the optimization problem of data migration scheduling using dynamic programming include:

[0053] Based on the optimal control strategy and the scheduling objective function , discretize the scheduling problem into sub-problems at several moments;

[0054] Through the dynamic programming algorithm, gradually calculate the optimal data migration volume between devices at each moment , and consider the constraint conditions of bandwidth and device health state.

[0055] The present invention also provides a data migration scheduling system based on device anomaly detection, including:

[0056] A health state monitoring module for real-time monitoring of the health state of each device and estimating the device health state through stochastic process modeling;

[0057] Anomaly detection module, used to determine whether the device is in an abnormal state according to the estimated value of the health status;

[0058] Scheduling decision-making module, used to generate a data migration scheduling strategy according to the change of the device health status and bandwidth constraints;

[0059] Optimization calculation module, used to calculate the optimal data migration path and scheduling strategy based on the Lagrange multiplier method and the stochastic optimal control method;

[0060] Real-time scheduling execution module, used to adjust the data migration path and scheduling strategy in the system according to the optimization result.

[0061] The present invention provides a data migration scheduling method based on device anomaly detection. It has the following beneficial effects:

[0062] 1. The present invention models and real-time estimates the device health status through the stochastic differential equation and the Kalman filter algorithm, realizes accurate anomaly state detection, and effectively solves the problems of untimely response and insufficient accuracy to the change of the device state in the prior art.

[0063] 2. The present invention constructs a multi-objective optimization function by comprehensively optimizing the migration delay, device load fluctuation and bandwidth resource consumption, and combines the health status and resource constraints, providing an efficient data migration path and scheduling strategy, and solving the problems of low migration efficiency and low resource utilization rate in the prior art.

[0064] 3. The present invention can always maintain the optimality of the migration path when the device health status and system resource conditions change by using the stochastic optimal control method to dynamically adjust the scheduling strategy, and solves the problem of insufficient adaptability of the fixed scheduling strategy to the complex dynamic environment in the prior art.

[0065] 4. The present invention ensures that the system can efficiently complete data migration under the abnormal state of the device through the collaborative work of the health status real-time monitoring, optimization calculation and scheduling execution modules, and solves the problem of lack of closed-loop optimization of the scheduling strategy under the abnormal state in the prior art. Description of the Drawings

[0066] Figure 1 is the method flow chart of the present invention;

[0067] Figure 2 is the system architecture diagram of the present invention. Detailed Embodiment

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0069] Please refer to the attached Figure 1 , the embodiment of the present invention provides a data migration scheduling method based on device anomaly detection, including the following steps:

[0070] S1. Through the health state modeling of the device, monitor and real-time estimate the health state of the device, and determine whether the device is in an abnormal state based on the estimated value;

[0071] S2. When the device is in an abnormal state, trigger data migration scheduling;

[0072] S3. Design an optimized scheduling objective to minimize data migration latency, device load fluctuations, and consumption of bandwidth resources, and calculate the data migration path and scheduling strategy based on the device health state and bandwidth constraints;

[0073] S4. Use the Lagrange multiplier method to solve the optimal scheduling problem with constraints to determine the data migration volume between each device;

[0074] S5. Use the stochastic optimal control method to model the randomness of the device health state, thereby dynamically adjusting the scheduling strategy to ensure the optimal data migration path of the device in normal or abnormal states;

[0075] S6. Apply the obtained optimal migration scheduling strategy to the system in real time.

[0076] For step S1, the embodiment of the present invention provides a data migration scheduling method based on device anomaly detection, which is used to trigger data migration scheduling when a device anomaly is detected, and realizes the dynamic adjustment of the data migration path and strategy through an optimization algorithm to ensure system performance and efficient utilization of resources. Compared with the prior art, the method of the present invention can achieve efficient and reliable data migration management through real-time modeling and estimation of the device health state, combined with the optimized scheduling objective and constraint conditions, and has significant advantages especially in scenarios where the device operating state changes randomly.

[0077] In this embodiment, first, by modeling the health state of the device, a stochastic process is used to describe the dynamic change of the device health state. The health state modeling is based on a stochastic differential equation, specifically expressed as:

[0078]

[0079] Among them, Denotes the device At time Of the health state, Is the drift term, representing the average change trend of the device health state, Is the fluctuation term, reflecting the fluctuation intensity of the health state, Is the standard Brownian motion, representing the random perturbation term. Through this model, the random evolution process of the device health state between normal operation and abnormal state can be characterized.

[0080] Next, obtain the real-time monitoring data of the device, including operating parameters (such as temperature, vibration frequency, current, etc.) and historical data records. Through data preprocessing, the raw data is converted into the initial value of the health state , As the input condition of the stochastic differential equation. It should be noted that the accuracy of the initial value has an important impact on the accuracy of subsequent health state estimation and anomaly detection.

[0081] Specifically, by fusing and calculating the real-time observation data of the device With the health state model, the Kalman filter algorithm is used to recursively estimate the health state. The state equation and observation equation of the Kalman filter are respectively:

[0082]

[0083]

[0084]

[0085]

[0086] Among them, Is the health state of the device at the Th time, Is the corresponding observed value, And Are the process noise and observation noise respectively, assuming that they follow a zero-mean normal distribution.

[0087] Based on the above model and real-time monitoring data, the estimated value of the device health state is obtained. By comparing the estimated value with the preset normal health state threshold, it can be judged whether the device is in an abnormal state. When the estimated value exceeds the threshold, that is, when the condition Is satisfied, the system will immediately trigger an anomaly alarm and enter the data migration scheduling phase.

[0088] In one implementation, the threshold of the health state can be dynamically adjusted according to the type of the device and the operating environment. For example, for a server with high-load tasks, the threshold of its health state may be set lower so as to perform data migration at an early stage of potential failures. For low-priority devices, the threshold can be appropriately increased to reduce unnecessary scheduling frequency.

[0089] It should be noted that, to improve the accuracy of health state estimation, the present invention can introduce a multi-model filtering method. By constructing multiple hypothesis models (such as different health state drift rates and fluctuation intensities), and using a weighted fusion technique to synthesize multiple estimation results, a more robust predicted value of the health state can be generated.

[0090] In an exemplary embodiment, the method of the present invention also supports long-term prediction of the changing trend of the device health state. For example, by using the time series of the health state estimation value and combining it with a time series analysis algorithm, the changing trend of the future health state can be predicted, providing more forward-looking decision-making support for anomaly detection and data migration scheduling.

[0091] The above-mentioned health state modeling and real-time estimation process lay the foundation for data migration scheduling. The core lies in using a stochastic differential equation and a Kalman filtering algorithm to effectively capture the dynamic changes and stochastic characteristics of the device health state, so as to accurately judge the abnormal state of the device.

[0092] For step S2, in the process of data migration scheduling, accurately judging whether the device is in an abnormal state is the key prerequisite for the effectiveness of the scheduling strategy. To achieve this goal, the present invention accurately detects the abnormal state of the device through the modeling and real-time estimation of the device health state, combined with a threshold judgment mechanism, thereby providing a reliable basis for triggering data migration. Specifically, the core of step S2 lies in using a stochastic differential equation and real-time monitoring data to construct a modeling and estimation framework for the device health state

[0093] In this embodiment, the health state modeling process is based on the form of a stochastic differential equation, described as:

[0094]

[0095] where, represents the health state of the device at time , is the drift term, representing the trend change of the health state, is the fluctuation term, indicating the fluctuation intensity of the health state, is a standard Brownian motion, reflecting the impact of random perturbations on the health state. This model can better characterize the state changes of the equipment during operation and is particularly suitable for the non-linear and random characteristics of the equipment health state.

[0096] Specifically, the modeling steps include the following:

[0097] First, obtain the real-time monitoring data of the equipment, such as temperature, power consumption, vibration frequency, current, etc., and preprocess it according to the characteristics of the data, standardize it to a unified range to eliminate the dimensional differences between different physical quantities.

[0098] Next, determine the initial values of the model parameters and according to the historical data and domain knowledge of the equipment. These parameters can be estimated by statistical methods or machine learning algorithms. For example, calculate the drift term using the mean change rate of the historical health state, and estimate the volatility term using the variance of the health state.

[0099] To improve the accuracy of health state estimation, the Kalman filter algorithm is used to update the model in real time. The Kalman filter algorithm includes a state equation and an observation equation, and their specific forms are as follows:

[0100]

[0101]

[0102] where, represents the health state of the equipment at the th moment, represents the observed data, and are the state transition matrix and the observation matrix respectively, and are the process noise and the observation noise, assuming that they follow a zero-mean normal distribution.

[0103] In a possible implementation, by comparing the observed data and the modeled health state , calculate the estimation error in real time and dynamically correct the estimated value of the health state based on the Kalman gain to obtain a more realistic predicted value of the health state .

[0104] It should be noted that the real-time estimation result of the health state is compared with the preset health threshold to determine whether the equipment is in an abnormal state. The specific judgment condition is:

[0105] If , the device is in a normal state;

[0106] If , the device is in an abnormal state.

[0107] The above threshold can be dynamically adjusted according to the importance of the device and the operating environment. For example, for mission-critical devices, the threshold can be set lower to detect abnormal states earlier; for non-critical devices, the threshold can be appropriately relaxed to reduce the false alarm rate.

[0108] As an option, to enhance the robustness of the health state modeling, the present invention also supports multi-model filtering techniques. By simultaneously constructing multiple hypothesis models (e.g., different combinations of drift terms and fluctuation terms) and performing weighted averaging on the prediction results of each model, the accuracy of the estimated value can be improved.

[0109] In an exemplary embodiment, the present invention also provides a prediction function for the health state. By analyzing the health state time series data and using methods such as autoregressive integrated moving average model or long short-term memory network, the future change trend of the device's health state can be predicted. This prediction ability provides important support for triggering data migration in advance and formulating scheduling strategies.

[0110] Through the above health state modeling and estimation methods, the present invention can not only accurately detect the abnormal state of the device, but also capture the random change characteristics of the health state, laying a solid foundation for subsequent optimal scheduling.

[0111] For step S3, it involves designing an optimization scheduling objective to minimize data migration latency, device load fluctuations, and bandwidth resource consumption, and calculating the data migration path and scheduling strategy based on the device health state and bandwidth constraints. This process effectively guides the solution of the optimal scheduling strategy in subsequent steps by reasonably setting the optimization objective function and related constraint conditions.

[0112] In the overall method, the design of the optimization scheduling objective needs to comprehensively consider the balance relationship of multiple performance indicators to ensure that data migration can be efficiently executed under the conditions of meeting the device health state and system resource limitations. This is an important prerequisite for realizing the dynamic optimization of the scheduling strategy.

[0113] In this embodiment, the design of the optimization scheduling objective and related steps are as follows:

[0114] First, determine the form of the objective function to comprehensively reflect the impact of migration latency, device health state fluctuations, and bandwidth resource consumption on system performance. The objective function can be expressed as:

[0115]

[0116] Where: Represents the device and the device The migration delay between them, Indicates the impact of fluctuations in the device health status on migration, Indicates the consumption of bandwidth resources between devices, Are the weight coefficients for migration delay, health status fluctuations, and bandwidth resource consumption, respectively.

[0117] As an option, the system can dynamically adjust the weight coefficients according to actual requirements .

[0118] Specifically, when designing the objective function, the following constraint conditions need to be considered:

[0119] Device health status constraint: To ensure that the device does not fail due to the deterioration of the health status during migration, the change in the health status of the device needs to meet certain conditions. Modeling the health status through a stochastic differential equation, its constraint can be expressed as:

[0120]

[0121] Represents the device At time .

[0122] Bandwidth resource constraint: The amount of data migration shall not exceed the available bandwidth limit between devices, and this constraint is expressed as:

[0123]

[0124] Among them, Limit.

[0125] Device carrying capacity constraint: The amount of migrated data cannot exceed the storage and processing capabilities of the target device, and this constraint is expressed as:

[0126]

[0127] Among them, Is the device The maximum carrying capacity.

[0128] In a possible implementation, by comprehensively optimizing the objective function and constraint conditions, the data migration path and scheduling strategy between devices can be determined. For example, an optimization algorithm based on the gradient descent method can be used to iteratively solve the objective function, and the amount of data migration between devices is updated in each iteration

[0129] It should be noted that, in order to improve the efficiency and stability of the optimization process, heuristic algorithms such as particle swarm optimization or simulated annealing can also be combined to avoid being trapped in local optima while searching for the global optimal solution.

[0130] In some embodiments, the part of the device health state fluctuation in the objective function can be modeled in detail. For example, the fluctuation of the device health state is regarded as a dynamic time series, and a prediction model is constructed using historical data, so as to predict potential health state changes in advance and make corresponding adjustments during the optimization process. This method can further improve the accuracy of the scheduling strategy.

[0131] The design of the optimized scheduling objective is the core part of the data migration scheduling method, and its rationality and integrity directly determine the quality of the scheduling strategy. By comprehensively considering factors such as data migration delay, device health state fluctuation, and bandwidth resource consumption, and performing optimization calculations under constraint conditions, it can ensure that the system always maintains an efficient and reliable operating state under different operating conditions. At the same time, this design provides clear theoretical guidance for subsequent optimization algorithms.

[0132] For step S4, in order to accurately solve the balance problem between system constraints and objectives in the optimized scheduling, the present invention proposes an optimization solution step based on the Lagrange multiplier method, which is applicable to solving non-linear optimal scheduling problems with constraints. This step combines various constraint conditions with the objective function by constructing a Lagrangian function, theoretically ensuring the global optimality and feasibility of the scheduling scheme.

[0133] In this embodiment, first, the Lagrangian function is constructed through the following steps. This function comprehensively considers multiple factors such as data migration delay, device health state constraints, and bandwidth limitations. The form of the Lagrangian function is expressed as:

[0134]

[0135] Where:

[0136] represents the Lagrangian function, is the optimization objective function, specifically designed to minimize data migration delay, device load fluctuation, and bandwidth consumption, is the equality constraint related to the device health state, ensuring that the device operating state is within a safe range, is the inequality expression of the bandwidth resource constraint, and are the Lagrange multipliers of the health state and bandwidth limitation respectively, and represent the number of constraints respectively.

[0137] In the above formula, the change in the device health state is modeled by a stochastic differential equation, specifically:

[0138]

[0139] .

[0140] It should be noted that the bandwidth limit between devices is described in the form of inequality constraints:

[0141]

[0142] where , .

[0143] Next, by taking the partial derivatives of the Lagrangian function with respect to the decision variables and the Lagrange multipliers respectively, the following necessary conditions are obtained:

[0144]

[0145] These conditions ensure that the optimization result reaches the optimal solution under all constraint conditions.

[0146] Specifically, in one implementation, the optimization problem of the Lagrangian function can be solved by an iterative algorithm. Initially, the Lagrange multiplier is set to zero and adjusted step by step so that the amount of data migration between devices meets the bandwidth limit while ensuring the health state constraint . To improve the computational efficiency, the gradient descent method can be combined to dynamically adjust the Lagrange multiplier.

[0147] As an option, to further ensure the global optimality of the solution, the Lagrangian solution can be combined with the stochastic optimal control method. For example, when the randomness of the device health state is strong, the influence of random perturbations is modeled by the optimal control method, which can improve the robustness of the system.

[0148] It should be noted that the evaluation result of the device health state is directly related to the allocation of bandwidth resources. Therefore, during the calculation process, the weight coefficient can be dynamically adjusted to make the balance between the health state fluctuation and the bandwidth resource consumption more in line with the actual needs. For example, when the device state is close to abnormal, the weight of the health state constraint can be increased to give priority to ensuring the safe operation of the device.

[0149] In summary, by introducing the Lagrange multiplier method, not only the effective combination of multiple constraint conditions is realized, but also the optimality and stability of the system resource utilization are ensured, laying a theoretical foundation for the dynamic adjustment of the subsequent scheduling strategy.

[0150] For step S5, to further optimize the data migration scheduling in the device abnormal state, the present invention proposes a method based on stochastic optimal control, aiming to dynamically adjust the scheduling strategy by modeling the randomness of the device health state, so as to ensure the optimality of the data migration path in both normal and abnormal device states. This method realizes global optimization by introducing the Hamilton-Jacobi-Bellman equation and combining the correlation between the device health state and the migration strategy.

[0151] In this embodiment, first, a stochastic model of the device health state is established, and the dynamic change of the health state is described by the following stochastic differential equation:

[0152]

[0153] where is the device health state, represents the drift term of the health state, describing the deterministic change trend of the health state, is the fluctuation term, describing the randomness of the health state, is the standard Brownian motion.

[0154] Based on the model, the data migration problem is regarded as an optimal control problem with stochastic constraints. The objective function is defined as:

[0155]

[0156] where , is the operating cost function, including migration delay, device health state fluctuation, and bandwidth consumption, is the penalty coefficient of the migration strategy, used to balance the magnitude of the migration volume, is the terminal cost function, describing the performance of the device in the final state.

[0157] It should be noted that by minimizing the objective function , the migration delay can be effectively reduced, and at the same time, the device health state fluctuation can be suppressed.

[0158] In a possible implementation, the optimal control strategy is obtained by solving the Hamilton-Jacobi-Bellman equation. The form of the Hamilton-Jacobi-Bellman equation is:

[0159]

[0160] where is the value function, indicating the system in the state .

[0161] As an option, a numerical iteration method can be used to solve the above Hamilton-Jacobi-Bellman equation. Initially, based on the current observation of the device health status .

[0162] Specifically, in some embodiments, the migration path between devices can be adjusted in real time according to the optimal policy. For example, when the device health status fluctuates greatly, the migration volume will be dynamically reduced, thereby reducing the pressure on bandwidth resources. At the same time, for devices whose health status is close to abnormal, the migration volume can be preferentially increased to ensure the safe transfer of data.

[0163] It should be noted that in practical applications, the adjustment of the migration policy needs to comprehensively consider the system bandwidth limit and the load distribution of devices. For example, in the scenario of data migration between multiple devices, a priority mechanism can be introduced to allocate bandwidth resources according to the degree of deterioration of the health status to maximize the overall efficiency of the system.

[0164] Through the above method, this embodiment successfully solves the problem that traditional methods cannot handle random perturbations and non-linear constraints. This solution can not only ensure the global optimality of data migration scheduling but also has strong dynamic adaptability, providing an effective solution for data migration scheduling in the case of device abnormal states.

[0165] For step S6, in order to ensure the effectiveness and real-time performance of data migration scheduling in the case of device abnormal states, the present invention further proposes a method for applying the optimal migration scheduling policy to the system in real time. The core of this method lies in ensuring that the system can adapt to the dynamic changes of device health status and system resources through real-time policy execution and monitoring, while effectively achieving the migration scheduling goal. Specifically, through the scheduling execution module, combined with the health status monitoring and feedback mechanism, the theoretically optimal policy is transformed into actual scheduling operations.

[0166] In this embodiment, the optimal migration scheduling policy is first transmitted to the scheduling execution module in the system. During this process, the scheduling execution module will parse the scheduling policy generated by the optimization calculation module and convert it into executable instructions. As an option, these instructions can be transmitted to the corresponding device end nodes through the network to guide the devices to perform data transfer operations according to the preset optimal migration path.

[0167] Specifically, the execution of the scheduling policy requires real-time collection of the health status, load distribution, and bandwidth utilization of devices in the system. For example:

[0168] The health status monitoring module will continuously monitor the health status of the devices and feedback the real-time data to the scheduling decision module.

[0169] After receiving the instruction, the scheduling execution module will dynamically adjust the execution order and migration volume according to the current health status and load changes of the device.

[0170] In one implementation, the dynamic adjustment mechanism of the scheduling policy includes the following steps:

[0171] First, based on the health status data fed back by the health status monitoring module, evaluate whether the device is still in an abnormal state.

[0172] Second, when significant changes occur in the device load or bandwidth resources, the scheduling execution module will re-evaluate the feasibility of the migration path. For example, when the bandwidth utilization rate of a certain path is approaching saturation, the migration traffic can be reallocated to other available paths.

[0173] It should be noted that during the execution process, the scheduling execution module is not only responsible for the real-time operation of data migration, but also needs to continuously monitor the possible exceptions during the execution process. For example:

[0174] When the bandwidth resources are tight, the module will reduce the migration rate to avoid large fluctuations in system performance.

[0175] When the health status of the target device deteriorates, the current migration operation can be immediately interrupted and the key data can be preferentially protected instead.

[0176] As an extension, in one embodiment, the execution of the scheduling policy can be further optimized by combining intelligent algorithms. For example, the dynamic change trend of the device health status can be predicted based on a deep reinforcement learning model, so as to adjust the migration policy in advance. In this case, the scheduling execution module is not only a passive execution tool, but can also predict potential risks through a machine learning model and actively optimize the execution policy.

[0177] To ensure that the scheduling policy is always in an optimal state, the system also designs a closed-loop feedback mechanism. During the data migration process, the scheduling execution module will real-time feedback the latest data of the bandwidth resource utilization rate and the device health status to the optimization calculation module. The optimization calculation module updates the Lagrange multiplier or the random control policy parameters accordingly, so that the scheduling decision of the system can adapt to dynamic changes.

[0178] The advantage of this method is that it realizes the seamless connection from theoretical optimization to practical application. Through real-time policy execution and dynamic adjustment, it not only ensures the system performance during the data migration process, but also greatly improves the system's ability to handle abnormal states. This method is particularly suitable for scenarios with large-scale device networks and complex bandwidth resource constraints, providing a reliable and efficient solution for data migration scheduling under abnormal device states.

[0179] Please refer to the appendix Figure 2, the present invention also provides a data migration scheduling system based on device anomaly detection, including:

[0180] A health status monitoring module, which is used to monitor the health status of each device in real time and estimate the device health status through stochastic process modeling;

[0181] An anomaly detection module, which is used to determine whether the device is in an abnormal state according to the health status estimation value;

[0182] A scheduling decision module, which is used to generate a data migration scheduling strategy according to the change of device health status and bandwidth constraints;

[0183] An optimization calculation module, which is used to calculate the optimal data migration path and scheduling strategy based on the Lagrange multiplier method and the stochastic optimal control method;

[0184] A real-time scheduling execution module, which is used to adjust the data migration path and scheduling strategy in the system according to the optimization result.

[0185] The health status monitoring module, the main function of this module is to monitor and estimate the health status of the device in real time through stochastic process modeling, and provide basic data support for anomaly detection and the formulation of scheduling strategies.

[0186] The specific functions include:

[0187] Collect real-time data of device operation, such as status information such as temperature, workload, and running time.

[0188] Model the evolution of the device health status through stochastic processes, for example, a model based on stochastic differential equations, to describe the drift term, fluctuation term, and stochastic perturbation of the health status.

[0189] Calculate and provide the health status value of the device at the current moment as the basis for subsequent determination of whether it is abnormal.

[0190] Ensure the accuracy and reliability of the monitoring data, and provide real-time health status input values for the anomaly detection module.

[0191] The anomaly detection module, which is used to determine whether the device is in an abnormal state based on the health status estimation value, so as to trigger data migration scheduling in a timely manner in case of anomalies.

[0192] The specific functions include:

[0193] Compare the device health status estimation value with the set threshold range to distinguish the normal and abnormal states of the device.

[0194] When the health status exceeds the threshold, generate an anomaly alarm signal and trigger the start of the data migration scheduling strategy.

[0195] Provide clear information about abnormal devices, serving as input for the goal setting of the scheduling decision-making module.

[0196] Ensure that the judgment of the abnormal state has high precision and real-time performance, avoiding false alarms or missed alarms.

[0197] Scheduling decision-making module, which generates specific data migration scheduling strategies based on changes in device health status and bandwidth constraints, ensuring the reasonable allocation and efficient utilization of system resources.

[0198] Specific functions include:

[0199] Comprehensively consider the health status information and bandwidth constraint conditions, and set the target devices and migration paths for data migration.

[0200] Based on the real-time health status and network resource conditions, formulate preliminary data migration amounts and scheduling strategies.

[0201] Clarify the allocation of migration data and transmission paths between devices, providing a feasible scheduling framework for the optimization calculation module.

[0202] Ensure that the generated scheduling strategies take into account minimizing latency, load balancing, and resource consumption control.

[0203] Optimization calculation module, which calculates the optimal data migration paths and scheduling strategies through mathematical optimization methods (such as Lagrange multiplier method and stochastic optimal control method), ensuring the global performance optimization of the system under constraint conditions.

[0204] Specific functions include:

[0205] Construct the objective function, combining data migration latency, fluctuations in device health status changes, and bandwidth resource consumption metrics.

[0206] Combine the optimization objective function with bandwidth constraints and device health status constraints to construct the Lagrangian function and solve for the optimal migration amount.

[0207] Through the stochastic optimal control method, dynamically respond to the randomness of changes in device health status, and optimize data migration paths and strategies.

[0208] Output the optimal data migration amounts and scheduling strategies between devices, providing precise guidance for the real-time scheduling execution module.

[0209] Real-time scheduling execution module, which applies the scheduling strategies output by the optimization calculation module to the system, dynamically adjusts the data migration paths and strategies between devices, and ensures the efficient and smooth progress of the entire scheduling process.

[0210] Specific functions include:

[0211] Receive the optimization calculation results and distribute the optimal migration strategy to specific devices.

[0212] Dynamically adjust the migration path and the migration volume according to the real-time monitored device health status and system load changes.

[0213] Continuously monitor the bandwidth utilization and device resource occupancy during the migration process to ensure that the scheduling policy meets various constraints during actual operation.

[0214] Respond in a timely manner to changes in the device health status or sudden anomalies, iteratively update the policy, and maintain the optimal state of the overall system performance.

[0215] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A data migration scheduling method based on device anomaly detection, characterized in that It includes the following steps: S1. Through the health state modeling of the device, monitor and real-time estimate the health state of the device, and judge whether the device is in an abnormal state based on the estimated value; S2. When the device is in an abnormal state, trigger data migration scheduling; S3. Design an optimized scheduling objective to minimize data migration latency, device load fluctuation, and bandwidth resource consumption, and calculate the data migration path and scheduling strategy based on the device health state and bandwidth constraints; S4. Use the Lagrange multiplier method to solve the optimal scheduling problem with constraints to determine the data migration volume between each device; S5. Use the stochastic optimal control method to model the randomness of the device health state, thereby dynamically adjusting the scheduling strategy to ensure the optimal data migration path of the device in normal or abnormal states; S6. Apply the obtained optimal migration scheduling strategy to the system in real time; The health state modeling step includes: Obtain various monitoring data of the device and convert them into the initial value of the device health state; Use the stochastic differential equation to model the change of the device health state, expressed as: dx i (t) = μ i (t)dt + σ i (t)dW i (t) where x i (t) is the health state of device i at time t, μ i (t) is the drift term, σ i (t) is the diffusion term, dW i (t) is the stochastic perturbation; by comparing the observed data z i (t) with the health state model, a real-time estimated value of the device health state is obtained; Compare the estimated value with the preset threshold to judge whether the device is in a normal state or an abnormal state, and provide a basis for subsequent anomaly detection and data migration decision-making; The step of the optimized scheduling objective includes: Determine the objective function J of data migration, where J includes minimizing data migration latency, device health state fluctuation, and system resource consumption; Set weight coefficients α1, α2, α3 to balance the relative importance of migration latency, health state fluctuations, and bandwidth resource consumption; by calculating the data migration volume d ij (t) between each device, combined with the maximum carrying capacity C i of the device and the remaining available resource r i (t), optimize the objective function; Under the bandwidth constraint, calculate the optimal data migration path between devices to meet the system performance optimization objective.

2. The data migration scheduling method based on device anomaly detection according to claim 1, wherein The device health state estimation step includes: Obtain the observed data z of the device i (t) and assume it has a linear relationship with the device health state x i (t); Estimate the health state of the device using the Kalman filtering algorithm, and obtain the estimated health state of the device at each moment t through recursive calculation Compare the estimated health status with a preset normal health status threshold θ to determine whether the device is in an abnormal state; when the device health status exceeds the threshold θ, trigger data migration scheduling.

3. The data migration scheduling method based on device anomaly detection according to claim 1, wherein The step of optimizing the scheduling objective function includes: By minimizing the data migration latency d ij (t) 2 / C i , the fluctuations in the changes of the device health status and the bandwidth resource consumption d ij (t) / r i (t), an objective function is constructed; Assign weight coefficients α1, α2, α3 to each index in the objective function, so that the migration latency, health state fluctuation, and resource consumption are optimized in a predetermined proportion; According to the remaining available resources r i (t) of the device and the bandwidth constraint, adjust the data migration volume d ij (t) between devices, and ensure that the migration process does not exceed the bandwidth limit; Solve the optimization problem to obtain the optimal data migration scheduling strategy.

4. The data migration scheduling method based on device anomaly detection according to claim 1, wherein The step of using the Lagrange multiplier method to solve the optimal scheduling problem with constraints includes: Construct the Lagrangian function, combine the objective function with the bandwidth constraint and the device health state constraint, expressed as: Among them, represents the Lagrangian function, J represents the optimization objective function, and λ i (t) represents the Lagrange multiplier for the health state constraint of device i, represents the change rate of the health state of device i at time t, and μ i (t) represents the drift term of the change in the health state of device i, and γ ij (t) represents the Lagrange multiplier for the bandwidth constraint between devices, and d ij (t) represents the data migration volume from device i to device j at time t, and B ij represents the bandwidth limit between device i and device j, represents the summation operation over all devices i in the system, and N is the total number of devices in the system; Take the partial derivatives of the Lagrangian function with respect to the device health state \(x\) i (t) and the data migration volume \(d\) ij (t) respectively; Obtain the optimal data migration volume d according to Lagrange's equation ij (t) and the evolution path x of the device health state i (t) to meet the system performance and resource constraints; According to the solution result, determine the optimal data migration path and scheduling strategy between each device.

5. The data migration scheduling method based on device anomaly detection according to claim 1, wherein The step of using the stochastic optimal control method includes: Model the device health state by establishing the Hamilton-Jacobi-Bellman equation, considering the random change of the device health state; Define the state variable x i (t) is the health state of device i, and the control variable u i (t) is the data migration strategy, and the goal is to minimize the risk of device failure and migration latency; Solve the Hamilton-Jacobi-Bellman equation to obtain the optimal control strategy That is, the optimal strategy for data migration between devices; based on the optimal control strategy Dynamically adjust the data migration path between devices to ensure that the system always maintains optimal scheduling during the change of device states.

6. The data migration scheduling method based on device anomaly detection according to claim 1, wherein The step of applying the obtained optimal migration scheduling strategy to the system in real time includes: Transfer the optimal migration scheduling strategy to the scheduling execution module to ensure that each device migrates data along the optimal path; Dynamically adjust the scheduling strategy based on the real-time monitored device health state and system load change; During the data migration process, continuously monitor the bandwidth and device resource utilization rate to ensure that the scheduling strategy always meets the bandwidth limit and device health state constraint during actual operation; According to the change of the device state, update the scheduling strategy in time to ensure that the system is always in the optimal operating state.

7. The data migration scheduling method based on device anomaly detection according to claim 1, wherein The step of using dynamic programming to solve the optimization problem of data migration scheduling includes: Based on the optimal control strategy and the scheduling objective function J, discretize the scheduling problem into sub-problems at several moments; Through the dynamic programming algorithm, gradually calculate the optimal data migration volume d ij (t) between devices at each moment, and consider the constraint conditions of bandwidth and device health status.

8. A data migration scheduling system based on device anomaly detection, characterized in that, It includes: A health status monitoring module, which is used to monitor the health status of each device in real time and estimate the device health status through stochastic process modeling; An anomaly detection module, which is used to judge whether the device is in an abnormal state according to the health status estimation value; A scheduling decision-making module, which is used to generate a data migration scheduling strategy according to the change of device health status and bandwidth constraints; An optimization calculation module, which is used to calculate the optimal data migration path and scheduling strategy based on the Lagrange multiplier method and the stochastic optimal control method; A real-time scheduling execution module, which is used to adjust the data migration path and scheduling strategy in the system according to the optimization result; The data migration scheduling system based on device anomaly detection is used for a data migration scheduling method based on device anomaly detection. The method comprises the following steps: S1. Through the health status modeling of the device, monitor and estimate the health status of the device in real time, and judge whether the device is in an abnormal state based on the estimation value; S2. When the device is in an abnormal state, trigger data migration scheduling; S3. Design an optimized scheduling objective to minimize data migration delay, device load fluctuation and consumption of bandwidth resources, and calculate the data migration path and scheduling strategy based on the device health status and bandwidth constraints; S4. Use the Lagrange multiplier method to solve the optimal scheduling problem with constraints to determine the data migration amount between each device; S5. Use the stochastic optimal control method to model the randomness of the device health status, so as to dynamically adjust the scheduling strategy to ensure the optimal data migration path of the device in normal or abnormal states; S6. Apply the obtained optimal migration scheduling strategy to the system in real time; The health status modeling step includes: Obtain various monitoring data of the device and convert them into the initial value of the device health status; Use a stochastic differential equation to model the change of the device health status, expressed as: dx i d(t) = μ i d(t)dt + σ i d(t)dW i d(t) where x i (t) is the health state of device i at time t, μ i (t) is the drift term, σ i (t) is the volatility term, dW i (t) is the random perturbation; by comparing the observed data z i (t) with the health state model, the real-time estimated value of the device health state is obtained; Compare the estimation value with a preset threshold to judge whether the device is in a normal state or an abnormal state, and provide a basis for subsequent anomaly detection and data migration decision-making; The step of the optimized scheduling objective includes: Determine the objective function J of data migration, where J includes minimizing data migration delay, device health status fluctuation and system resource consumption; Set weight coefficients α1, α2, α3 to balance the relative importance of migration latency, health state fluctuations, and bandwidth resource consumption; by calculating the data migration volume d ij (t) between each device, combined with the maximum carrying capacity C i of the device and the remaining available resource r i (t), optimize the objective function; Under the bandwidth constraint, calculate the optimal data migration path between devices to meet the system performance optimization objective.

Images