Data migration method and device, electronic equipment and storage medium

Abstract

The invention discloses a data migration method and device, electronic equipment and a storage medium, and relates to the technical field of computer storage, the data migration method comprises the steps that N sets of target state data are obtained, the ith set of target state data in the N sets of target state data is target state data corresponding to the ith storage equipment in N pieces of storage equipment, and the ith set of target state data is the target state data corresponding to the ith storage equipment in the N pieces of storage equipment; the target state data is used for describing storage attributes of the storage device; a target migration strategy is determined according to the N sets of target state data, the target migration strategy is used for indicating that M pieces of data to be migrated are migrated to the N storage devices, and each storage device in the N storage devices at least stores one piece of data to be migrated; and migrating the M pieces of data to be migrated to the N pieces of storage equipment according to the target migration strategy. Therefore, the problem of low data migration efficiency caused by manual selection of the storage device can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer storage, and in particular to a data migration method and device, electronic equipment and storage medium. BACKGROUND

[0002] Distributed storage systems play a crucial role in modern information technology fields such as big data, cloud computing, and artificial intelligence. By distributing data across multiple storage devices, such systems not only provide massive data storage capabilities, but also ensure high availability and fault tolerance of data. With the continuous expansion of business and the explosive growth of data volume, regular expansion of storage clusters has become an essential operation and maintenance activity. Expansion not only increases the overall capacity of the storage system, but also optimizes data distribution structures, improves system performance and efficiency, and enhances system scalability and disaster recovery capabilities.

[0003] However, in the expansion scenario of traditional distributed storage systems, the object storage device (OSD) for data migration is usually selected manually. Improper selection will lead to a continuous rise in water level, triggering a backfill_too_full error. After the error occurs, only post-remedial measures such as a warning list can be relied on, lacking risk prediction capabilities. During the data migration process, the migration strategy needs to be manually adjusted by the operation and maintenance personnel, which is inefficient.

[0004] Therefore, the data migration method in the related art has the problem of low data migration efficiency due to manual selection of storage devices. SUMMARY

[0005] The present application provides a data migration method, device, electronic equipment and storage medium to at least solve the problem of low data migration efficiency due to manual selection of storage devices.

[0006] The present application provides a data migration method, comprising: obtaining N groups of target state data, wherein the i-th group of target state data in the N groups of target state data is the target state data corresponding to the i-th storage device in the N storage devices, and the target state data is used to describe the storage attribute of the storage device; N is a positive integer, and i is a positive integer less than or equal to N; determining a target migration strategy according to the N groups of target state data, wherein the target migration strategy is used to indicate that M pieces of to-be-migrated data are migrated to the N storage devices, each storage device in the N storage devices stores at least one to-be-migrated data, and M is a positive integer; and migrating the M pieces of to-be-migrated data to the N storage devices according to the target migration strategy.

[0007] The application further provides a data migration device, comprising: an acquisition module, configured to acquire N groups of target state data, wherein the i-th group of target state data in the N groups of target state data is target state data corresponding to an i-th storage device in N storage devices, and the target state data is used to describe a storage attribute of the storage device; N is a positive integer, and i is a positive integer less than or equal to N; a determination module, configured to determine a target migration strategy according to the N groups of target state data, wherein the target migration strategy is used to indicate that M pieces of to-be-migrated data are migrated to the N storage devices, and each of the N storage devices stores at least one piece of to-be-migrated data; and a migration module, configured to migrate the M pieces of to-be-migrated data to the N storage devices according to the target migration strategy.

[0008] The application further provides an electronic device, comprising: a memory, configured to store a computer program; and a processor, configured to implement steps of the data migration method when executing the computer program.

[0009] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement steps of the data migration method.

[0010] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement steps of the data migration method.

[0011] According to the application, target state data of multiple storage devices is acquired, and then a migration strategy is formulated according to the acquired multiple target state data, so as to determine how to migrate multiple to-be-migrated data to multiple storage devices, and then the multiple to-be-migrated data is migrated to the multiple storage devices according to the migration strategy, thereby avoiding manual selection of storage devices and formulation of a migration strategy, and thus the problem of low data migration efficiency caused by manual selection of storage devices can be solved, and the technical effects of improving data migration efficiency and optimizing data storage performance are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 An application scenario diagram of the data migration method provided by the embodiments of the application is shown.

[0014] Figure 2A flowchart of an optional data migration method according to an embodiment of the present application;

[0015] Figure 3 A schematic diagram of an overall architecture of an optional data migration method according to an embodiment of the present application;

[0016] Figure 4 A schematic diagram of component interaction of an optional data migration according to an embodiment of the present application;

[0017] Figure 5 A flowchart of another optional data migration method according to an embodiment of the present application;

[0018] Figure 6 A schematic diagram of a processing procedure of an optional data migration exception according to an embodiment of the present application;

[0019] Figure 7 A timing diagram of an optional data migration method according to an embodiment of the present application;

[0020] Figure 8 A structural block diagram of an optional data migration apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, any other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0022] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0023] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0024] According to an aspect of an embodiment of the present application, a data migration method is provided. Optionally, in the present embodiment, the above-mentioned data migration method can be applied to, for example, Figure 1The hardware environment shown by the terminal device 102 and the server 104. As shown in Figure 1 The server 104 is connected with the terminal device 102 through the network, and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal. A database can be set on the server or independently of the server, and is used to provide data storage services for the server 104.

[0025] The network can include but is not limited to at least one of the following: wired network, wireless network. The wired network can include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network. The wireless network can include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth.

[0026] The data migration method of the embodiment of the application can be executed by the server 104, or by the terminal device 102, or by the server 104 and the terminal device 102 together. Wherein, the terminal device 102 executes the data migration method of the embodiment of the application can also be executed by the client installed thereon.

[0027] Taking the data migration method executed by the server 104 in the embodiment as an example, Figure 2 is a flowchart of an optional data interaction method according to the embodiment of the application, as shown in Figure 2 The flow of the method includes the following steps:

[0028] Step S202: obtaining N groups of target state data, wherein the i-th group of target state data in the N groups of target state data is the target state data corresponding to the i-th storage device in the N storage devices, and the target state data is used to describe the storage attribute of the storage device; N is a positive integer, and i is a positive integer less than or equal to N;

[0029] Optionally, the storage device is an OSD, and the N storage devices include a storage device that has stored data and a newly expanded storage device (the amount of data stored in the newly expanded storage device is 0).

[0030] It should be noted that through step S202, the health status and resource usage of each storage device in the storage cluster can be grasped in real time, thereby providing data support for subsequent intelligent data migration strategy generation.

[0031] Step S204: determining a target migration strategy according to the N groups of target state data, wherein the target migration strategy is used to indicate that M pieces of to-be-migrated data are migrated to the N storage devices, and each storage device in the N storage devices stores at least one piece of to-be-migrated data, and M is a positive integer;

[0032] Optionally, the M pieces of data to be migrated are stored in N-P storage devices before migration, that is, the M pieces of data to be migrated are data in a data set that needs to be redistributed in the distributed storage system before expansion, wherein P is the number of storage devices with a stored capacity of 0, and P is an integer greater than or equal to 1 and less than N.

[0033] Optionally, the target migration strategy can indicate that data with a larger size is stored in a storage device with a smaller stored capacity, for example, indicating that data A and data B are stored in the second storage device of the N storage devices, and data C and data D are stored in the third storage device of the N storage devices, wherein the stored capacity of the second storage device is greater than that of the third storage device, and the sum of the data sizes of the data A and the data B is less than the sum of the data sizes of the data C and the data D.

[0034] Optionally, the generation of the target migration strategy will comprehensively consider the stored capacity of the storage device, the topology based on the CRUSH algorithm, the priority of the business type, and the resource allocation state, aiming to optimize the data migration and ensure that the further load of the high-water storage device is avoided or minimized during the data migration process, while maximizing the utilization of the newly expanded storage device.

[0035] It should be noted that this step can predict and avoid the high-water deadlock risk in the expansion process through an intelligent decision mechanism, dynamically optimize the data migration path, improve the expansion efficiency and stability of the storage system, reduce manual intervention, and make the resource allocation more balanced and reasonable.

[0036] Step S206: Migrate the M pieces of data to be migrated to the N storage devices according to the target migration strategy.

[0037] It should be noted that this step uses automatic migration control to ensure the efficiency of data migration and the balance of data distribution, avoids business interruption and data inconsistency problems during the expansion process, improves the overall performance and resource utilization of the storage cluster, and provides a solid foundation for the continuous operation of the business.

[0038] It should be noted that the above steps realize the automation of data migration in the storage cluster expansion process. From real-time acquisition of state data, to dynamic generation of strategies, to intelligent execution of data migration, the entire process is closely connected, forming a closed-loop optimization mechanism. Not only does it solve the key pain points in the traditional expansion process, such as high water level deadlock risk, uneven resource allocation, and uncontrollable business impact, but it also improves the overall operational efficiency of the system and the intelligent level of data management. For Ceph distributed storage systems and other distributed storage systems based on the CRUSH algorithm, it provides a more efficient and secure expansion solution, bringing significant technological progress to large-scale data storage and management.

[0039] The above steps obtain target state data of a plurality of storage devices, thereby formulating a migration strategy based on the obtained plurality of target state data to determine how to migrate a plurality of data to be migrated to the plurality of storage devices, and then migrating the plurality of data to be migrated to the plurality of storage devices according to the migration strategy. This avoids manual selection of storage devices and formulation of migration strategies, thereby solving the problem of low data migration efficiency caused by manual selection of storage devices, achieving the technical effects of improving data migration efficiency and optimizing data storage performance.

[0040] In an exemplary embodiment, according to the N groups of target state data, a target migration strategy is determined, which can be achieved by the following steps S11-S12:

[0041] Step S11: Based on the N groups of target state data, a target evaluation model is used to obtain N water level risk levels, wherein the i-th water level risk level in the N water level risk levels is the water level risk level corresponding to the i-th storage device in the N storage devices, and the water level risk level is used to indicate the storage efficiency of the corresponding storage device.

[0042] Optionally, the target state data includes but is not limited to stored capacity, storage speed, disk read-write pressure, and network status.

[0043] Optionally, the target evaluation model can be a Long Short-Term Memory (LSTM) model, which can determine the storage efficiency of the future N storage devices for data storage based on the N groups of target state data in the past 10 cycles.

[0044] Optionally, the water level risk level includes safe, warning, and dangerous.

[0045] Step S12: According to the N water level risk levels, determine the target migration strategy based on the migration strategy rule table, wherein the migration strategy rule table records the corresponding relationship between the target migration strategy and the water level risk level.

[0046] Optionally, by querying the migration strategy rule table, the migration strategy that the storage device should adopt for different risk levels is determined, and the overall target migration strategy is formulated.

[0047] Optionally, the migration strategy rule table records the corresponding relationship between the target migration strategy and the water level risk level, for example:

[0048] If the proportion of storage devices with a safe water level risk level in the N storage devices is greater than 80%, the target migration strategy is to migrate data at a normal migration rate, fully utilize the network bandwidth (without limiting the network bandwidth), and preferentially consider storage devices with larger stored capacity for storage;

[0049] Or, if the proportion of storage devices with a safe water level risk level in the N storage devices is greater than 50% and less than or equal to 80%, the target migration strategy is to migrate data at a limited migration rate, limit the size of the network bandwidth, and preferentially consider storage devices with smaller stored capacity for storage;

[0050] Or, if the proportion of storage devices with a dangerous water level risk level in the N storage devices is greater than 70%, the target migration strategy is to migrate data at an extremely low migration rate, greatly limit the network bandwidth, and preferentially consider storage devices with a stored amount of 0 for storage, wherein the limitation of migration rate and network bandwidth can be reduced when migrating data to storage devices with a stored amount of 0.

[0051] Or, if the proportion of storage devices with a dangerous water level risk level in the N storage devices is greater than 90%, the target migration strategy is to suspend data migration and recalculate the migration path of the storage device.

[0052] It should be noted that by dynamically adjusting the migration strategy, the high water level deadlock risk in the data migration process can be effectively prevented, the need for manual intervention can be reduced, resource allocation can be optimized, and the utilization rate of newly expanded storage devices can be avoided while the high water level storage devices are continuously under pressure. At the same time, by intelligently controlling the migration rate, the interference of data migration on business operation can be minimized, the stability and business continuity of the storage cluster during expansion and data migration can be ensured, and the overall performance and user experience of the storage system are improved.

[0053] It should be noted that the above steps, through intelligent prediction and dynamic adjustment of migration strategy, realize the automation and optimization of data migration in the storage cluster expansion process. This method not only effectively avoids the common high water level risk in the expansion process, but also intelligently adjusts the path and rate of data migration based on the type of business and performance requirements, reduces the negative impact on business, and improves the efficiency of resource allocation and the overall performance of the storage system. For distributed storage systems, especially large-scale storage clusters such as Ceph, the present embodiment provides a more efficient and intelligent expansion solution, enhancing the stability of the storage system and the intelligent level of data management.

[0054] In an exemplary embodiment, according to the N sets of target state data, based on the target evaluation model, N water level risk levels can be obtained by the following steps S21-S23:

[0055] Step S21: According to the N sets of target state data, based on the target evaluation model, the first weight corresponding to the stored capacity, the second weight corresponding to the storage speed and the third weight corresponding to the disk read-write pressure are obtained, wherein the target state data includes: stored capacity, storage speed, disk read-write pressure, and the sum of the first weight, second weight and third weight is 1;

[0056] Optionally, the target evaluation model will dynamically adjust the weight, which can automatically identify which factors are the key factors leading to water level risk according to the current data storage and migration environment, for example, in the context of high storage utilization, the weight of the stored capacity (first weight) may be increased to emphasize the impact of storage space shortage on risk.

[0057] It should be noted that through the dynamic weight mechanism, the target evaluation model can more accurately reflect the current health status and risk level of different storage devices, providing a more detailed and scientific basis for subsequent water level risk level evaluation, thereby enhancing the accuracy and reliability of risk prediction.

[0058] Step S22: According to N stored capacities and the first weight, N storage speeds and the second weight, N disk read-write pressures and the third weight, N target scores are obtained, wherein the i-th target score in the N target scores is the target score corresponding to the i-th storage device in the N storage devices;

[0059] Optionally, the stored capacity, storage speed, and disk read-write pressure of each storage device in the N storage devices are multiplied by the respective corresponding weight, and then summed to obtain the target score of each storage device. The target score here is actually a quantitative indicator that combines the three key factors of storage capacity, migration speed and read-write pressure, which can comprehensively reflect the current risk state of each storage device.

[0060] Optionally, for example, there are 4 storage devices OSD1 to OSD4 (N = 4), and the target evaluation model dynamically adjusts the weights as follows by analyzing the state data of each storage device: stored capacity (first weight): 0.5, storage speed (second weight): 0.3, disk read-write pressure (third weight): 0.2, and the target state data of each OSD is as follows:

[0061] OSD1: stored capacity 85%, storage speed 1 megabyte per millisecond, disk read-write pressure 60%;

[0062] OSD2: stored capacity 70%, storage speed 0.5 megabyte per millisecond, disk read-write pressure 40%;

[0063] OSD3: stored capacity 60%, storage speed 1.5 megabyte per millisecond, disk read-write pressure 20%;

[0064] OSD4: stored capacity 50%, storage speed 0.75 megabyte per millisecond, disk read-write pressure 30%;

[0065] Then, the target score of each OSD is as follows:

[0066] The target score of OSD1 is: 0.5*0.85+0.3*1+0.2*0.60 = 0.845;

[0067] The target score of OSD2 is: 0.5*0.70+0.3*0.5+0.2*0.40 = 0.58;

[0068] The target score of OSD3 is: 0.5*0.60+0.3*1.5+0.2*0.20 = 0.79;

[0069] The target score of OSD4 is: 0.5*0.50+0.3*0.75+0.2*0.30 = 0.535;

[0070] Through the above calculation, the target scores obtained are 0.845, 0.58, 0.79 and 0.535 respectively. These four scores reflect the risk state of each OSD after considering the storage capacity, storage speed and disk read-write pressure.

[0071] It should be noted that through this example, it can be intuitively seen that the target scoring mechanism can quantitatively evaluate the water level risk of the storage device, providing a data basis for subsequent water level risk level evaluation and corresponding migration strategy formulation. In this way, the system can intelligently identify high-risk storage devices, such as OSD1, and take appropriate flow limiting or path recalculation measures, while low-risk devices such as OSD4 can be accelerated for data migration, fully utilizing their resources. This strategy helps to avoid the common high-water dead lock risk during expansion, optimizes resource allocation, reduces business impact, and improves the overall performance and stability of the storage system.

[0072] Step S23: According to the N target scores, determine the N water level risk levels based on a water level risk level evaluation table, wherein the water level risk level evaluation table records the correspondence between the target scores and the water level risk levels.

[0073] Optionally, the evaluation table records the water level risk levels (such as safe, warning, and dangerous) corresponding to different score intervals. Through matching, each storage device is assigned a water level risk level, and this step is the final result of the entire risk evaluation process and the basis for formulating a migration strategy.

[0074] Optionally, for example, in the example in step S22:

[0075] OSD1: Due to the high storage capacity (85%) and high disk read-write pressure (60%), even though its storage speed is fast, the overall score of 0.845 indicates that its water level risk level is relatively high, and it may be at the "warning" or "danger" level;

[0076] OSD2: The storage capacity is moderate (70%), and the storage speed and disk read-write pressure are lower than OSD1, so the overall score of 0.58 is lower, and it may be at the "safe" level;

[0077] OSD3: The storage capacity is low (60%), the storage speed is fast, and the disk read-write pressure is the smallest (20%), and the overall score of 0.79 indicates that its risk is moderate, and it may be at the "safe" or "warning" level;

[0078] OSD4: Each indicator is at a low level, and the overall score of 0.535 is the lowest, indicating that its water level risk level is the lowest, and it is at the "safe" level.

[0079] It should be noted that by converting the target score into a specific water level risk level, a clear division of risk states is achieved, which facilitates subsequent adoption of appropriate migration control measures according to different risk levels, effectively preventing further aggravation of high-water storage devices, and protecting the stability of the storage system and the safety of the data.

[0080] It should be noted that the whole water level risk level evaluation process builds an intelligent and dynamic risk evaluation system through dynamic weight adjustment, target score calculation and risk level division based on the score. This system not only can monitor the health status of the distributed storage cluster in real time, and predict the potential risks in the expansion and data migration process in advance, but also can automatically adjust the migration strategy according to different risk levels, such as adjusting the migration rate, path recalculation and business-aware scheduling, so as to realize the optimal configuration of resources, minimize the business impact, and improve the storage efficiency and data security.

[0081] In an exemplary embodiment, according to the target migration strategy, migrating M pieces of data to be migrated to N storage devices can be achieved through the following steps S31 to S35, wherein steps S31, S32, S33, S34 have no execution order:

[0082] Step S31: determining N first migration speeds corresponding to the N storage devices according to the N water level risk levels, wherein the i-th first migration speed in the N first migration speeds is the first migration speed corresponding to the i-th storage device in the N storage devices; the higher the water level risk level, the smaller the corresponding first migration speed;

[0083] Optionally, for the storage device with the water level risk level at the "danger" level, the migration speed can be set to the lowest to prevent further aggravating the storage pressure; and for the device with the water level risk level at the "safe" level, a higher migration speed can be set to speed up the data migration process.

[0084] Step S32: determining N first migration speeds corresponding to the N storage devices according to N business types corresponding to the N target devices, wherein the i-th business type in the N business types is the business type corresponding to the i-th storage device in the N storage devices; the business types include online transaction processing business, online analytical processing business and mixed business;

[0085] Optionally, online transaction processing (OLTP) usually requires low latency and high read-write performance, so the migration speed of the storage device carrying such business can be set to a lower level; online analytical processing (OLAP) can tolerate higher latency and lower read-write performance, so the migration speed of the storage device of this business can be higher. The migration speed of the mixed business is dynamically adjusted according to the priority and real-time performance requirement of the specific business.

[0086] It should be noted that the online transaction processing business mainly refers to a large number of short transactions that can be processed in real time, and these transactions usually involve database operations such as insertion, update and deletion. The OLTP system is designed to support daily operations, such as online transaction processing of banks, airline ticketing systems or any customer interaction service that requires immediate response; the online analytical processing business mainly targets complex queries and data analysis of data warehouses, which mainly focuses on fast reports and data analysis, rather than real-time transaction processing. The OLAP system is suitable for business intelligence applications such as market analysis, sales forecasting and financial reporting, which usually require in-depth mining and multi-angle analysis of a large amount of historical data; the mixed business refers to the workload containing both OLTP and OLAP characteristics, which requires both real-time transaction processing and complex data analysis.

[0087] Step S33: determining N first migration speeds corresponding to the N storage devices according to N failure probabilities corresponding to the N target devices, wherein an i-th failure probability in the N failure probabilities is a failure probability corresponding to an i-th storage device in the N storage devices; the greater the failure probability, the smaller the corresponding first migration speed;

[0088] Optionally, the greater the failure probability, the smaller the migration speed, so as to reduce the risk of data migration interruption and potential data loss caused by failure, thereby effectively reducing the data risk in the expansion process, ensuring the continuity of data migration and data integrity, and improving the high availability of the storage cluster.

[0089] Step S34: determining N first migration speeds corresponding to the N storage devices according to N function consumptions corresponding to the N target devices, wherein an i-th function consumption in the N function consumptions is a function consumption corresponding to an i-th storage device in the N storage devices; the greater the function consumption, the smaller the corresponding first migration speed;

[0090] Optionally, the function consumption refers to the resource consumption of the storage device and the entire system, such as memory and network bandwidth, when performing the data migration operation. The greater the function consumption of the device, the smaller the migration speed is set, so as to avoid excessive resource consumption affecting the overall performance and stability of the system.

[0091] Step S35: migrating M to-be-migrated data to the N storage devices according to the N first migration speeds according to the target migration strategy.

[0092] It should be noted that the whole data migration strategy execution process realizes fine and intelligent control of data migration in the expansion process of the storage cluster by intelligently adjusting the migration speed, which not only can effectively prevent the high water level risk, but also can intelligently balance resource allocation, reduce business impact, improve storage efficiency and the overall performance of the system based on the water level risk level, business type, fault probability and function consumption, and is a key technology for optimizing the expansion process of the distributed storage system, enhancing the stability of the storage system and the business continuity.

[0093] In an exemplary embodiment, in the process of migrating M pieces of to-be-migrated data to different storage devices in the N storage devices according to the target migration strategy, the method further comprises the following steps S41-S43:

[0094] Step S41: detecting the real stored capacity of the N storage devices;

[0095] It should be noted that in the data migration process, the system continuously monitors the real stored capacity of the N storage devices. This monitoring mechanism ensures that the expansion and data migration operations can respond to the changes in the state of the storage devices in real time, and is the basis for preventing high water level risk and dynamically optimizing the migration strategy.

[0096] Step S42: in the case that there is a storage device with real stored capacity greater than the preset stored capacity in the N storage devices, determining the storage device with real stored capacity greater than the preset stored capacity as a dangerous storage device;

[0097] Optionally, once it is detected that the real stored capacity of a storage device exceeds the preset threshold, it becomes a "dangerous storage device". The preset stored capacity threshold is usually set according to the health status of the storage cluster and the business demand, for example, when the stored capacity reaches 90%, the system considers that the device is in a dangerous state, triggering the subsequent protection measures.

[0098] It should be noted that by setting the preset stored capacity threshold, the system can actively identify and isolate the high water level risk device, avoid it becoming a bottleneck of data migration, and reduce the risk of data processing interruption or delay caused by overfull storage space.

[0099] Step S43: stopping migrating the to-be-migrated data to the dangerous storage device, and migrating the to-be-migrated data corresponding to the dangerous storage device to a target preset storage device.

[0100] Optionally, for the storage device identified as "dangerous storage device", immediately stop migrating any to-be-migrated data to it, and redirect the data originally planned to be migrated to the device to other target preset storage devices, which are usually storage devices with lower water level (stored capacity), good performance and meet the requirements of migration strategy.

[0101] It should be noted that this mechanism ensures the continuity and safety of data migration, avoids further migration of data to high-risk devices, effectively prevents the occurrence of backfill_toofull error, and guarantees the integrity of data and the stability of the storage system.

[0102] It should be noted that the above steps S41 to S43 constitute the core of the real-time monitoring and dynamic protection mechanism, which aims to evaluate the water level state of the storage device in real time, and immediately adjust the data migration strategy according to the evaluation result to avoid high water level risk: by stopping data migration to high water level devices and intelligently redirecting, the system can ensure the continuity of data migration process, even if high water level risk is encountered during capacity expansion, the system can maintain normal operation of business through dynamic scheduling, reducing business jitter and data processing delay; through real-time risk identification and active migration strategy adjustment, the risk of high water level deadlock during capacity expansion is greatly reduced, and the success rate and stability of the storage cluster during capacity expansion are improved.

[0103] In an exemplary embodiment, migrating the to-be-migrated data corresponding to the dangerous storage device to the target preset storage device can be achieved by the following steps S51-S55:

[0104] Step S51: intercepting the to-be-migrated data corresponding to the dangerous storage device to obtain target part to-be-migrated data;

[0105] Optionally, referring to Figure 7 After discovering that the water level of the storage device exceeds the preset threshold, i.e. becoming a "dangerous storage device", the system first intercepts the to-be-migrated data on the storage device to determine the data blocks (shards) that need to be immediately redirected.

[0106] It should be noted that through data interception, rapid response to high water level risk can be achieved, while unnecessary resource waste and system load increase are avoided, which helps to improve the efficiency and flexibility of data redirection operation.

[0107] Step S52: migrating the target part to-be-migrated data to the first preset storage device and the second preset storage device simultaneously, wherein the target preset storage device includes the first preset storage device and the second preset storage device;

[0108] Optionally, the target part of the data to be migrated is simultaneously migrated to the first preset storage device and the second preset storage device, ensuring the redundancy and security of the data. This double-writing mechanism can prevent data loss caused by a single device failure during data migration, and also provides a basis for data verification and selection in subsequent steps.

[0109] The life is, through data double-writing, not only improves the reliability of data migration, but also provides redundant data for subsequent intelligent selection, enhancing the robustness of the system, and ensuring the integrity and consistency of the data even in abnormal conditions.

[0110] Step S53: After migrating the target part of the data to be migrated to the first preset storage device and the second preset storage device, detecting the real-time stored capacity of the first preset storage device and the second preset storage device;

[0111] Optionally, after the double-writing operation is completed, the real-time stored capacity of the first preset storage device and the second preset storage device is detected in real time to evaluate the ability of the two devices to receive more data.

[0112] It should be noted that real-time detection can ensure that the system continuously monitors the use of storage resources during data redirection, providing real-time data support for subsequent decision-making, avoiding the occurrence of overload, and ensuring the health status of the storage device.

[0113] Step S54: In the case where the real-time stored capacity of the first preset storage device and the second preset storage device is less than the preset stored capacity, comparing the real-time stored capacity of the first preset storage device and the real-time stored capacity of the second preset storage device, and confirming the preset storage device with smaller real-time stored capacity as the target preset storage device;

[0114] It should be noted that step S54 intelligently determines the storage device for subsequent data migration by comparing the storage capacity, realizes the optimized use of storage resources, avoids the large water level difference between storage devices, and at the same time ensures the continuity and efficiency of data migration.

[0115] Step S55: Migrating the data to be migrated other than the target part of the data to be migrated corresponding to the dangerous storage device to the target preset storage device.

[0116] It should be noted that step S55 ensures that the migration of all data can be carried out according to the optimized strategy, effectively isolating and solving the high water level risk.

[0117] It should be noted that the above steps realize the optimization control of data migration of high water level risk devices through intelligent data interception, double write redundancy, real-time storage capacity detection, pre-set storage device selection and remaining data redirection. Not only does it solve the risk of data migration to high water level devices, but also improves the security, efficiency and resource utilization of data migration through double writing and intelligent selection of data, significantly improving the stability, efficiency and data security of the storage system during the expansion process.

[0118] In one exemplary embodiment, after migrating M pieces of data to be migrated to different storage devices in the N storage devices according to the target migration strategy, the method further comprises the following steps S61-S62:

[0119] Step S61: detecting the real-time stored capacity of the N storage devices;

[0120] Optionally, after completing a round of data migration, the real-time stored capacity of the N storage devices is detected again. This detection mechanism ensures the consistency between the actual state of the storage devices after the expansion operation and the expected target, and timely discovers and responds to any situation that may exceed the standard stored capacity.

[0121] Step S62: in the case that there is a storage device in the N storage devices whose real-time stored capacity is greater than the standard stored capacity, sending an adjustment instruction to the target evaluation model, wherein the adjustment instruction is used to instruct the target evaluation model to adjust the first weight, the second weight and the third weight according to the target state data to obtain a fourth weight corresponding to the stored capacity, a fifth weight corresponding to the storage speed and a sixth weight corresponding to the disk read-write pressure.

[0122] Optionally, if the real-time stored capacity of any storage device exceeds its standard stored capacity in step S61, the system will send an adjustment instruction to the target evaluation model, requiring the model to reevaluate the state data of the storage device, including the stored capacity, the storage speed and the disk read-write pressure, and dynamically adjust the corresponding weights.

[0123] Optionally, the target evaluation model will adjust the weight corresponding to the storage speed according to the following formula to obtain the fifth weight:

[0124] W_speed = 0.2 + 0.1 * (1 - e^(-0.5*t));

[0125] Wherein, W_speed is the fifth weight, and t is the migration duration (hours).

[0126] It should be noted that steps S61 to S62 embody the core idea of system closed-loop optimization. By monitoring and dynamically adjusting the model weight in real time, the system can respond to the high water level risk of the storage device after the expansion operation, and adjust the data migration strategy through intelligent decision-making to maintain the efficiency and stability of the entire storage cluster. By dynamically adjusting the model weight, the system can make more reasonable decisions based on the latest device state data, ensuring the continuous effectiveness of data migration path optimization. Even if the environmental conditions change during the expansion process, the efficiency and accuracy of data migration can be guaranteed, ensuring that the storage system can still operate in the best state after the expansion operation.

[0127] In one exemplary embodiment, before migrating M pieces of data to be migrated to the N storage devices according to the target migration strategy, the method further comprises steps S71-S74:

[0128] Step S71: Obtain priority information of M pieces of data to be migrated;

[0129] Optionally, before the data migration starts, the priority information of the M pieces of data to be migrated is first analyzed and obtained, and the priority information includes the business attributes of the data, such as data access frequency, business type, business importance, and other key indicators.

[0130] Step S72: Establish a data priority mapping table to determine M priorities of M pieces of data to be migrated, wherein the M pieces of data to be migrated and the M priorities have a one-to-one correspondence, and the data priority mapping table records the correspondence between the priority information and the priority;

[0131] Optionally, according to the data priority obtained in step S71, a priority mapping table is created. The priority mapping table can be a data structure such as a list or a dictionary, where each data block has a priority value associated with it for sorting and selection during data migration.

[0132] Optionally, when the business type of the data to be migrated is user logs, the priority corresponding thereto is the highest; when the business type of the data to be migrated is backup data, the priority corresponding thereto is medium; and when the business type of the data to be migrated is archive data, the priority corresponding thereto is the lowest.

[0133] Step S73: Sort the migration order of the M pieces of data to be migrated according to the M priorities to obtain a target migration order, wherein the higher the priority of the data to be migrated, the earlier the corresponding migration order;

[0134] Step S74: Determine a target migration strategy according to the target migration order, wherein the target migration strategy includes the target migration order.

[0135] Through the above steps, not only can the priority of data migration be dynamically adjusted according to the business attribute, ensuring the priority migration of key business data, but also the change of the storage environment can be monitored and responded in real time, and the migration strategy can be dynamically adjusted to maintain the efficiency of data migration and the stability of the storage system. This mechanism significantly improves the intelligence and automation level of the expansion operation, reduces the need for manual intervention, and reduces the business impact. It is an important practice of intelligent decision-making and dynamic optimization in the expansion of a distributed storage system.

[0136] Obviously, the above-described embodiments are only a part of the embodiments of the present application, not all. In order to better understand the above method, the above process is described below in combination with the embodiments, but not used to limit the technical solutions of the embodiments of the present application, specifically:

[0137] As shown in Figure 3 , an optional data migration method in an embodiment of the present application includes the following components in Table 1:

[0138] Table 1

[0139]

[0140]

[0141] As shown in Figure 4 , the interaction objects and processes of each component.

[0142] Optionally, as shown in Figure 5 , the embodiments of the present application include the following stages:

[0143] I. Pre-inspection stage (scan risk OSD before expansion, generate expansion plan):

[0144] Scan the risk storage device (OSD) before expansion (or before migrating data); identify high water level OSD (memory usage rate ≥ 85%); detect potential fault domain bottlenecks;

[0145] Generate expansion plan: recommend the number and location of expansion nodes; estimate migration time and resource requirements;

[0146] Perform environment check: verify network bandwidth (≥10 Gbps per second); ensure the health status of the new OSD.

[0147] II. Prediction stage (real-time running of water level prediction model (i.e. the above target evaluation model)):

[0148] Real-time data collection (5-second interval): OSD usage rate, migration speed, input / output (I / O) pressure index, business load type;

[0149] Real-time response logic: if risk_level!= previous_risk_level: # risk level changes immediately trigger decision; trigger_strategy_recalculation ().

[0150] Risk prediction calculation: run LSTM model to output risk score; dynamically adjust feature weights; generate risk heat map.

[0151] III. Decision phase (trigger different migration strategies according to risk level):

[0152] Pseudo code for generating migration strategy as follows:

[0153]

[0154] Path optimization decision: calculate candidate OSD comprehensive score, select Top-K target nodes (preferentially migrate data), and generate migration transaction plan.

[0155] IV. Execution phase (dynamically adjust migration rate / path):

[0156] Elastic migration control, where risk level and corresponding path switching are as shown in Table 2:

[0157] Table 2

[0158] Risk level Copy pool PG Erasure pool PG Metadata PG Safe 90% bandwidth 80% bandwidth 50% bandwidth Warning 30% bandwidth 40% bandwidth 20% bandwidth Danger Path switch Path switch Pause migration

[0159] V. Optimization phase (adjust model parameters based on effect feedback):

[0160] Effect monitoring indicators: migration deviation = | actual progress - predicted progress |; water level change slope; business impact index;

[0161] Closed-loop optimization mechanism: evaluate migration effect every 5 minutes; dynamically adjust model parameters; 24-hour full model retraining.

[0162] Optionally, as shown in Figure 6 if an exception occurs during the process of migrating data, a risk event (with a probability greater than 80%) will be reported to the controller, and the migration path will be re-planned.

[0163] Optionally, the water level prediction model in the embodiments of the present application predicts the risk level that may occur during the expansion process by dynamically analyzing the real-time state data of the OSD nodes. The specific implementation includes the following key links:

[0164] I. Data collection and preprocessing:

[0165] Collection frequency: collect OSD node data every 5 seconds;

[0166] Core indicators: Current usage: OSD storage space usage ratio (0-1); Migration speed: Current data migration rate, I / O pressure: Composite indicator reflecting business load (0-1);

[0167] Historical data window: Retain the last 10 cycles of data for trend analysis.

[0168] II. Feature engineering processing:

[0169] Current usage: Directly use the latest collected value;

[0170] Trend slope calculation, its pseudo code is as follows:

[0171] # Based on linear regression to calculate the water level change trend;

[0172] x = np.arange(len(data_points));

[0173] y = [dp['current_usage'] for dp in data_points];

[0174] slope = np.polyfit(x, y, 1)[0];

[0175] return max(0, slope * 10) # amplify the slope value to enhance sensitivity;

[0176] Dynamic weight adjustment mechanism, its pseudo code is as follows:

[0177] # When detecting rapid water level rise, automatically adjust the weight;

[0178] trend_factor = min(1.0, trend_slope * 5);

[0179] self.weights['current_usage'] = 0.55 - trend_factor * 0.1;

[0180] self.weights['trend_slope'] = 0.25 + trend_factor * 0.1;

[0181] III. Risk score and rating:

[0182] Optionally, the scoring formula is as follows:

[0183] risk_score = 0.55 * current_usage + 0.25 * trend_slope + 0.10 * migration_speed + 0.10 * io_pressure;

[0184] Three-level risk grading standards:

[0185] Safe (<0.6): Normal migration rate;

[0186] Warning (0.6-0.8): Start flow limiting measures;

[0187] Danger (≥0.8): Trigger path switching.

[0188] It should be noted that the above steps achieve trend-sensitive self-adaptation: automatically increase the trend weight when the water level rises rapidly; I / O pressure warning: include business load into the risk assessment system; dynamic amplification mechanism: 10 times amplification of the slope to enhance risk perception.

[0189] Optionally, the path optimization algorithm in the embodiment of the application calculates the optimal migration path for each PG, avoiding data migration to high water level OSD:

[0190] I. Candidate OSD screening:

[0191] Screening range: prefer new expansion OSD, when new nodes do not meet the conditions, include stock low water level OSD (usage <60%);

[0192] Screening condition: available capacity > migrated data volume x safety factor (1.2);

[0193] Current load < preset threshold (70%);

[0194] II. Multi-dimensional scoring system:

[0195] Capacity dimension (weight 100): capacity score = (1-current usage) x 100;

[0196] Performance dimension (weight 50): (1-load index) x 50;

[0197] Topology dimension (weight 30): topology score = number of source OSDs on the same rack / total number of source OSDs x 30;

[0198] Risk dimension (negative weight 20): risk prediction value x 20;

[0199] III. Optimal path generation:

[0200] Calculate the comprehensive score of all candidate OSDs--sort in descending order of score--select the target OSD number according to the PG type;

[0201] IV. Differentiated handling strategy:

[0202] Replica pool: Prefer low-loaded OSDs;

[0203] Erasure pool: Ensure data is distributed across different failure domains;

[0204] Metadata PG: Dedicated high-speed channel for low latency.

[0205] Optionally, the embodiments of the application further comprise dynamically adjusting the migration strategy according to the risk prediction results:

[0206] I. Wind selection aggregation mechanism:

[0207] Evaluate the risk of all target OSDs on the migration path, and the pseudo code is as follows:

[0208]

[0209] II. Three-level response strategy:

[0210] Safe state (risk < 0.6): Enable 90% migration bandwidth; Increase the number of parallel migrations by 50%;

[0211] Warning state (0.6 ≤ risk < 0.8): Limit migration bandwidth to 30%; Add monitoring markers for real-time tracking;

[0212] Dangerous state (risk ≥ 0.8), then trigger the pseudo code as follows:

[0213]

[0214] III. Business-aware scheduling:

[0215] OLTP business: Maximum bandwidth limit: 30%; Priority guarantee period: business peak period;

[0216] OLAP business: Minimum bandwidth guarantee: 70%; Migration window: business off-peak period;

[0217] Mixed business: Dynamic bandwidth allocation; Weighted scheduling based on business priority;

[0218] IV. Interruption recovery mechanism:

[0219] Transaction log records all migration operations; Supports resume transmission at breakpoint.

[0220] Optionally, in the embodiments of the present application, the water level prediction model is a LSTM time series prediction, and the network structure thereof comprises: an input layer: a 7-dimensional feature vector; a hidden layer: 32-unit LSTM; and an output layer: a risk probability value; and the training data thereof comprises: 10,000+ historical expansion cases; 5,000+ fault scenario simulation data; and the pseudo code for online learning is as follows:

[0221] # Model is automatically updated every 24 hours;

[0222] if time.now()-last_train>24h:;

[0223] retrain_model(current_data);

[0224] Migration speed dynamic weight.

[0225] Optionally, in the embodiments of the present application, the path optimization comprises:

[0226] I. CRUSH position awareness:

[0227] Rack affinity calculation, and the pseudo code thereof is as follows:

[0228]

[0229]

[0230] Cross-machine room avoidance strategy: automatically identify cross-machine room migration; increase distance penalty coefficient (deduct 5 points per 100 kilometers);

[0231] II. Business priority injection: the priority score is referenced to the following formula:

[0232] priority_score=base_score*(1+priority_level*0.3);

[0233] The mapping relationship between the business type and the priority is shown in Table 3 as follows:

[0234] Table 3

[0235] Business type Priority Coefficient Payment transaction Highest 1.3 User log High 1.2 Backup data Medium 1.1 Archive data Low 1.0

[0236] Optionally, in the embodiments of the present application, the elastic control enhancement implementation comprises:

[0237] I. Multi-level rate control:

[0238] The bandwidth allocation matrix is shown in Table 4 as follows:

[0239] Table 4

[0240] Risk level Base bandwidth OLTP upper limit OLAP lower limit Safe 90％ 30％ 70％ Warning 30％ 15％ 50％ Danger 10％ 0％ 30％

[0241] II. Seamless switching technology:

[0242] The double-write verification process is shown in Figure 7 : In the case of being unable to continue data migration, the data to be migrated is fragmented and migrated to different OSDs, and after verification, the OSDs to be truly migrated are determined; wherein the selected OSDs have switching performance guarantee: data verification delay: <50 milliseconds; path switching time: <200 milliseconds; double-write verification delay <50 milliseconds, only applicable to the same rack network environment; asynchronous verification mode is enabled for cross-machine room scenarios;

[0243] III. Self-healing suspension mechanism:

[0244] The suspension decision tree is shown in the pseudo code as follows:

[0245]

[0246] Automatic recovery detection: check water level change every 10 seconds; slope decreases by 50% immediately restore migration.

[0247] It should be noted that the present application embodiment realizes the complete intelligent expansion closed loop system through the collaborative innovation of multiple core modules. The water level prediction model realizes risk pre-sensing, the path optimization algorithm ensures maximum migration efficiency, and the elastic control logic guarantees business continuity, which together solves the high water level problem in distributed storage expansion.

[0248] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better implementation.

[0249] According to another aspect of the embodiment of the present application, a device for implementing the above-mentioned data migration method is also provided. Figure 8 is a structural block diagram of an optional data migration device according to an embodiment of the present application, as shown in Figure 8 , the device can include:

[0250] The acquisition module 802 is configured to acquire N groups of target state data, wherein the i-th group of target state data in the N groups of target state data is the target state data corresponding to the i-th storage device in the N storage devices, and the target state data is used to describe the storage attribute of the storage device; N is a positive integer, and i is a positive integer less than or equal to N;

[0251] The determining module 804 is configured to determine a target migration strategy according to the N sets of target state data, where the target migration strategy is used to indicate migration of M to-be-migrated data to the N storage devices, each of the N storage devices stores at least one to-be-migrated data, and M is a positive integer.

[0252] The migration module 806 is configured to migrate the M to-be-migrated data to the N storage devices according to the target migration strategy.

[0253] By the above device, target state data of a plurality of storage devices is acquired, and a migration strategy is formulated according to the acquired target state data, so as to determine how to migrate a plurality of to-be-migrated data to a plurality of storage devices, and then the plurality of to-be-migrated data is migrated to the plurality of storage devices according to the migration strategy. Therefore, manual selection of storage devices and formulation of a migration strategy are avoided, and thus the problem of low data migration efficiency caused by manual selection of storage devices can be solved, and the technical effects of improving data migration efficiency and optimizing data storage performance are achieved.

[0254] The features of the embodiments of the data migration device can be referred to the related descriptions of the embodiments of the data migration method, which will not be repeated here.

[0255] In an example embodiment, the determining module 804 is further configured to obtain N water level risk levels based on a target evaluation model according to the N sets of target state data, where the i th water level risk level in the N water level risk levels is a water level risk level corresponding to an i th storage device in the N storage devices, and the water level risk level is used to indicate a storage efficiency of the corresponding storage device; and determine a target migration strategy based on a migration strategy rule table according to the N water level risk levels, where the migration strategy rule table records a corresponding relationship between the target migration strategy and the water level risk level.

[0256] In an example embodiment, the determining module 804 is further configured to obtain, according to the N groups of target state data, a first weight corresponding to the stored capacity, a second weight corresponding to the storage speed, and a third weight corresponding to the disk read-write pressure based on a target evaluation model, wherein the target state data comprises the stored capacity, the storage speed, and the disk read-write pressure, the sum of the first weight, the second weight, and the third weight is 1; obtain N target scores according to the N stored capacities and the first weight, the N storage speeds and the second weight, and the N disk read-write pressures and the third weight, wherein the i-th target score in the N target scores is a target score corresponding to the i-th storage device in the N storage devices; determine the N water level risk grades based on a water level risk grade evaluation table according to the N target scores, wherein the water level risk grade evaluation table records a corresponding relationship between the target score and the water level risk grade.

[0257] In an example embodiment, the determining module 804 is further configured to determine N first migration speeds corresponding to the N storage devices according to the N water level risk grades, wherein the i-th first migration speed in the N first migration speeds is a first migration speed corresponding to the i-th storage device in the N storage devices; the higher the water level risk grade, the smaller the corresponding first migration speed; and / or determine N first migration speeds corresponding to the N storage devices according to N service types corresponding to the N target devices, wherein the i-th service type in the N service types is a service type corresponding to the i-th storage device in the N storage devices; the service types comprise online transaction processing services, online analytical processing services, and mixed services; and / or determine N first migration speeds corresponding to the N storage devices according to N failure probabilities corresponding to the N target devices, wherein the i-th failure probability in the N failure probabilities is a failure probability corresponding to the i-th storage device in the N storage devices; the greater the failure probability, the smaller the corresponding first migration speed; and / or determine N first migration speeds corresponding to the N storage devices according to N function consumptions corresponding to the N target devices, wherein the i-th function consumption in the N function consumptions is a function consumption corresponding to the i-th storage device in the N storage devices; the greater the function consumption, the smaller the corresponding first migration speed; and the migrating module 806 is further configured to migrate the M to-be-migrated data to the N storage devices according to the N first migration speeds according to the target migration strategy.

[0258] In an example embodiment, during the process of migrating M pieces of to-be-migrated data to different storage devices in the N storage devices according to the target migration strategy, the migration module 806 is further configured to detect real-time stored capacities of the N storage devices; in a case where there is a storage device with a real-time stored capacity greater than a preset stored capacity among the N storage devices, determine the storage device with the real-time stored capacity greater than the preset stored capacity as a dangerous storage device; stop migrating the to-be-migrated data to the dangerous storage device, and migrate the to-be-migrated data corresponding to the dangerous storage device to a target preset storage device.

[0259] In an example embodiment, the migration module 806 is further configured to intercept the to-be-migrated data corresponding to the dangerous storage device to obtain target partial to-be-migrated data; simultaneously migrate the target partial to-be-migrated data to a first preset storage device and a second preset storage device, wherein the target preset storage device includes the first preset storage device and the second preset storage device; after migrating the target partial to-be-migrated data to the first preset storage device and the second preset storage device, detect real-time stored capacities of the first preset storage device and the second preset storage device; in a case where the real-time stored capacities of the first preset storage device and the second preset storage device are both less than a preset stored capacity, compare the real-time stored capacity of the first preset storage device with the real-time stored capacity of the second preset storage device, and determine the preset storage device with the smaller real-time stored capacity as the target preset storage device; migrate the to-be-migrated data other than the target partial to-be-migrated data in the to-be-migrated data corresponding to the dangerous storage device to the target preset storage device.

[0260] In an example embodiment, the apparatus further includes a sending module configured to, after migrating M pieces of to-be-migrated data to different storage devices in the N storage devices according to the target migration strategy, detect real-time stored capacities of the N storage devices; in a case where there is a storage device with a real-time stored capacity greater than a standard stored capacity among the N storage devices, send an adjustment instruction to the target evaluation model, wherein the adjustment instruction is used to instruct the target evaluation model to adjust the first weight, the second weight, and the third weight according to the target state data to obtain a fourth weight corresponding to the stored capacity, a fifth weight corresponding to the storage speed, and a sixth weight corresponding to the disk read-write pressure.

[0261] Embodiments of the present application also provide an electronic device including a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the above data migration method embodiments.

[0262] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is configured to execute the steps in any of the above data migration method embodiments when running.

[0263] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0264] Embodiments of the present application also provide a computer program product, which includes a computer program. The computer program is executed by a processor to implement the steps in any of the above data migration method embodiments.

[0265] Embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium. The non-volatile computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps in any of the above data migration method embodiments.

[0266] The skilled in the art can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0267] The above provides a detailed description of the data migration method, device, electronic equipment and storage medium provided by the present application. The principles and implementation modes of the present application are described by applying specific examples. The above example description is only used to help understand the method and its core idea of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application. These improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data migration method, characterized by, The method comprises: obtaining N sets of target state data, wherein the i-th set of target state data in the N sets of target state data is target state data corresponding to an i-th storage device in N storage devices, and the target state data is used to describe a storage attribute of the storage device; N is a positive integer, and i is a positive integer less than or equal to N; determining a target migration strategy according to the N sets of target state data, wherein the target migration strategy is used to indicate that M pieces of to-be-migrated data are migrated to the N storage devices, and each of the N storage devices stores at least one piece of to-be-migrated data, and M is a positive integer; migrating the M pieces of to-be-migrated data to the N storage devices according to the target migration strategy.

2. The method of claim 1, wherein, Determining a target migration strategy according to the N sets of target state data comprises: obtaining N water level risk grades based on a target evaluation model according to the N sets of target state data, wherein an i-th water level risk grade in the N water level risk grades is a water level risk grade corresponding to an i-th storage device in the N storage devices, and the water level risk grade is used to indicate a storage efficiency of the corresponding storage device; determining a target migration strategy based on a migration strategy rule table according to the N water level risk grades, wherein the migration strategy rule table records a corresponding relationship between the target migration strategy and the water level risk grade.

3. The method of claim 2, wherein, Obtaining N water level risk grades based on a target evaluation model according to the N sets of target state data comprises: obtaining a first weight corresponding to a stored capacity, a second weight corresponding to a storage speed, and a third weight corresponding to a disk read-write pressure based on a target evaluation model according to the N sets of target state data, wherein the target state data comprises the stored capacity, the storage speed, and the disk read-write pressure, and the sum of the first weight, the second weight, and the third weight is 1; obtaining N target scores according to N stored capacities and the first weight, N storage speeds and the second weight, and N disk read-write pressures and the third weight, wherein an i-th target score in the N target scores is a target score corresponding to an i-th storage device in the N storage devices; determining the N water level risk grades based on a water level risk grade evaluation table according to the N target scores, wherein the water level risk grade evaluation table records a corresponding relationship between the target score and the water level risk grade.

4. The method of claim 2, wherein, Migrating M pieces of to-be-migrated data to the N storage devices according to the target migration strategy comprises: determining N first migration speeds corresponding to the N storage devices according to the N water level risk grades, wherein an i-th first migration speed in the N first migration speeds is a first migration speed corresponding to an i-th storage device in the N storage devices; the higher the water level risk grade is, the smaller the corresponding first migration speed is; and / or According to the N service types corresponding to the N target devices, determine N first migration speeds corresponding to the N storage devices, wherein the i-th service type in the N service types is the service type corresponding to the i-th storage device in the N storage devices; the service type includes: online transaction processing service, online analytical processing service, and hybrid service; and / or According to the N failure probabilities corresponding to the N target devices, determine N first migration speeds corresponding to the N storage devices, wherein the i-th failure probability in the N failure probabilities is the failure probability corresponding to the i-th storage device in the N storage devices; the greater the failure probability, the smaller the corresponding first migration speed; and / or According to the N function consumptions corresponding to the N target devices, determine N first migration speeds corresponding to the N storage devices, wherein the i-th function consumption in the N function consumptions is the function consumption corresponding to the i-th storage device in the N storage devices; the greater the function consumption, the smaller the corresponding first migration speed; According to the target migration strategy, migrate M to-be-migrated data to the N storage devices according to the N first migration speeds.

5. The method of claim 1, wherein, In the process of migrating M to-be-migrated data to different storage devices in the N storage devices according to the target migration strategy, the method further comprises: Detect the real-time stored capacity of the N storage devices; In the case that there is a storage device with a real-time stored capacity greater than a preset stored capacity in the N storage devices, determine the storage device with the real-time stored capacity greater than the preset stored capacity as a dangerous storage device; Stop migrating the to-be-migrated data to the dangerous storage device, and migrate the to-be-migrated data corresponding to the dangerous storage device to a target preset storage device.

6. The method of claim 5, wherein, Migrating the to-be-migrated data corresponding to the dangerous storage device to the target preset storage device comprises: Intercepting the to-be-migrated data corresponding to the dangerous storage device to obtain target partial to-be-migrated data; Migrating the target partial to-be-migrated data to a first preset storage device and a second preset storage device simultaneously, wherein the target preset storage device includes the first preset storage device and the second preset storage device; After migrating the target partial to-be-migrated data to the first preset storage device and the second preset storage device, detect the real-time stored capacity of the first preset storage device and the second preset storage device; In the case that the real-time stored capacity of the first preset storage device and the second preset storage device is less than the preset stored capacity, compare the real-time stored capacity of the first preset storage device and the real-time stored capacity of the second preset storage device, and confirm the preset storage device with the smaller real-time stored capacity as the target preset storage device; Migrate the to-be-migrated data corresponding to the dangerous storage device except the target partial to-be-migrated data to the target preset storage device.

7. The method of claim 3, wherein, After the M pieces of data to be migrated are migrated to different storage devices in the N storage devices according to the target migration strategy, the method further comprises: detecting real-time stored capacities of the N storage devices; in a case where there is a storage device in the N storage devices whose real-time stored capacity is greater than the standard stored capacity, sending an adjustment instruction to the target evaluation model, wherein the adjustment instruction is used to instruct the target evaluation model to adjust the first weight, the second weight and the third weight according to the target state data to obtain a fourth weight corresponding to the stored capacity, a fifth weight corresponding to the storage speed and a sixth weight corresponding to the disk read-write pressure.

8. A data migration apparatus, characterized by comprising: comprise: an acquisition module configured to acquire N sets of target state data, wherein the i th set of target state data in the N sets of target state data is target state data corresponding to an i th storage device in the N storage devices, the target state data is used to describe a storage attribute of the storage device; N is a positive integer, and i is a positive integer less than or equal to N; a determination module configured to determine a target migration strategy according to the N sets of target state data, wherein the target migration strategy is used to instruct to migrate M pieces of data to be migrated to the N storage devices, and each storage device in the N storage devices stores at least one piece of data to be migrated; a migration module configured to migrate the M pieces of data to be migrated to the N storage devices according to the target migration strategy.

9. An electronic device, comprising: comprise: a memory configured to store a computer program; a processor configured to implement the steps of the data migration method according to any one of claims 1 to 7 when the computer program is executed.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the data migration method according to any one of claims 1 to 7.

Images