Replica migration method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a copy migration method and device, a storage medium and electronic equipment, and relates to the field of computers.The method comprises the steps that access popularity evaluation is conducted on multiple copies based on access data of the multiple copies, and access popularity scores of the multiple copies are obtained; a first copy with the access popularity score larger than a first score threshold value is determined from the multiple copies, a second copy with the access popularity score smaller than a second score threshold value is determined from the multiple copies, and the first score large threshold value is larger than the second score threshold value; a first replica migration operation is performed on the first replica and a second replica migration operation is performed on the second replica.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the computer field, in particular, to a replica migration method and device, a storage medium and an electronic device. BACKGROUND

[0002] In a distributed storage system, due to the dynamic change of data access mode, how to effectively identify and respond to the change of data hotness has become a key problem that needs to be solved urgently. The traditional fixed replica strategy often cannot adapt to such changes, resulting in uneven allocation of resources, affecting the overall system performance and cost efficiency. Therefore, there is a technical problem of uneven allocation of replica resources in the related art. SUMMARY

[0003] Embodiments of the present application provide a replica migration method and device, a storage medium and an electronic device to at least solve the technical problem of uneven allocation of replica resources in the related art.

[0004] According to one embodiment of the present application, a replica migration method is provided, comprising: based on access data of a plurality of replicas, performing access hotness evaluation on the plurality of replicas to obtain access hotness scores of the plurality of replicas; determining a first replica with an access hotness score greater than a first score threshold and a second replica with an access hotness score less than a second score threshold from the plurality of replicas, wherein the first score threshold is greater than the second score threshold; performing a first replica migration operation on the first replica, and performing a second replica migration operation on the second replica.

[0005] According to another embodiment of the present application, a replica migration device is provided, comprising: an evaluation unit configured to perform access hotness evaluation on a plurality of replicas based on access data of the plurality of replicas to obtain access hotness scores of the plurality of replicas; a determination unit configured to determine a first replica with an access hotness score greater than a first score threshold and a second replica with an access hotness score less than a second score threshold from the plurality of replicas, wherein the first score threshold is greater than the second score threshold; a migration unit configured to perform a first replica migration operation on the first replica, and perform a second replica migration operation on the second replica.

[0006] According to still another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is set to execute the steps in any of the above method embodiments when running.

[0007] According to still another embodiment of the present application, an electronic device is also provided, comprising a memory and a processor, the memory stores a computer program, and the processor is set to run the computer program to execute the steps in any of the above method embodiments.

[0008] Through the embodiments provided in the present application, real-time hotness evaluation based on multiple copy access data can quickly identify hot data and cold data, dynamically adjust copy distribution, effectively alleviate the pressure brought by hotspot access, avoid access bottleneck phenomenon, and improve data access speed and user experience. The access hotness scoring mechanism and the pre-defined scoring threshold are used in the present embodiment to quickly identify the copies that need to be migrated, speed up the decision-making process of copy migration, reduce unnecessary calculation and delay, and improve the accuracy and efficiency of copy scheduling. In summary, the copy migration method of the present embodiment realizes the technical effect of improving the rationality and utilization rate of copy resource allocation by dynamically evaluating and migrating copies based on access hotness, effectively solving the problem of uneven allocation of copy resources. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a hardware structure block diagram of a copy migration method according to an embodiment of the present application;

[0010] Figure 2 is a flowchart of a copy migration method according to an embodiment of the present application;

[0011] Figure 3 is a flowchart of a cold and hot aware distributed storage copy dynamic migration method according to an embodiment of the present application;

[0012] Figure 4 is a structure block diagram of a copy migration device according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] Hereinafter, the embodiments of the present application will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0014] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0015] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device. Taking the case of running on a computer terminal, Figure 1This is a hardware structure block diagram of a computer terminal of a copy migration method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. The computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0016] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the replica migration method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0017] The transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a communication provider of a computer terminal. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0018] As an alternative, a replica migration method, such as Figure 2 As shown, the specific steps include:

[0019] S202, based on the access data of the multiple replicas, performing access popularity evaluation on the multiple replicas to obtain access popularity scores of the multiple replicas;

[0020] S204, determining a first copy with an access heat score greater than a first score threshold and a second copy with an access heat score less than a second score threshold from the plurality of copies, wherein the first score threshold is greater than the second score threshold;

[0021] S206, performing a first copy migration operation on the first copy and performing a second copy migration operation on the second copy.

[0022] Optionally, in the embodiment, the plurality of copies refer to a plurality of replicated versions of the same data object stored on different nodes in a distributed storage system to ensure high availability and redundancy of data. The distribution of copies helps to improve the access speed of data and the system disaster recovery capability.

[0023] Optionally, in the embodiment, the access data includes but is not limited to the access frequency, access time, access volume, and visitor information of the data object. These data are the key basis for evaluating data heat and guiding copy management.

[0024] Optionally, in the embodiment, the access heat evaluation is based on statistical information of data access, and the heat score of the data object is calculated by an algorithm. The score reflects the current access state of the data object and is the basis for determining whether the data is hot or cold. The access heat score is a numerical value representing the access heat of the data object obtained by quantitative analysis of the access data. A high score indicates high data heat, i.e. hot data, and a low score indicates cold data, i.e. cold data.

[0025] Optionally, in the embodiment, the first copy and the second copy respectively refer to a hot data copy with a score higher than the first score threshold and a cold data copy with a score lower than the second score threshold after the access heat evaluation.

[0026] Optionally, in the embodiment, the first score threshold and the second score threshold are preset numerical standards for defining the heat level of the data object. The first score threshold is used to identify hot data, and the second score threshold is used to identify cold data, and the first score threshold is greater than the second score threshold, ensuring effective differentiation between hot data and cold data.

[0027] Optionally, in the embodiment, the copy migration operation refers to the process of transferring a data copy from one node to another node. The migration operation can be the migration of a hot data copy to a node closer to active users or richer in resources, and the migration of a cold data copy to a node with lower cost or lighter load.

[0028] Optionally, in this embodiment, the system periodically collects access records of data replicas on all storage nodes, including access frequency, time interval, and other information. Based on these data, the system uses a pre-set evaluation model to calculate the heat score, which usually includes a time decay formula and access frequency weight distribution to quantify the heat status of the data. The score results are used for subsequent replica classification and migration decisions.

[0029] According to the score results, the system classifies replicas into two categories: hot data replicas and cold data replicas. Hot data replicas refer to replicas with scores higher than a first score threshold, and cold data replicas refer to replicas with scores lower than a second score threshold. This classification process is the basis for replica migration decisions, ensuring that hot data and cold data can be treated differently and optimized.

[0030] For hot data replicas (first replicas), the system initiates migration mechanisms to migrate them to nodes close to active users or with more abundant resources to reduce access latency and improve service performance. For cold data replicas (second replicas), the system performs operations to migrate them to low-cost or low-load nodes, or directly reduces the number of replicas to release storage space and optimize resource utilization. These replica migration operations are performed in an asynchronous manner without affecting existing business continuity and data consistency, ensuring stable system operation.

[0031] It can be understood that through the above steps, by collecting and analyzing access data, the heat score of each replica is calculated, which ensures the quantification and objective evaluation of replica heat. Next, the system classifies replicas according to pre-set score thresholds, clearly distinguishing high-heat data replicas from low-heat data replicas, providing clear guidance for subsequent migration strategies. Finally, the system performs targeted replica migration operations, migrating hot data replicas to nodes with small access pressure and abundant resources, while migrating cold data replicas to nodes with low cost and light load, or optimizing the number of replicas. The entire process is automated and intelligent, effectively improving the resource utilization efficiency of the distributed storage system, reducing access latency, improving user experience, and reducing the system's operational burden, improving the response speed and adaptability to business changes.

[0032] Through the embodiments provided in the present application, real-time hotness evaluation based on multiple copy access data can quickly identify hot data and cold data, dynamically adjust copy distribution, effectively alleviate the pressure brought by hotspot access, avoid access bottleneck phenomenon, and improve data access speed and user experience. The access hotness scoring mechanism and the pre-defined scoring threshold are used in the present embodiment to quickly identify the copies that need to be migrated, speed up the decision-making process of copy migration, reduce unnecessary calculation and delay, and improve the accuracy and efficiency of copy scheduling. In summary, the copy migration method of the present embodiment realizes the technical effect of improving the rationality and utilization rate of copy resource allocation through dynamic evaluation and migration of copies based on access hotness.

[0033] As an optional solution, performing the first copy migration operation on the first copy includes:

[0034] migrating the first copy to a first target data node, wherein the first target data node is a data node with a user access frequency higher than a preset frequency threshold;

[0035] In the case where the first target data node is associated with at least one other data node, copying the first copy to the at least one other data node.

[0036] Optionally, in the present embodiment, the first target data node is a target node dynamically selected by the first target data node system according to the user access pattern, for receiving the migration of the hot data copy. The user access frequency of this node is higher than the preset frequency threshold, which means that it is closer to the active user group or more suitable as a storage location for hot data.

[0037] Optionally, in the present embodiment, the preset frequency threshold is a standard value defined in advance for measuring the user access frequency of a data node. Nodes with a frequency higher than this threshold are considered as candidate nodes for preferentially migrating hot data copies.

[0038] Optionally, in the present embodiment, the other data nodes are storage nodes that are associated with the first target data node in some form, such as nodes with similar geographical locations, nodes at the same network level, or nodes sharing similar resource pools. These nodes can be accessed through the first target data node for access optimization, further improving the access efficiency of hot data.

[0039] Optionally, in the present embodiment, once the hot data copy (first copy) is identified, the system will automatically migrate it from the original storage location to the first target data node. This node is considered as the most preferred target due to its high user access frequency, which can effectively reduce the user's cross-node access delay and improve the service response speed. This operation is based on the metadata update mechanism within the system, ensuring the transparency of copy migration and the consistency of data.

[0040] To achieve more extensive access optimization, when the first target data node is found to have an association with at least one other data node, the system will further copy the first copy to these nodes. The selection of other data nodes is also based on the user's access mode and the network optimization or geographical proximity with the first target data node, aiming to build a more extensive and faster response data access network, especially in multi- regional or edge computing scenarios.

[0041] Through the embodiments provided in the present application, through quantitative analysis, a first copy (hot data copy) with an access popularity score higher than a first score threshold is identified. Subsequently, according to the principle that the user access frequency is higher than a preset frequency threshold, the system selects the best first target data node to receive the copy, and this selection ensures that hot data can be closer to active users, reduces access delay, and improves user experience. In order to further enhance the access performance of hot data, especially in large distributed systems or edge computing environments, if it is detected that the first target data node has an association with other data nodes and these nodes also meet the optimization conditions, the system will also copy the first copy to these nodes. This multi-copy replication and migration strategy not only improves the accessibility and redundancy of data, but also significantly improves the overall performance and quality of service of the distributed storage system by reducing the network overhead of cross-node access.

[0042] As an optional solution, performing a second copy migration operation on the second copy includes:

[0043] Migrating the second copy to a second target data node, wherein the second target data node is a data node with a user access frequency lower than a preset frequency threshold.

[0044] Optionally, in this embodiment, the second target data node is a node selected by the system according to the user access mode and resource configuration, used to receive the migration of the cold data copy (second copy). The user access frequency of this node is lower than the preset frequency threshold, and it usually has lower cost or looser resource constraints, and is suitable for storing low-heat data.

[0045] Optionally, in this embodiment, the preset frequency threshold is a standard frequency value set by the system to distinguish between data nodes with high and low access frequencies. Nodes below this threshold are considered ideal choices to receive the migration of cold data copies.

[0046] Optionally, in this embodiment, when the system identifies the cold data copies with access heat scores lower than the second score threshold, it will migrate these copies to the second target data node. This migration operation aims to release the high-performance node resources originally storing the cold data copies, and transfer the cold data to the nodes with higher cost-effectiveness, so as to better allocate resources and reduce storage costs. Through this step, the system can more flexibly cope with the access demand of different data, ensure the efficient storage and fast access of high-heat data, and at the same time avoid the over-allocation of resources on low-demand data.

[0047] Through the embodiments provided in this application, by continuously monitoring and analyzing the access patterns of data objects, the system can identify those second copies that have not been frequently accessed for a long time, and intelligently select the second target data node as the migration destination of the cold data copies based on the criterion that the user access frequency is lower than the preset threshold. The second target data node is usually located in an environment with lower cost or higher resource utilization efficiency, such as a low-cost storage unit in edge computing or a low-performance node in a data center. The implementation of this strategy not only reduces the resource occupation of cold data on high-performance nodes and reduces storage costs, but also ensures the integrity and availability of data.

[0048] As an optional solution, based on the access data of multiple copies, the access heat of the multiple copies is evaluated to obtain the access heat scores of the multiple copies, which includes:

[0049] The access frequency data of the multiple copies in the sliding time window is obtained, and the access time data of the multiple copies in the sliding time window is obtained, wherein the access data includes the access frequency data and the access time data;

[0050] The access frequency data and the access time data are weighted calculated to obtain the access heat scores.

[0051] Optionally, in this embodiment, the sliding time window is a dynamic time range set in the system for statistical analysis of data access patterns. The window slides over time, always maintaining the latest data within a certain period of time, for reflecting the latest access trend of the data.

[0052] Optionally, in this embodiment, the access frequency data is the statistical result of the number of times each copy is accessed within the sliding time window, which is used to quantify the access intensity of the data object.

[0053] Optionally, in this embodiment, the access time data is the specific time point of each access copy within the sliding time window, and the time interval between adjacent accesses, which is used to evaluate the timeliness and regularity of the access behavior.

[0054] Optionally, in this embodiment, the access logs of each replica are collected and analyzed in real time, with particular attention to the access frequency and access time of the data object within a set sliding time window. This step aims to capture both the transient fluctuations and long-term trends of data access, providing accurate statistical data for the subsequent hotness score.

[0055] After obtaining the basic data of access frequency and access time, the system uses a specific weighted calculation formula to consider parameters such as time decay factor and access frequency weight to comprehensively score the data. The access hotness score directly reflects the hotness level of the data object at the current time and is an important basis for replica dynamic migration decision-making.

[0056] Through the embodiments provided by the present application, firstly, the access frequency and access time of each replica are tracked and recorded within the sliding time window, which requires the system to update the data within the window in real time to ensure that the statistical results can reflect the latest access situation. The setting of the sliding time window helps the system to capture short-term access peaks and long-term access mode changes, which is beneficial to consider the timeliness when estimating data hotness. Then, by weighting the access frequency data and access time data, the system generates an access hotness score. This calculation process is comprehensive, not only considering the importance of access quantity, but also emphasizing the influence of the new and old degree of access time on data hotness through the time decay factor. The score result will clearly distinguish hot data from cold data, providing decision basis for subsequent intelligent replica migration.

[0057] As an optional solution, obtaining the access frequency data of multiple replicas within the sliding time window includes:

[0058] Obtaining the cumulative access times of multiple replicas within the sliding time window, and obtaining the frequency weight corresponding to each access in the cumulative access times;

[0059] Obtaining the access time data of multiple replicas within the sliding time window includes:

[0060] Obtaining the access time difference between each access and the previous access of each access in the cumulative access times, and the access time difference is used to determine the time decay parameter corresponding to each access.

[0061] Optionally, in this embodiment, the cumulative access times are the results of counting all access events of the specified data replica within the sliding time window, reflecting the overall access density of the data object within a certain time range.

[0062] Optionally, in this embodiment, the frequency weight indicates the relative importance of each access in the cumulative access times, which is usually calculated based on the order or characteristics of the access event to evaluate the uniformity of the access pattern and the degree of hot access.

[0063] Optionally, in the embodiment, the access time difference indicates the time interval between two consecutive accesses within the sliding time window. This time interval reflects the continuity and urgency of data access, and is a key factor in calculating the time decay parameter.

[0064] Optionally, in the embodiment, the time decay parameter is used to quantify the mathematical coefficient of the decreasing contribution of access time to the data popularity score, which determines the influence of historical access events on the current score calculation. The more remote the access event, the lower its weight in the calculation.

[0065] Optionally, in the embodiment, the total number of access events for each data replica within the sliding time window is monitored and recorded in real time, and each access event is assigned a corresponding frequency weight. The calculation of the weight depends on the closeness of the access and the relative position of the access event in the window, to reflect the high weight of recent frequent access and the low weight of inactive access.

[0066] By analyzing the timing between each access event in the cumulative access count, the system calculates the time interval between each access and its previous access, i.e. the access time difference. The key to this step is the determination of the time decay parameter, which reflects the decay of the influence of time on access events. The more recent the access event, the greater its weight, and vice versa.

[0067] Through the embodiments provided by the present application, by collecting the cumulative access count of each replica within the sliding time window, combined with the frequency weight of each access, the access intensity of data objects can be quantitatively evaluated. The design of the frequency weight takes into account the distribution characteristics of access events, ensuring the sensitivity to hot access and the stability to uniform access. Secondly, the system measures the time difference between each access and the previous access to calculate the time decay parameter. The introduction of this parameter makes time a dynamic weight factor in evaluating data popularity, and the recent access events have a more significant impact on the score, while the past access events gradually fade out of the consideration range. This design can help the system more accurately capture the real-time changes in data popularity and avoid evaluation bias due to historical access patterns.

[0068] As an optional solution, the access frequency data and access time data are weighted and calculated to obtain the access popularity score, which includes:

[0069] The frequency weight corresponding to each access and the time decay parameter corresponding to each access are weighted and summed to obtain the single access popularity score of each access.

[0070] The single access popularity scores of each access in the cumulative access count are summed to obtain the access popularity score.

[0071] Optionally, in the embodiment, the single-visit heat score is a heat score value assigned to each visit event, which comprehensively considers the frequency weight and the time decay parameter of the visit, and reflects the contribution of the visit event to the overall data heat.

[0072] Optionally, in the embodiment, two weights, the frequency weight and the time decay parameter, are applied to each visit event. The frequency weight reflects the relative importance of the visit event in the cumulative visit times, and the time decay parameter measures the influence of the time distance between the visit event and the current time on the score. The weighted sum of the two weights is defined as the single-visit heat score, which intuitively represents the contribution value of the visit event to the data heat.

[0073] After obtaining the single-visit heat scores of all visit events, the system sums up these scores to form the final visit heat score. This score represents the overall access heat level of the data object within the current sliding time window, and is an important basis for the system to determine whether to migrate the data replica or not.

[0074] Through the embodiments provided in the present application, first, each visit event is independently scored, and the product of the frequency weight and the time decay parameter forms a single-visit heat score that reflects the influence of the visit event on the data heat. This scoring mechanism ensures that recent high-frequency access events significantly improve the data heat, while earlier or less frequent access events contribute less, effectively capturing the dynamic change characteristics of the data heat. Subsequently, the system sums up the single-visit heat scores of all visit events to generate the overall visit heat score. This process not only accumulates the single-visit heat scores, but also comprehensively evaluates the data heat trend. The visit heat score not only reflects the current popularity of the data, but also considers the historical evolution and future prediction of the access pattern, providing a comprehensive heat perspective for the replica migration decision.

[0075] As an optional solution, the method further comprises:

[0076] determining a third replica from the plurality of replicas that has not been accessed in N time windows, wherein the N time windows are equal-length and consecutive time windows;

[0077] deleting the third replica.

[0078] Optionally, in the embodiment, the third replica refers to a data replica that has not been accessed in N equal-length and consecutive sliding time windows. Such a replica is considered to be extremely cold data or a redundant replica, and needs to be cleaned from the system to save storage resources.

[0079] Optionally, in the embodiment, the N time windows are a set of equal-length and continuous sliding time windows, which are set for tracking and analyzing the access history of the data object. The value of N can be adjusted according to the actual needs of the system and the expected period of data heat variation.

[0080] Optionally, in the embodiment, equal-length and continuous time windows mean that each time window has the same span and is adjacent to each other without time gaps, so as to ensure the continuity and integrity of the data access pattern analysis.

[0081] Optionally, in the embodiment, the system identifies the third copy that has no access record in the N time windows by monitoring and analyzing the access records of all data copies in the plurality of continuous sliding time windows. This step requires the system to have efficient data access tracking capability and fine time window division mechanism to accurately identify the redundant copy.

[0082] Once the existence of the third copy is determined, the system performs a deletion operation to release the storage resources occupied by the third copy. The operation is part of the copy management mechanism, which aims to optimize the use efficiency of storage space, reduce storage cost, and simplify data management within the cluster by cleaning up the extremely cold data copies that have not been accessed for a long time.

[0083] Through the embodiments provided in the present application, the system not only dynamically adjusts the copy layout through the access heat score mechanism, but also adds the identification and cleaning function of the redundant copy. The introduction of this step is based on the demand for limited storage resources and system performance optimization, aiming to remove unnecessary data copies through fine management, reduce storage overhead, and improve the overall operation efficiency of the storage system.

[0084] As an optional solution, the above-mentioned copy migration method can be used, but not limited to, for the method of dynamically sensing, intelligently scheduling and automatically migrating copies according to data access heat under the extended cluster architecture. The method can be widely applied to cloud computing platforms, edge computing environments, content distribution networks (CDN), data lakes, big data platforms and enterprise storage systems.

[0085] Further exemplify, the above-mentioned copy migration method is applied to the distributed storage copy dynamic migration scene based on hot and cold perception. In this scene, as the data scale continues to grow and the user access behavior diversifies, the traditional fixed copy strategy cannot meet the business requirements of strong real-time, cost sensitivity and high scalability. In a distributed environment, the data access pressure of different nodes is prone to deviation. The centralized access of part of the "hot data" will cause node bottlenecks, and the "cold data" will be idle for a long time but still occupy high-performance resources. How to dynamically allocate data copy distribution and quantity based on business access to achieve resource load balancing, performance improvement and cost control has become a key technical challenge for current intelligent storage scheduling.

[0086] To solve the above problems, the embodiment combines access frequency and time decay factor, has an intelligent copy migration system with real-time scoring capability and copy dynamic decision and execution capability, can automatically perform hot data replication, cold data migration or copy quantity optimization without interrupting service, and comprehensively improves the intelligence and resource efficiency of the distributed storage system.

[0087] Optionally, in the embodiment, based on the above-mentioned copy migration method, a hot and cold perception-based distributed storage copy dynamic migration method is proposed, and a flowchart is shown in Figure 3 The method realizes intelligent migration and redistribution of copies by collecting and analyzing data access logs, establishing a hotness scoring model for data objects, and using a time decay function and access frequency for weighted calculation. According to the scoring results, hot data and cold data are dynamically identified, and information such as network bandwidth, node load and geographic location is combined to determine whether to trigger copy migration. In the case of meeting the conditions, the system performs asynchronous copy replication or migration operation, updates the metadata and access path at the same time, and ensures business continuity. In addition, the application also includes a copy reduction and redundancy cleaning mechanism for controlling the number of copies and optimizing resource utilization.

[0088] Optionally, in the embodiment, the system periodically collects data access logs, and statistics the access frequency, last access time and other information of each data object; a hot and cold scoring model is constructed, and the access data is weighted and calculated using a sliding time window and a weight decay coefficient; the data objects are divided into "hot", "warm" and "cold" three levels according to the hot and cold scores.

[0089] Optionally, a calculation method of the hot and cold score is shown in formula (1):

[0090]

[0091] Wherein, H i : hot and cold score of data copy; N: total access record number of data copy; f ij: Frequency weight of jth visit; At ij : Time difference between current time and last visit; e^(-l·At ij ): Time decay term (newer visit is more important); a, b: Weight parameters for balancing visit frequency and visit time; l: Time decay rate (adjustable sensitivity).

[0092] When the hot-cold score of a certain data object exceeds the hot threshold, the system initiates the migration mechanism of the hot data copy, and deploys the copy to the node close to the active access user; if the hot-cold score is lower than the cold threshold, the copy is migrated from the high-performance node to the low-cost node or the copy is reduced; at the same time, the network bandwidth, node load and resource occupancy rate are considered to avoid excessive concentration or frequent migration.

[0093] Optionally, in the embodiment, an asynchronous migration mode is supported, data replication is performed without affecting normal read-write services; after migration is completed, metadata is updated and the cluster scheduler is notified to adjust the data routing strategy; a copy recycling mechanism is supported to clean up redundant copies to release storage space.

[0094] Optionally, in the embodiment, the system periodically evaluates the accuracy of the hot-cold score model, and automatically adjusts the weight parameters and time window according to the real-time access mode; migration jitter caused by score fluctuations is prevented, and a migration cooling-off period and a rollback strategy are introduced.

[0095] Further, in a set of cross-regional deployment extended cluster system, a certain video file is frequently accessed in the core node and is identified as "hot data", and the system automatically deploys copies in multiple edge nodes to reduce access delay. When the access heat decreases, the copy is migrated to a cold data node with lower cost, and only one copy is retained in the main center, thereby saving resources and maintaining service performance.

[0096] It can be understood that the embodiment constructs a heat score model that can be updated in real time based on multiple factors such as access frequency, time decay, sliding time window, etc., for determining the copy heat level, in combination with factors such as hot-cold score results, node load, network bandwidth, etc. Configurable thresholds are introduced to determine whether to perform copy migration operations; asynchronous migration of copies is supported to ensure that existing businesses are not affected; metadata is automatically updated after migration to maintain consistency of access paths and copies. Copy reduction (cold data reduction) and copy expansion (hot data expansion) are supported; low-access copies are cleaned up to save resources and improve storage utilization. The system can automatically adjust the score parameters and migration frequency according to access fluctuations; frequent migration or copy shock caused by score jitter is prevented.

[0097] Through the embodiments provided in the present application, the hot data access efficiency is improved, the cross-region access delay is reduced, the cold data occupies high-performance resources is reduced, the resource utilization is improved, the automatic copy management is supported, the operation and maintenance burden is reduced, and the response capability and expansion capability of the system to business load changes are improved.

[0098] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, and of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or in the form of a part that contributes to the prior art. The computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method of each embodiment of the present application.

[0099] In the present embodiment, a copy migration device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0100] Figure 4 is a structural block diagram of a copy migration device according to an embodiment of the present application, as shown in Figure 4 , the device includes:

[0101] An evaluation unit is configured to evaluate the access heat of a plurality of copies based on access data of the plurality of copies, and obtain an access heat score of the plurality of copies.

[0102] A determination unit is configured to determine a first copy with an access heat score greater than a first score threshold from the plurality of copies, and determine a second copy with an access heat score less than a second score threshold from the plurality of copies, wherein the first score threshold is greater than the second score threshold.

[0103] A migration unit is configured to perform a first copy migration operation on the first copy, and perform a second copy migration operation on the second copy.

[0104] As an optional solution, the migration unit 406 includes:

[0105] A first migration module is configured to migrate the first copy to a first target data node, wherein the first target data node is a data node with a user access frequency higher than a preset frequency threshold.

[0106] copying the first copy to at least one other data node in a case that the first target data node is associated with the at least one other data node.

[0107] As an optional solution, the migration unit 406 comprises:

[0108] a second migration module, configured to migrate the second copy to a second target data node, wherein the second target data node is a data node with a user access frequency lower than a preset frequency threshold.

[0109] As an optional solution, the evaluation unit 402 comprises:

[0110] a obtaining module, configured to obtain access frequency data of the multiple copies within a sliding time window, and obtain access time data of the multiple copies within the sliding time window, wherein the access data comprises the access frequency data and the access time data;

[0111] a calculation module, configured to perform weighted calculation on the access frequency data and the access time data to obtain the access heat score.

[0112] As an optional solution, the obtaining module comprises:

[0113] a first obtaining sub-module, configured to obtain a cumulative access number of the multiple copies within the sliding time window, and obtain a frequency weight corresponding to each access in the cumulative access number;

[0114] The obtaining module comprises:

[0115] a second obtaining sub-module, configured to obtain an access time difference between each access in the cumulative access number and a previous access of each access, the access time difference being used to determine a time decay parameter corresponding to each access.

[0116] As an optional solution, the calculation module comprises:

[0117] a first calculation sub-module, configured to perform weighted summation on the frequency weight corresponding to each access and the time decay parameter corresponding to each access to obtain a single-access heat score of each access;

[0118] a second calculation sub-module, configured to sum the single-access heat scores of each access in the cumulative access number to obtain the access heat score.

[0119] As an optional solution, the apparatus further comprises:

[0120] a determination module, configured to determine, from the multiple copies, a third copy that is not accessed within N time windows, wherein the N time windows are equal-length and continuous time windows.

[0121] The deleting module is configured to delete the third copy.

[0122] The specific examples in the embodiments can refer to the examples described in the above embodiments and exemplary embodiments, and the embodiments will not be described here again.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and a necessary general hardware platform, and of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the methods of the various embodiments of the present application.

[0124] It should be noted that the above modules can be realized by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: the above modules are located in the same processor; or the above modules are located in different processors in any combination.

[0125] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0126] In an exemplary embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0127] The embodiments of the present application also provide an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to execute the steps in any one of the above method embodiments.

[0128] In an exemplary embodiment, the above electronic device can also include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0129] The embodiments of the present application further provide a computer program product comprising a non-transitory computer readable storage medium storing the computer program, which, when executed by a processor, implements the steps of the method in the various embodiments of the present application.

[0130] The specific examples in the embodiments can refer to the examples described in the above embodiments and exemplary embodiments, which will not be repeated here.

[0131] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, which can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0132] The above is only the preferred embodiment of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A replica migration method, characterized in that: include: Based on the access data of the multiple copies, access popularity evaluation is performed on the multiple copies to obtain access popularity scores of the multiple copies; Determine a first replica from the multiple replicas whose access popularity score is greater than a first scoring threshold, and determine a second replica from the multiple replicas whose access popularity score is less than a second scoring threshold, wherein the first scoring threshold is greater than the second scoring threshold; A first replica migration operation is performed on the first replica, and a second replica migration operation is performed on the second replica.

2. The method according to claim 1, characterized in that The performing the first replica migration operation on the first replica includes: Migrating the first copy to a first target data node, wherein the first target data node is a data node whose user access frequency is higher than a preset frequency threshold; In a case where the first target data node is associated with at least one other data node, the first replica is copied to the at least one other data node.

3. The method according to claim 1, characterized in that The performing the second replica migration operation on the second replica includes: Migrate the second copy to a second target data node, where the second target data node is a data node whose user access frequency is lower than a preset frequency threshold.

4. The method according to claim 1, wherein The step of evaluating access popularity of the multiple copies based on the access data of the multiple copies to obtain access popularity scores of the multiple copies includes: Acquire access frequency data of the multiple replicas within a sliding time window, and acquire access time data of the multiple replicas within the sliding time window, wherein the access data includes the access frequency data and the access time data; The access frequency data and the access time data are weightedly calculated to obtain the access popularity score.

5. The method according to claim 4, characterized in that The acquiring of access frequency data of the plurality of replicas within the sliding time window comprises: Obtaining a cumulative number of accesses to the multiple replicas within the sliding time window, and obtaining a frequency weight corresponding to each access in the cumulative number of accesses; Acquiring access time data of the plurality of replicas within the sliding time window includes: An access time difference between each access in the cumulative number of accesses and a previous access before each access is obtained, where the access time difference is used to determine a time decay parameter corresponding to each access.

6. The method according to claim 5, characterized in that The weighted calculation of the access frequency data and the access time data to obtain the access popularity score includes: Performing a weighted summation on the frequency weight corresponding to each visit and the time decay parameter corresponding to each visit to obtain a single visit heat score for each visit; The single visit heat score of each visit in the cumulative number of visits is summed to obtain the visit heat score.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Determining a third replica from the multiple replicas that has not been accessed within N time windows, wherein the N time windows are equal-length and continuous time windows; The third copy is deleted.

8. A replica migration device, characterized in that: include: An evaluation unit, configured to evaluate access popularity of the multiple replicas based on access data of the multiple replicas, and obtain access popularity scores of the multiple replicas; a determining unit, configured to determine, from the plurality of replicas, a first replica having an access popularity score greater than a first scoring threshold, and to determine, from the plurality of replicas, a second replica having an access popularity score less than a second scoring threshold, wherein the first scoring threshold is greater than the second scoring threshold; The migration unit is configured to perform a first replica migration operation on the first replica and a second replica migration operation on the second replica.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the steps of the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images