Cross-computer room data processing method, equipment, storage medium and program product

Abstract

The disclosed embodiments provide a cross-computer room data processing method, device, storage medium and program product, which determines the target survival time and earliest use time of the data to be accessed by analyzing the task information of the periodic offline data access tasks in the current computer room; generates a data migration task according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, and determines the earliest use time as the execution period of the data migration task; executes the data migration task according to the execution period to read the data to be migrated across computer rooms, and caches it as a copy in the first cache space of the current computer room for local access by subsequent data access tasks in the current computer room. Generate a data migration task based on the target survival time and earliest use time of the data to be accessed, cache the copy in advance, so that the offline data access task does not need to read data across computer rooms, improves the task execution speed, is applicable to any storage scenario, and avoids the copy occupying too much cache space.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer and network communication technology, and in particular, to a method, device, storage medium and program product for cross-computer room data processing. Background Art

[0002] There is an intricate dependency relationship between the computing logic of the business and the data, which is difficult to completely decouple. The reason is that there is a gap between the data production end and the consumption end. The consumer end cannot accurately know the data storage strategy, which makes cross-computer room data access inevitable.

[0003] For cross-computer room data reading and writing, especially in big data offline scenarios, when computing tasks need to transmit data across computer rooms, it not only takes up a lot of bandwidth but also greatly increases the task running time. Summary of the invention

[0004] The embodiments of the present disclosure provide a cross-computer room data processing method, device, storage medium and program product to improve the execution speed of cross-computer room data access tasks.

[0005] In a first aspect, an embodiment of the present disclosure provides a cross-computer room data processing method, comprising:

[0006] Get the task information of periodic offline data access tasks in the current computer room;

[0007] Determine the target survival time and earliest usage time of the data to be accessed according to the task information;

[0008] According to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, determine the data to be migrated in the data to be accessed and the target survival time of the data to be migrated, generate a data migration task, and determine the earliest use time as the execution period of the data migration task;

[0009] According to the execution period, the data migration task is executed, and the data to be migrated is read from the computer room where the data to be accessed is located across computer rooms;

[0010] The read data to be migrated is cached as a copy in the first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

[0011] In a second aspect, an embodiment of the present disclosure provides a cross-computer room data processing device, including:

[0012] An acquisition unit, used to acquire task information of periodic offline data access tasks in the current computer room;

[0013] The analysis unit is used to determine the target survival time and the earliest use time of the data to be accessed according to the task information; determine the data to be migrated and the target survival time of the data to be migrated in the data to be accessed according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, generate a data migration task, and determine the earliest use time as the execution period of the data migration task;

[0014] A data migration unit, configured to execute the data migration task according to the execution deadline, and read the data to be migrated from the computer room where the data to be accessed is located across computer rooms;

[0015] The copy cache unit is used to cache the read data to be migrated as a copy in the first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

[0016] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0017] The memory stores computer-executable instructions;

[0018] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the cross-computer room data processing method as described in the first aspect and various possible designs of the first aspect.

[0019] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the cross-computer room data processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0020] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including computer execution instructions. When a processor executes the computer execution instructions, it implements the cross-computer room data processing method described in the first aspect and various possible designs of the first aspect.

[0021] The cross-computer room data processing method, device, storage medium and program product provided by the embodiment of the present disclosure obtain the task information of the periodic offline data access task in the current computer room; determine the target survival time and the earliest use time of the data to be accessed according to the task information; determine the data to be migrated in the data to be accessed and the target survival time of the data to be migrated according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, generate a data migration task, and determine the earliest use time as the execution period of the data migration task; execute the data migration task according to the execution period, read the data to be migrated from the computer room where the data to be accessed is located across the computer room; cache the read data to be migrated as a copy in the first cache space of the current computer room, so that the subsequent data access tasks in the current computer room can access the copy in the first cache space locally. Generate a data migration task based on the target survival time and the earliest use time of the data to be accessed, cache the cross-computer room data copy in advance, so that the offline data access task does not need to read the data across the computer room, which can effectively improve the execution speed of the offline data access task, can be applied to any storage scenario, and use the cache space as a constraint to avoid the copy occupying too much cache space. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 This is an example diagram of cross-computer room data processing in the prior art;

[0024] Figure 2 A schematic diagram of a cross-computer room data processing method flow chart provided by an embodiment of the present disclosure;

[0025] Figure 3 A schematic diagram of a cross-computer room data processing method flow chart provided by another embodiment of the present disclosure;

[0026] Figure 4 A schematic diagram of a cross-computer room data processing method flow chart provided by another embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram of a cross-computer room data processing method flow chart provided by another embodiment of the present disclosure;

[0028] Figure 6 A structural block diagram of a cross-computer room data processing device provided in an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0031] Due to the intricate dependency between the computing logic of the business and the data, it is difficult to completely decouple them. The reason is that there is a gap between the data production end and the consumption end. The consumption end cannot accurately know the data storage strategy, which makes cross-computer room data access inevitable.

[0032] For cross-computer room data reading and writing, especially in big data offline scenarios, when computing tasks need to transmit data across computer rooms, it not only takes up a lot of bandwidth but also greatly increases the task running time.

[0033] In the existing technology, vertical deployment is usually performed, that is, computing and storage are deployed in the same computer room to avoid cross-computer room data access. However, this limits the granularity of resource sharing. When the computer room capacity reaches the upper limit, capacity expansion is extremely difficult. The dependency relationship between computing logic and data is complex, and not all businesses can be fully deployed vertically.

[0034] In addition, some technologies use storage in exchange for network resources. Read-only copies of data across computer rooms are cached in local computer rooms to reduce repeated cross-computer room accesses, thereby exchanging disk space for cross-computer room bandwidth resources. However, their supported scenarios are limited. For structured data scenarios such as data warehouses, expansion to other scenarios requires major transformation. They are only applicable to append writes, and cached copies cannot be updated. Once the copies are modified, data inconsistency will occur. They only have a computing perspective and optimize from a computing perspective. Exchanging storage space for network bandwidth may cause network traffic bursts during data migration, affecting the operation of online jobs.

[0035] In view of at least one of the above technical problems, the present disclosure provides a cross-computer room data processing method, which obtains the task information of the periodic offline data access task in the current computer room; determines the target survival time and the earliest use time of the data to be accessed according to the task information; determines the data to be migrated in the data to be accessed and the target survival time of the data to be migrated according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, generates a data migration task, and determines the earliest use time as the execution period of the data migration task; executes the data migration task according to the execution period, reads the data to be migrated from the computer room where the data to be accessed is located across the computer room; caches the read data to be migrated as a copy in the first cache space of the current computer room, so that the subsequent data access tasks in the current computer room can access the copy in the first cache space locally. Generate a data migration task based on the target survival time and the earliest use time of the data to be accessed, cache the cross-computer room data copy in advance, so that the offline data access task does not need to read the data across the computer room, can effectively improve the execution speed of the offline data access task, can be applied to any storage scenario, and use the cache space as a constraint to avoid the copy occupying too much cache space.

[0036] The cross-computer room data processing method disclosed in the present invention is applicable to Figure 1 The application scenario shown includes computing devices, network devices and storage devices, wherein the computing device is used to analyze the task information of periodic offline data access tasks in the current computer room (any of multiple computer rooms), determine the target survival time and the earliest use time of the data to be accessed, determine the data to be migrated in the data to be accessed and the target survival time of the data to be migrated according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, generate a data migration task, and determine the earliest use time as the execution period of the data migration task; the storage device is used to execute the data migration task according to the execution period, read the data to be migrated from the computer room where the data to be accessed is located across computer rooms, and cache it as a copy in the first cache space of the current computer room; the network device is used to perform cross-computer room network management and control, in particular, low-priority bandwidth can be allocated to the data migration task, wherein the low-priority bandwidth priority is lower than the bandwidth of the online task, and when it is detected that the bandwidth of the online task is insufficient, the data migration task is restricted from using the low-priority bandwidth.

[0037] In addition, the computing device can also determine the target file to be accessed by the real-time data access task in the current computer room, and generate a lazy loading task for the target file; the storage device can adopt a lazy loading method to read the target file from the computer room where the target file is located across computer rooms at the file granularity, and cache the target file as a copy in the second cache space of the current computer room, so that the subsequent real-time data access tasks in the current computer room can access the copy in the second cache space locally. The storage device can also determine whether the target file exceeds the remaining capacity of the second cache space. If it exceeds, the file with the lowest access frequency in the second cache space is eliminated, and then the target file is cached.

[0038] The cross-computer room data processing method disclosed in the present invention will be described in detail below in conjunction with specific embodiments.

[0039] refer to Figure 2 , Figure 2 A schematic flow chart of a cross-machine room data processing method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device or a server, and the cross-machine room data processing method includes:

[0040] S201. Obtain task information of periodic offline data access tasks in the current computer room.

[0041] In this embodiment, there is an access task for accessing offline data that is periodically executed in the current computer room. The task information of the offline data access task can be the task information of the offline data access task in the historical period, or it can be preset task information, which can be used as source data for the following processing steps, wherein the characteristic information may specifically include but is not limited to the data accessed, how many cycles of data have been accessed (for example, data A within the last 7 days), the computer room where the data is located, the access time, and the like.

[0042] S202: Determine the target survival time and earliest usage time of the data to be accessed according to the task information.

[0043] In this embodiment, based on the above-mentioned task information, the target survival time (Time To Live, TTL) and the earliest usage time of the data to be accessed that needs to be accessed by the offline data access task in the current cycle are analyzed and determined, wherein the target survival time is the length of time that the data will be cached in the first cache space. For example, if the target survival time is 7 days, the data will be cleared after being cached in the first cache space for 7 days; and the earliest usage time is the earliest time when the data is used in the current cycle, that is, the earliest execution time of the offline data access task to access the data.

[0044] Optionally, there are multiple offline data access tasks in the current computer room that need to access the same data to be accessed; further, when determining the target survival time of the data to be accessed, it may include:

[0045] According to the task information of multiple periodic offline data access tasks in the current computer room, the use period of each offline data access task for the data to be accessed is determined, and the target survival time of the data to be accessed is determined according to the use period of each offline data access task for the data to be accessed.

[0046] In this embodiment, multiple offline data access tasks that need to access the same data to be accessed in the current computer room may have different usage cycles for the data to be accessed. For example, offline data access task a uses data within the last 7 days, and offline data access task b uses data within the last 14 days. Therefore, in order to satisfy the use of data by each offline data access task as much as possible, the target survival time of the data can be set to 14 days. That is, the longest usage cycle of the data to be accessed by each offline data access task can be determined as the target survival time of the data to be accessed. The data to be accessed can be cached locally according to the target survival time, which can satisfy the use of the data to be accessed by each offline data access task at the same time without crossing computer rooms.

[0047] Optionally, considering that the cache space is limited, the capacity upper limit is set for the first cache space in this embodiment. Therefore, when determining the target survival time of the data to be accessed, the remaining capacity of the first cache space also needs to be considered, that is, according to the usage cycle of the data to be accessed by each offline data access task and the read-write ratio of the data to be accessed across computer rooms, the target survival time of the data to be accessed is determined. For example, when the remaining capacity of the first cache space is insufficient and there are multiple data to be accessed that need to be migrated, the data to be accessed with a higher read-write ratio across computer rooms is migrated first. For example, data A has been accessed 10 times across computer rooms in the past 7 days, 3 times across computer rooms in the past 14 days, and data B has been accessed 6 times across computer rooms in the past 3 days, and 1 time across computer rooms in the past 7 days. Then, data A has been accessed 6 times across computer rooms in the past 7 days, and data B has been accessed 1 time across computer rooms in the past 7 days. Then, data A has been cached for the past 7 days and data B has been cached for the past 3 days. In the case of a surplus in the cache, consider caching data A for the past 8 to 14 days, and data B for the past 4 to 7 days does not need to be cached.

[0048] In addition, the earliest time of use of the data to be accessed may include:

[0049] According to the task information of multiple periodic offline data access tasks in the current computer room, the time for each offline data access task to access the data to be accessed in a cycle is determined, and the earliest time for accessing the data to be accessed is determined as the earliest use time of the data to be accessed. For example, the time for offline data access task c to access data A is 7:00, and the time for offline data access task d to access data A is 8:00, then the earliest use time of data A to be accessed is 7:00.

[0050] S203. According to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, determine the data to be migrated in the data to be accessed and the target survival time of the data to be migrated, generate a data migration task, and determine the earliest use time as the execution deadline of the data migration task.

[0051] In this embodiment, after determining the target survival time and earliest use time of the data to be accessed, the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room can be combined to determine part of the data or all of the data to be accessed as the data to be migrated, and the target survival time of the data to be accessed is determined as the target survival time of the data to be migrated (that is, the time cached in the first cache space), and a data migration task is generated, wherein the remaining capacity of the first cache space of the current computer room is used as a constraint, and in addition, the earliest use time of the data to be accessed is determined as the execution deadline of the data migration task. The data migration task can be specifically as follows:

[0052] {

[0053] "path":"hdfs: / / xxx",

[0054] "toDC":"dc",

[0055] "ttl":86400,

[0056] "deadline":1654427578235

[0057] }

[0058] The data migration task includes the path of the data to be accessed, the path to be cached in the current computer room, the target survival time (ttl), and the execution deadline (deadline).

[0059] S204: Execute the data migration task according to the execution deadline, and read the data to be migrated from the computer room where the data to be accessed is located across computer rooms.

[0060] In this embodiment, the data migration task can be executed according to the execution deadline, and the data to be migrated can be read across computer rooms from the computer room where the data to be accessed is located. Since the execution deadline of the offline data access task is the earliest time to access the data to be accessed, the data migration task needs to be executed before the execution deadline. If the data migration task has not been executed or has not been completed when the execution deadline is reached, the offline data access task cannot read the complete data to be migrated locally, and still needs to read the data to be migrated across computer rooms.

[0061] Optionally, when there are multiple data migration tasks, the data migration tasks can be sorted according to the order of execution deadlines, and the data migration tasks can be executed from front to back according to the sorting to ensure that the offline data access tasks executed earlier do not need to cross the computer room; however, for any data migration task, if the current time has exceeded its corresponding earliest use time, the data migration task is skipped and the subsequent data migration task is continued. In this embodiment, the concurrency of the data migration task can be controlled according to the idleness of the network.

[0062] Optionally, the network management center may allocate low-priority bandwidth to the data migration task, where the low-priority bandwidth has a lower priority than the bandwidth of the online task; when it is detected that the bandwidth of the online task is insufficient, the data migration task is restricted from using the low-priority bandwidth, and the operation of the online task is guaranteed by bandwidth restriction to avoid affecting the online task.

[0063] S205: Cache the read data to be migrated as a copy in a first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

[0064] In this embodiment, after the data to be migrated is read from the computer room where the data to be accessed is located across computer rooms, it is cached as a copy in the first cache space of the current computer room, so that subsequent data access tasks in the current computer room can access the copy in the first cache space locally without crossing computer rooms, thereby improving the data reading and writing speed, wherein the subsequent data access tasks include any data access tasks that need to access the data to be accessed, including but not limited to offline data access tasks. Among them, when the data to be accessed is updated, the copies of the same data to be accessed in all computer rooms can be updated.

[0065] The cross-computer room data processing method of this embodiment obtains the task information of the periodic offline data access task in the current computer room; determines the target survival time and the earliest use time of the data to be accessed according to the task information; determines the data to be migrated in the data to be accessed and the target survival time of the data to be migrated according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, generates a data migration task, and determines the earliest use time as the execution period of the data migration task; executes the data migration task according to the execution period, reads the data to be migrated from the computer room where the data to be accessed is located across the computer room; caches the read data to be migrated as a copy in the first cache space of the current computer room, so that the subsequent data access tasks in the current computer room can access the copy in the first cache space locally. Generate a data migration task based on the target survival time and the earliest use time of the data to be accessed, cache the cross-computer room data copy in advance, so that the offline data access task does not need to read data across the computer room, can effectively improve the execution speed of the offline data access task, can be applied to any storage scenario, and use the cache space as a constraint to avoid the copy occupying too much cache space.

[0066] Based on any of the above embodiments, the current computer room may also include some real-time data access tasks, which may be temporary, not periodic, and therefore cannot generate the above data migration tasks, such as Figure 3 As shown, the following steps can be used to process:

[0067] S301, for a real-time data access task in the current computer room, determining a target file to be accessed by the real-time data access task;

[0068] S302, using lazy loading to read the target file from the computer room where the target file is located across computer rooms at file granularity;

[0069] S303: Cache the target file as a copy in the second cache space of the current computer room, so that subsequent real-time data access tasks of the current computer room can locally access the copy in the second cache space.

[0070] In this embodiment, for real-time data access tasks, the target file to be accessed by the real-time data access task can be first determined, and then the target file can be read across computer rooms in a lazy loading manner, wherein the lazy loading manner means that the target file is read across computer rooms only when the target file is needed when executing the real-time data access task, and the cross-computer room reading is performed at the granularity of the file rather than the granularity of the partition, and only the required target files are read to avoid reading useless files. Reading data across computer rooms reduces bandwidth and improves the speed of accessing data across computer rooms.

[0071] After the target file is read across computer rooms, in addition to being used for real-time data access tasks, the target file can also be cached as a copy in the second cache space of the current computer room. Subsequent real-time data access tasks do not need to read the target file across computer rooms again when they need to access the target file. Instead, they are converted to locally accessing a copy of the target file in the second cache space, thereby improving the execution speed of real-time data access tasks.

[0072] Optionally, due to limited cache space, in this embodiment, a capacity upper limit is also set for the second cache space. When the target file is cached as a copy in the second cache space of the current computer room, the following steps may be specifically included:

[0073] Determining whether the target file exceeds the remaining capacity of the second cache space;

[0074] If the target file does not exceed the remaining capacity of the second cache space, directly cache the target file as a copy in the second cache space of the current computer room;

[0075] If the target file exceeds the remaining capacity of the second cache space, the file with the lowest access frequency in the second cache space is eliminated, and the target file is cached as a copy in the second cache space of the current computer room.

[0076] Among them, the least frequently accessed file in the second cache space may be eliminated by using an LFU (Least Frequently Used) algorithm or other algorithms, which will not be described in detail here.

[0077] It should be noted that, based on the above embodiment, the data access task is divided into a periodic offline data access task and a real-time data access task, which are executed separately. Figure 2 and Figure 3 Cross-computer room data processing method; such as Figure 4 As shown in the figure, the data access task can be judged as an offline data access task or a real-time data access task through job feature analysis; when it comes to periodic offline data access tasks, Figure 2 The cross-computer room data processing method shown in the figure, that is, finally determining the data migration task according to the task information (which can be obtained from the database DB), wherein the above process can be implemented by the computing cluster. If it is identified that the data access task is a non-periodic offline data access task (such as a real-time data access task), it can be directly skipped, and the above cross-computer room data processing method for the periodic offline data access task is not executed; while for the real-time data access task, it is performed Figure 3The cross-computer room data processing method shown, that is, after the real-time data access task is submitted, it is determined to be a real-time data access task through job feature analysis, that is, the target file is accessed in a lazy loading manner. If it is identified that the data access task is a periodic offline data access task, the above-mentioned cross-computer room data processing method for the real-time data access task is not executed, but the above-mentioned cross-computer room data processing method for the periodic offline data access task is executed, and it can be submitted to the above-mentioned computing cluster for processing.

[0078] Furthermore, network bandwidth management and control can be performed by network devices. For example, low-priority bandwidth can be allocated to data migration tasks, where the low-priority bandwidth has a lower priority than the bandwidth of online tasks. When insufficient bandwidth is detected for online tasks, the data migration tasks are restricted from using low-priority bandwidth to avoid affecting online tasks.

[0079] Based on any of the above embodiments, this embodiment can also provide a data governance solution based on the read-write ratio of the replica, such as Figure 5 As shown, it may specifically include:

[0080] S401, determining a target replica whose read-write ratio is higher than a first threshold and whose data volume is larger than a second threshold among various replicas cached in each computer room;

[0081] S402, determining a distribution strategy of the target copy in each computer room according to the number of reads and writes of the target copy by each computer room and the number of copies of the target copy, so that the number of cross-computer room reads and writes of the target copy by each computer room is minimized;

[0082] S403: Control the target copy to be cached in each computer room according to the distribution strategy.

[0083] In this embodiment, for some target replicas (such as some hot tables) with high read / write rates and large data volumes cached in each computer room, the optimal distribution strategy in each computer room can be analyzed to adjust the replica distribution to achieve the best benefit. When the target replicas are distributed in n computer rooms, there will be at least n-1 cross-computer room writes; when the computer room where the data access task is located does not have the target replica, there will be at least 1 cross-computer room read. By adjusting the replica distribution, the cross-computer room data read / write volume can be reduced, and the cross-computer room network bandwidth can be reduced. Among them, Figure 4As shown, the storage device can be configured with an analysis and learning center, a storage policy center, and a replica acceleration executor. During the analysis process, the analysis and learning center can analyze the replica access history metadata to determine some target replicas (such as some hot tables) with high read and write rates and large data volumes cached in each computer room. The storage policy center can analyze the optimal distribution strategy (data governance strategy) in each computer room based on some target replicas (such as some hot tables) with high read and write rates and large data volumes cached in each computer room. The replica acceleration executor can adjust the replica distribution according to the optimal distribution strategy.

[0084] For any target copy, the number of reads and writes of each computer room to the target copy can be analyzed. For example, the number of reads of the target copy by computer rooms x, y, and z is 3, 7, and 5 respectively. Assuming that the number of copies of the target copy is set to 2, that is, the target copy is cached in at most 2 computer rooms, different distribution strategies can be exhaustively enumerated to calculate the read and write costs across computer rooms.

[0085] For writes, the target replica cache has at least one cross-datacenter write in any two datacenters;

[0086] For reading, assuming that the target replicas are distributed in computer rooms X and Y, computer room Z needs to read the target replicas across computer rooms, and the number of reads is 5 times; assuming that the target replicas are distributed in computer rooms Y and Z, computer room X needs to read the target replicas across computer rooms, and the number of reads is 3 times; assuming that the target replicas are distributed in computer rooms X and Z, computer room Y needs to read the target replicas across computer rooms, and the number of reads is 7 times; therefore, the target replicas are distributed in computer rooms Y and Z, and the target replicas are read the least number of times across computer rooms, which is determined as the best distribution strategy, and then the distribution of the target replicas is adjusted so that they are cached in computer rooms Y and Z, which can reduce the amount of data read and written across computer rooms and reduce the network bandwidth across computer rooms. The best distribution strategy is as follows:

[0087] {

[0088] "write_client":["Y","Z"],

[0089] "replicas":["Y","Z"]

[0090] }

[0091] Corresponding to the cross-computer room data processing method of the above embodiment, Figure 6 This is a block diagram of the structure of the cross-computer room data processing device provided by the embodiment of the present disclosure. For the convenience of explanation, only the part related to the embodiment of the present disclosure is shown. Figure 6 The cross-computer room data processing device 500 includes: an acquisition unit 501, an analysis unit 502, a data migration unit 503, and a copy cache unit 504.

[0092] The acquisition unit 501 is used to acquire task information of periodic offline data access tasks in the current computer room;

[0093] The analysis unit 502 is used to determine the target survival time and the earliest use time of the data to be accessed according to the task information; determine the data to be migrated and the target survival time of the data to be migrated in the data to be accessed according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, generate a data migration task, and determine the earliest use time as the execution period of the data migration task;

[0094] A data migration unit 503 is used to execute the data migration task according to the execution period, and read the data to be migrated from the computer room where the data to be accessed is located across computer rooms;

[0095] The copy cache unit 504 is used to cache the read data to be migrated as a copy in the first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

[0096] In one or more embodiments of the present disclosure, there are multiple offline data access tasks in the current computer room that need to access the data to be accessed; when the analysis unit 502 determines the target survival time and the earliest use time of the data to be accessed according to the task information, it is used to:

[0097] Determine, based on task information of multiple periodic offline data access tasks in the current computer room, the usage period of each offline data access task for the data to be accessed, and determine, based on the usage period of each offline data access task for the data to be accessed and the cross-computer room read-write ratio of the data to be accessed, the target survival time of the data to be accessed; and / or

[0098] According to the task information of multiple periodic offline data access tasks in the current computer room, the time for each offline data access task to access the data to be accessed in a cycle is determined, and the earliest time for accessing the data to be accessed is determined as the earliest use time of the data to be accessed.

[0099] In one or more embodiments of the present disclosure, when determining the target survival time of the data to be accessed according to the usage cycle of the data to be accessed by each offline data access task, the analyzing unit 502 is configured to:

[0100] Determine the longest usage period of each offline data access task for the data to be accessed as the target survival time of the data to be accessed; or

[0101] According to the usage cycle of the to-be-accessed data by each offline data access task and the read-write ratio across computer rooms, the usage cycle of the data volume that satisfies the remaining capacity of the first cache space is determined as the target survival time of the to-be-accessed data.

[0102] In one or more embodiments of the present disclosure, when executing the data migration task according to the execution deadline, the data migration unit 503 is configured to:

[0103] The data migration tasks are sorted according to the order of the execution deadlines, and the data migration tasks are executed from the beginning to the end according to the order;

[0104] Among them, for any data migration task, if the current time has exceeded its corresponding execution deadline, the data migration task will be skipped.

[0105] In one or more embodiments of the present disclosure, when executing the data migration task, the data migration unit 503 is used to:

[0106] Allocating low-priority bandwidth to the data migration task, wherein the low-priority bandwidth has a lower priority than the bandwidth of the online task;

[0107] When it is detected that the bandwidth of the online task is insufficient, the data migration task is restricted from using the low-priority bandwidth.

[0108] In one or more embodiments of the present disclosure, the analysis unit 502 is further configured to determine, for a real-time data access task in the current computer room, a target file to be accessed by the real-time data access task;

[0109] The data migration unit 503 is further used to read the target file from the computer room where the target file is located across computer rooms at file granularity by using a lazy loading method;

[0110] The copy cache unit 504 is further used to cache the target file as a copy in the second cache space of the current computer room, so that subsequent real-time data access tasks of the current computer room can locally access the copy in the second cache space.

[0111] In one or more embodiments of the present disclosure, when the copy cache unit 504 caches the target file as a copy in the second cache space of the current computer room, it is used to:

[0112] Determining whether the target file exceeds the remaining capacity of the second cache space;

[0113] If the target file does not exceed the remaining capacity of the second cache space, directly cache the target file as a copy in the second cache space of the current computer room;

[0114] If the target file exceeds the remaining capacity of the second cache space, the file with the lowest access frequency in the second cache space is eliminated, and the target file is cached as a copy in the second cache space of the current computer room.

[0115] In one or more embodiments of the present disclosure, the data migration unit 503 is further configured to:

[0116] Determine a target replica whose read-write ratio is higher than a first threshold and whose data volume is higher than a second threshold among various replicas cached in each computer room;

[0117] According to the number of reads and writes of the target copy by each computer room and the number of copies of the target copy, a distribution strategy of the target copy in each computer room is determined so that the number of cross-computer room reads and writes of the target copy by each computer room is minimized;

[0118] The target copy is controlled to be cached in each computer room according to the distribution strategy.

[0119] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0120] refer to Figure 7 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure, and the electronic device 600 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0121] like Figure 7As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0122] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0123] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0124] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0125] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0126] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0127] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0130] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] In a first aspect, according to one or more embodiments of the present disclosure, a method for processing data across computer rooms is provided, comprising:

[0133] Get the task information of periodic offline data access tasks in the current computer room;

[0134] Determine the target survival time and earliest usage time of the data to be accessed according to the task information;

[0135] According to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, determine the data to be migrated in the data to be accessed and the target survival time of the data to be migrated, generate a data migration task, and determine the earliest use time as the execution period of the data migration task;

[0136] According to the execution period, the data migration task is executed, and the data to be migrated is read from the computer room where the data to be accessed is located across computer rooms;

[0137] The read data to be migrated is cached as a copy in the first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

[0138] According to one or more embodiments of the present disclosure, there are multiple offline data access tasks in the current computer room that need to access the data to be accessed; determining the target survival time and the earliest use time of the data to be accessed according to the task information includes:

[0139] Determine, based on task information of multiple periodic offline data access tasks in the current computer room, the usage period of each offline data access task for the data to be accessed, and determine, based on the usage period of each offline data access task for the data to be accessed and the cross-computer room read-write ratio of the data to be accessed, the target survival time of the data to be accessed; and / or

[0140] According to the task information of multiple periodic offline data access tasks in the current computer room, the time for each offline data access task to access the data to be accessed in a cycle is determined, and the earliest time for accessing the data to be accessed is determined as the earliest use time of the data to be accessed.

[0141] According to one or more embodiments of the present disclosure, determining the target survival time of the data to be accessed according to the usage cycle of the data to be accessed by each offline data access task includes:

[0142] Determine the longest usage period of each offline data access task for the data to be accessed as the target survival time of the data to be accessed; or

[0143] According to the usage cycle of the to-be-accessed data by each offline data access task and the read-write ratio across computer rooms, the usage cycle of the data volume that satisfies the remaining capacity of the first cache space is determined as the target survival time of the to-be-accessed data.

[0144] According to one or more embodiments of the present disclosure, executing the data migration task according to the execution deadline includes:

[0145] The data migration tasks are sorted according to the order of the execution deadlines, and the data migration tasks are executed from the beginning to the end according to the order;

[0146] Among them, for any data migration task, if the current time has exceeded its corresponding execution deadline, the data migration task will be skipped.

[0147] According to one or more embodiments of the present disclosure, executing the data migration task includes:

[0148] Allocating low-priority bandwidth to the data migration task, wherein the low-priority bandwidth has a lower priority than the bandwidth of the online task;

[0149] When it is detected that the bandwidth of the online task is insufficient, the data migration task is restricted from using the low-priority bandwidth.

[0150] According to one or more embodiments of the present disclosure, the method further includes:

[0151] For a real-time data access task in the current computer room, determining a target file to be accessed by the real-time data access task;

[0152] Using lazy loading, the target file is read from the computer room where the target file is located across computer rooms at file granularity;

[0153] The target file is cached as a copy in the second cache space of the current computer room, so that subsequent real-time data access tasks of the current computer room can locally access the copy in the second cache space.

[0154] According to one or more embodiments of the present disclosure, caching the target file as a copy in the second cache space of the current computer room includes:

[0155] Determining whether the target file exceeds the remaining capacity of the second cache space;

[0156] If the target file does not exceed the remaining capacity of the second cache space, directly cache the target file as a copy in the second cache space of the current computer room;

[0157] If the target file exceeds the remaining capacity of the second cache space, the file with the lowest access frequency in the second cache space is eliminated, and the target file is cached as a copy in the second cache space of the current computer room.

[0158] According to one or more embodiments of the present disclosure, the method further includes:

[0159] Determine a target replica whose read-write ratio is higher than a first threshold and whose data volume is higher than a second threshold among various replicas cached in each computer room;

[0160] According to the number of reads and writes of the target copy by each computer room and the number of copies of the target copy, a distribution strategy of the target copy in each computer room is determined so that the number of cross-computer room reads and writes of the target copy by each computer room is minimized;

[0161] The target copy is controlled to be cached in each computer room according to the distribution strategy.

[0162] In a second aspect, according to one or more embodiments of the present disclosure, a cross-computer room data processing device is provided, including:

[0163] An acquisition unit, used to acquire task information of periodic offline data access tasks in the current computer room;

[0164] The analysis unit is used to determine the target survival time and the earliest use time of the data to be accessed according to the task information; determine the data to be migrated and the target survival time of the data to be migrated in the data to be accessed according to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, generate a data migration task, and determine the earliest use time as the execution period of the data migration task;

[0165] A data migration unit, configured to execute the data migration task according to the execution deadline, and read the data to be migrated from the computer room where the data to be accessed is located across computer rooms;

[0166] The copy cache unit is used to cache the read data to be migrated as a copy in the first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

[0167] According to one or more embodiments of the present disclosure, there are multiple offline data access tasks in the current computer room that need to access the data to be accessed; when the analysis unit determines the target survival time and the earliest use time of the data to be accessed according to the task information, it is used to:

[0168] Determine, based on task information of multiple periodic offline data access tasks in the current computer room, the usage period of each offline data access task for the data to be accessed, and determine, based on the usage period of each offline data access task for the data to be accessed and the cross-computer room read-write ratio of the data to be accessed, the target survival time of the data to be accessed; and / or

[0169] According to the task information of multiple periodic offline data access tasks in the current computer room, the time for each offline data access task to access the data to be accessed in a cycle is determined, and the earliest time for accessing the data to be accessed is determined as the earliest use time of the data to be accessed.

[0170] According to one or more embodiments of the present disclosure, when determining the target survival time of the data to be accessed according to the usage cycle of the data to be accessed by each offline data access task, the analyzing unit is configured to:

[0171] Determine the longest usage period of each offline data access task for the data to be accessed as the target survival time of the data to be accessed; or

[0172] According to the usage cycle of the to-be-accessed data by each offline data access task and the read-write ratio across computer rooms, the usage cycle of the data volume that satisfies the remaining capacity of the first cache space is determined as the target survival time of the to-be-accessed data.

[0173] According to one or more embodiments of the present disclosure, when executing the data migration task according to the execution deadline, the data migration unit is configured to:

[0174] The data migration tasks are sorted according to the order of the execution deadlines, and the data migration tasks are executed from the beginning to the end according to the order;

[0175] Among them, for any data migration task, if the current time has exceeded its corresponding execution deadline, the data migration task will be skipped.

[0176] According to one or more embodiments of the present disclosure, when executing the data migration task, the data migration unit is used to:

[0177] Allocating low-priority bandwidth to the data migration task, wherein the low-priority bandwidth has a lower priority than the bandwidth of the online task;

[0178] When it is detected that the bandwidth of the online task is insufficient, the data migration task is restricted from using the low-priority bandwidth.

[0179] According to one or more embodiments of the present disclosure, the analysis unit is further used to determine, for a real-time data access task in the current computer room, a target file to be accessed by the real-time data access task;

[0180] The data migration unit is further used to read the target file from the computer room where the target file is located across computer rooms at file granularity in a lazy loading manner;

[0181] The copy cache unit is further used to cache the target file as a copy in the second cache space of the current computer room, so that subsequent real-time data access tasks of the current computer room can locally access the copy in the second cache space.

[0182] According to one or more embodiments of the present disclosure, when the copy cache unit caches the target file as a copy in the second cache space of the current computer room, it is used to:

[0183] Determining whether the target file exceeds the remaining capacity of the second cache space;

[0184] If the target file does not exceed the remaining capacity of the second cache space, directly cache the target file as a copy in the second cache space of the current computer room;

[0185] If the target file exceeds the remaining capacity of the second cache space, the file with the lowest access frequency in the second cache space is eliminated, and the target file is cached as a copy in the second cache space of the current computer room.

[0186] According to one or more embodiments of the present disclosure, the data migration unit is further used for:

[0187] Determine a target replica whose read-write ratio is higher than a first threshold and whose data volume is higher than a second threshold among various replicas cached in each computer room;

[0188] According to the number of reads and writes of the target copy by each computer room and the number of copies of the target copy, a distribution strategy of the target copy in each computer room is determined so that the number of cross-computer room reads and writes of the target copy by each computer room is minimized;

[0189] The target copy is controlled to be cached in each computer room according to the distribution strategy.

[0190] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0191] The memory stores computer-executable instructions;

[0192] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the cross-computer room data processing method as described in the first aspect and various possible designs of the first aspect.

[0193] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the cross-computer room data processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0194] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising computer execution instructions. When a processor executes the computer execution instructions, the cross-computer room data processing method as described in the first aspect and various possible designs of the first aspect is implemented.

[0195] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0196] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0197] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A cross-computer room data processing method, characterized in that: include: Get the task information of periodic offline data access tasks in the current computer room; Determine, according to the task information, a usage cycle of the data to be accessed by each offline data access task, and determine, according to the usage cycle and / or the cross-computer room read-write ratio of the data to be accessed, a target survival time of the data to be accessed, and determine the time when each offline data access task accesses the data to be accessed in one cycle, and determine the earliest time when the data to be accessed is accessed as the earliest usage time of the data to be accessed; According to the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, determine the data to be migrated in the data to be accessed and the target survival time of the data to be migrated, generate a data migration task, and determine the earliest use time as the execution period of the data migration task; Execute the data migration task according to the execution deadline, and read the data to be migrated from the computer room where the data to be accessed is located across computer rooms; The read data to be migrated is cached as a copy in the first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

2. The method according to claim 1, characterized in that Determining a target survival time of the data to be accessed according to the usage cycle includes: Determine the longest usage period of each offline data access task for the data to be accessed as the target survival time of the data to be accessed; or According to the usage cycle of the to-be-accessed data by each offline data access task and the read-write ratio across computer rooms, the usage cycle of the data volume that satisfies the remaining capacity of the first cache space is determined as the target survival time of the to-be-accessed data.

3. The method according to any one of claims 1 or 2, characterized in that: The executing the data migration task according to the execution deadline includes: The data migration tasks are sorted according to the order of the execution deadlines, and the data migration tasks are executed from the beginning to the end according to the order; Among them, for any data migration task, if the current time has exceeded its corresponding execution deadline, the data migration task will be skipped.

4. The method according to any one of claims 1 or 2, characterized in that: The executing the data migration task includes: Allocating low-priority bandwidth to the data migration task, wherein the low-priority bandwidth has a lower priority than the bandwidth of the online task; When it is detected that the bandwidth of the online task is insufficient, the data migration task is restricted from using the low-priority bandwidth.

5. The method according to claim 1, characterized in that The method further comprises: For a real-time data access task in the current computer room, determining a target file to be accessed by the real-time data access task; Using lazy loading, the target file is read from the computer room where the target file is located across computer rooms at file granularity; The target file is cached as a copy in the second cache space of the current computer room, so that subsequent real-time data access tasks of the current computer room can locally access the copy in the second cache space.

6. The method according to claim 5, characterized in that The step of caching the target file as a copy in the second cache space of the current computer room includes: Determining whether the target file exceeds the remaining capacity of the second cache space; If the target file does not exceed the remaining capacity of the second cache space, directly cache the target file as a copy in the second cache space of the current computer room; If the target file exceeds the remaining capacity of the second cache space, the file with the lowest access frequency in the second cache space is eliminated, and the target file is cached as a copy in the second cache space of the current computer room.

7. The method according to claim 1 or 5, characterized in that: The method further comprises: Determine a target replica whose read-write ratio is higher than a first threshold and whose data volume is higher than a second threshold among various replicas cached in each computer room; According to the number of reads and writes of the target copy by each computer room and the number of copies of the target copy, a distribution strategy of the target copy in each computer room is determined so that the number of cross-computer room reads and writes of the target copy by each computer room is minimized; The target copy is controlled to be cached in each computer room according to the distribution strategy.

8. A cross-computer room data processing device, characterized in that: include: An acquisition unit, used to acquire task information of periodic offline data access tasks in the current computer room; An analysis unit is used to determine, based on the task information, a usage cycle of the data to be accessed by each offline data access task, and determine, based on the usage cycle and / or the read-write ratio of the data to be accessed across computer rooms, a target survival time of the data to be accessed, and a time for each offline data access task to access the data to be accessed in a cycle, and determine the earliest time for accessing the data to be accessed as the earliest usage time of the data to be accessed; determine, based on the target survival time of the data to be accessed and the remaining capacity of the first cache space of the current computer room, the data to be migrated in the data to be accessed and the target survival time of the data to be migrated, generate a data migration task, and determine the earliest usage time as the execution period of the data migration task; A data migration unit, configured to execute the data migration task according to the execution deadline, and read the data to be migrated from the computer room where the data to be accessed is located across computer rooms; The copy cache unit is used to cache the read data to be migrated as a copy in the first cache space of the current computer room, so that subsequent data access tasks of the current computer room can locally access the copy in the first cache space.

9. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The method comprises computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 7 is implemented.

Images