Method and system for storing data in a distributed network
By using prediction models in global cloud storage services to determine the geographical area where data will be accessed in the future and transmit it to the corresponding data center or cache server, the problem of cross-region access latency and bandwidth costs in cloud storage is solved. Achieve more efficient data storage and access.
Patent Information
- Application Number
- CN202080071465.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-11-04
- Filing Date
- 2020-11-03
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-11-03
AI Technical Summary
In global cloud storage services, data access across geographic regions can lead to high latency and high network bandwidth costs, especially in the absence of network connections with sufficient fiber optic cables.
Improve data storage and access efficiency by using processors in a distributed network to predict the geographical area where data will be accessed in the future and transfer the data to a data center or cache server within the predicted area. The method trains a prediction model, such as a decision tree model, based on metadata and access information of previously uploaded data, to determine the storage and migration strategies of the data.
Improves the efficiency of data access in remote areas, reduces bandwidth costs and access time, and saves costs and time for users and service providers.
Smart Images

Figure CN114651433B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application is a continuation of U.S. patent application No. 16 / 673,128 filed on November 4, 2019, the disclosure of which is incorporated herein by reference. Background Art
[0003] Once the data is stored in the cloud, global cloud storage services provide access to large amounts of data from anywhere in the world. For example, an image uploaded in Europe can be instantly accessible for download in the United States.
[0004] Global cloud storage services are often divided into multiple geographic regions in order to manage large amounts of uploaded data. Thus, user requests to access data are often routed to the server closest to the user, particularly in the user's geographic region. The server then looks up the location of the requested data and then forwards the request for the data to the server storing the data, which may be located in a different geographic region.
[0005] When the requested data is stored far away from the requesting user, reading the requested data may incur high latency, which may degrade the requesting user's experience of the requested data. For service providers, long-distance reading also consumes valuable bandwidth, especially when there is a lack of network bandwidth between the user's geographic area and the data's geographic area, such as when not enough fiber optic cables are deployed between the two areas.
[0006] Global cloud storage services typically store uploaded data in the region where the data was uploaded. This may be effective in cases where the uploaded data is primarily downloaded in the same geographic region. However, in many cases, the uploaded data is primarily accessed from other geographic regions, which may result in high network bandwidth costs. Summary of the invention
[0007] One aspect of the present disclosure relates to a method for storing data in a distributed network having multiple data centers distributed over multiple geographic regions. The method may include: receiving, by one or more processors, data uploaded to a first data center of the distributed network, the uploaded data including metadata; receiving, by the one or more processors, access information about previously uploaded data; predicting, by the one or more processors, before the uploaded data is accessed, one or more of the multiple geographic regions from which the uploaded data will be accessed based on the metadata and the access information; and instructing, by the one or more processors, to transmit the uploaded data from the first data center to one or more second data centers located at one or more of the predicted geographic regions.
[0008] In some examples, the access information may be derived from a predictive model trained based on metadata of the previously uploaded data.
[0009] In some examples, the predicted model can be a decision tree model.
[0010] In some examples, the metadata may include an identification of a user who uploaded the upload data, and a location where the upload data was uploaded.
[0011] In some examples, the metadata of the previously uploaded data may include a location of the previously uploaded data, an identification of a user who uploaded the previously uploaded data, and a location where the previously uploaded data was uploaded.
[0012] In some examples, the metadata of the previously uploaded data may also include an identification of a directory or file path where the previously uploaded data is stored.
[0013] In some examples, the access information may indicate an amount of time between an initial upload of the previously uploaded data and a first download of the previously uploaded data. The method may include: predicting, by the one or more processors, an amount of time until the uploaded data is first downloaded; and instructing, by the one or more processors, based on the predicted amount of time, to transmit the uploaded data to one or more cache servers located at the one or more predicted geographic regions.
[0014] In some examples, instructing the uploaded data to be transmitted to one or more cache servers located at the one or more predicted geographic areas may include: determining, by the one or more processors, that the uploaded data is broadcast data based on the metadata and the access information; and for each given predicted geographic area to which the uploaded data is instructed to be transmitted, instructing, by the one or more processors, that the uploaded data be transmitted to at least one cache server in each data center of the given predicted geographic area.
[0015] In some examples, the method may include: instructing, by the one or more processors, to include the uploaded data in a file including previously uploaded data having a common predicted geographic area; and instructing, by the one or more processors, to transmit the file to one or more second data centers located at the common predicted geographic area.
[0016] In some examples, the file may be initially stored at one or more origin servers located at a first data center. Instructing the file to be transferred may include instructing, by the one or more processors, a data server of the one or more second data centers located at the common predicted geographic area to retrieve the file from the one or more origin servers.
[0017] Another aspect of the present disclosure relates to a system for storing data in a distributed network having multiple data centers distributed over multiple geographic regions. The system may include: one or more storage devices at a first data center of the distributed network, the one or more storage devices configured to store data uploaded to the first data center, the uploaded data including metadata; and one or more processors in communication with the one or more storage devices. The one or more processors may be configured to: receive access information about previously uploaded data previously stored in the multiple data centers of the distributed network; predict one or more geographic regions from which the uploaded data will be accessed based on the metadata and the access information before the uploaded data is accessed; and instruct the uploaded data to be transmitted from the first data center to one or more second data centers located at one or more predicted geographic regions of the predicted geographic regions.
[0018] In some examples, the access information may be derived from a predictive model trained based on metadata of the previously uploaded data.
[0019] In some examples, the predicted model can be a decision tree model.
[0020] In some examples, the metadata may include an identification of a user who uploaded the upload data, and a location where the upload data was uploaded.
[0021] In some examples, the metadata of the previously uploaded data may include a location of the previously uploaded data, an identification of a user who uploaded the previously uploaded data, and a location where the previously uploaded data was uploaded.
[0022] In some examples, the metadata for the previously uploaded data may include an identification of a directory or file path where the previously uploaded data is stored.
[0023] In some examples, the access information may indicate an amount of time between an initial upload of the previously uploaded data and a first download of the previously uploaded data. The one or more processors may be configured to: predict an amount of time until the uploaded data is first downloaded; and based on the predicted amount of time, instruct to transmit the uploaded data to one or more cache servers located at the one or more predicted geographic areas.
[0024] In some examples, the one or more processors may be configured to instruct transmission of the uploaded data to one or more cache servers located at the one or more predicted geographic areas based on a determination that the uploaded data is broadcast data based on the metadata, and for each given predicted geographic area to which the uploaded data is instructed to be transmitted, the one or more processors may be configured to instruct the uploaded data to be transmitted to at least one cache server in each data center of the given predicted geographic area.
[0025] In some examples, the one or more processors may be configured to: instruct inclusion of the uploaded data in a file including previously uploaded data having a common predicted geographic area; and instruct transmission of the file to one or more second data centers located at the common predicted geographic area.
[0026] In some examples, the file is initially stored at one or more origin servers located at a first data center. The one or more processors may be configured to instruct data servers of the one or more second data centers located at the common predicted geographic area to extract the file from the one or more origin servers. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 is a block diagram illustrating an example system in accordance with aspects of the present disclosure.
[0028] Figure 2 is a block diagram illustrating an example computing system in accordance with aspects of the present disclosure.
[0029] Figure 3 and Figure 4 is a block diagram illustrating an example data distribution scheme for a system according to aspects of the present disclosure.
[0030] Figure 5 is a flow chart illustrating an example method according to aspects of the present disclosure.
[0031] Figure 6 It is shown Figure 5 Flowcharts of various aspects of the flowchart. DETAILED DESCRIPTION
[0032] Overview
[0033] The technology generally relates to a system for efficiently storing uploaded data across a distributed network. The system may include a location predictor or prediction program that predicts one or more locations from which uploaded data files may be accessed in the future. The prediction may be based on upload and download patterns, also referred to as "access patterns," of previously uploaded data. Predicting access patterns for new uploaded data may improve storage efficiency of the uploaded data because the data may be strategically stored near those locations that will be accessed in the future.
[0034] In some implementations, the prediction program may be stored in a distributed network with multiple virtual machines across one or more data centers. Upload data may start from any data center stored in the network. The upload data is then analyzed by the prediction program and migrated to one or more other data centers where it is predicted that the upload data will be downloaded.
[0035] The prediction can be based on metadata included in each of the uploaded data and the previously uploaded data. For example, the metadata of the previously uploaded data can be used to train a prediction model, whereby the metadata can be associated with various predictors of the model. The metadata of the uploaded data can be the same or similar to the metadata of the previously uploaded data, whereby these similarities between the uploaded data and certain previously uploaded data can indicate future access patterns of the uploaded data. For example, if the uploaded data and the previously uploaded data have the same or similar customer identification, object name, object type, or some combination thereof, and originate from the same or similar region or data center, then similar access patterns can be expected to be followed. Therefore, given the common or similar metadata of the previously uploaded data and the uploaded data, the access pattern of the previously uploaded data can indicate the future access pattern of the uploaded data.
[0036] In some implementations, the upload data may be transferred to the data center using offline data migration techniques. For example, a separate large file may be established to store the upload data based on the destination data center of the data, whereby each large file has a different destination data center. The new upload data may then be appended to one or more large files based on the data center where the upload data is predicted to be downloaded. Data migration of the upload data may then be performed on a per-file basis, for example, when a given file reaches a certain size limit, or after a predefined amount of time (such as 12 hours) has passed.
[0037] In some implementations, metadata may also be used to predict the urgency of migrating uploaded data to its destination. For example, in the case of broadcast data, such as a streaming file that is broadcast from one user and made immediately accessible to other users worldwide, there may be high demand across multiple regions, either immediately or at a later time, while offline data migration may require a long time to deliver the uploaded data to its destination data center. In such cases, identifying the urgency of migrating the data may be used to initiate a cache injection of the uploaded data, whereby the data is transferred to a cache server at the destination data center, from which the data may be provided locally to the user. Cache injection may be performed in addition to and prior to offline data migration.
[0038] The above implementation can improve storage services for unstructured data within a distributed network, particularly for a distributed network with multiple data centers spread across multiple geographic regions. The improved storage service can make data uploaded in one region of the world more accessible in other regions of the world where the data is frequently accessed. This can thereby save costs and time for users and service providers, since accessing data from a distant location is generally more expensive and time-consuming than accessing data from a nearby location.
[0039] Example System
[0040] Figure 1 1 is a block diagram illustrating an example system including a distributed computing environment. System 100 may be a cloud storage service that provides users with the ability to upload data 101 to servers distributed across multiple geographic regions 110, 120, 130, 140 of system 100. Each geographic region may include one or more data centers. Figure 1 Data centers 110a and 110b for region 1 (110), data centers 120a and 120b for region 2 (120), data centers 130a and 130b for region 3 (130), and data centers 140a and 140b for region 4 (140) are shown, but the network may include additional regions, and each region may include additional data centers. Each data center may include one or more data servers 145 configured to store uploaded data.
[0041] The data centers 110a, 110b, 120a, 120b, 130a, 130b, 140a, 140b may be communicatively coupled, for example, via a network (not shown). The data center may further communicate with one or more client devices (not shown) via the network. Thus, for example, a client may perform operations in the "cloud". Such operations may include uploading and accessing data, such as uploading data 101. Accessing data may include downloading data, streaming data, copying data from one folder or directory to another folder or directory, or any other means of making data accessible in response to a user request received at a server of the system 100. In some examples, the data center may further communicate with a controller (not shown); thus, accessing data may include making data accessible in response to an instruction from the controller.
[0042] The data centers 110a, 110b, 120a, 120b, 130a, 130b, 140a, 140b may be located at locations that are relatively far from each other. For example, the data centers may be located in different countries around the world. The regions 110, 120, 130, 140 may group data centers that are relatively close to each other. Furthermore, in some examples, the data centers may be virtualized environments. Furthermore, although only a few data centers are shown, many data centers may be coupled via a network and / or additional networks.
[0043] like Figure 2 As shown in , each data center may include one or more computing devices 210, such as processors 220, servers, shards, cells, etc. It should be understood that each data center may include any number of computing devices, the number of computing devices in one data center may be different from the number of computing devices in another data center, and the number of computing devices in a given data center may change over time, for example, due to hardware being removed, replaced, upgraded, or expanded.
[0044] Each data center may also include a plurality of storage devices or memories 230, such as hard disk drives, random access memory, disks, disk arrays, tape drives, or any other type of storage device. The data center may implement any of a variety of architectures and technologies, including but not limited to direct attached storage (DAS), network attached storage (NAS), storage area networks (SANs), Fibre Channel (FC), Fibre Channel over Ethernet (FCoE), hybrid architecture networks, etc. In addition to storage devices, the data center may include many other devices, such as communication devices 260 (such as cabling, routers, etc.) to enable input and output between computing devices in the same data center or different data centers, between computing devices in the data center and a controller (not shown) of a network system, and between computing devices in the data center and client computing devices (not shown) connected to the network system.
[0045] The memory 230 of each computing device may store information accessible by the one or more processors 220 , including data 240 received at or generated by the one or more computing devices 210 , and instructions 250 executable by the one or more processors 220 .
[0046] The data 240 may include stored data 242, such as an uploaded data object; a metadata log 244 that tracks metadata for the uploaded data object 242; and one or more migration files 246; and a cached data file 248 where the uploaded data object 242 may be stored prior to being transferred from one data center to another data center. Details of the above examples of stored data are discussed in more detail below.
[0047] The instructions 250 may include a location access prediction program 252 configured to predict one or more locations where a given data object file may be accessed. Such locations may be one or more regions of a distributed network. The instructions 250 may also include a data migration program 254 and a data cache program 256 configured to perform transfers of data object files stored in the one or more migration files 246 and cache data files 248, respectively. Details of the above examples of stored programs are discussed in more detail below.
[0048] In some examples, the controller can communicate with computing devices in the data center and can facilitate the execution of programs. For example, the controller can track the capacity, state, workload, or other information of each computing device and use such information to assign tasks. The controller can include a processor and a memory, including data and instructions. In other examples, such operations can be performed by one or more computing devices in one of the data centers, and a separate controller can be omitted from the system.
[0049] The uploaded data 101 uploaded to the data center may include metadata indicating multiple attributes of the uploaded data. The metadata may be recorded in the data center where the data is uploaded, stored, or both (in Figure 1 In the example of , data center 140b). Figure 1 In the example of FIG. 1 , a metadata log 150 is provided in the data center 140 b to store metadata 155 of the uploaded data.
[0050] The metadata 155 may include identification of the region, data center, or both where the data was uploaded. As discussed in more detail below, because uploaded data is strategically migrated between geographic regions, in some cases the location where the data is uploaded may be different from the location where the data is stored. Additionally, in some cases, the data may be stored in multiple locations, including or excluding the location where the data was uploaded. In such cases, the metadata may also include identification of the region, data center, or both where the data is stored.
[0051] Other metadata may include, but are not limited to, client identification of the uploader, object name, object type, object size, name of the directory or folder to which the data is uploaded (such as a bucket name for storing objects), object name prefix (such as the file path of the uploaded object if more than one level of directory hierarchy is used to store objects), upload time, first download time, subsequent download times, number of access requests, etc. Basically, metadata may include both properties of the object and a running access log of the object.
[0052] Furthermore, where data is stored in multiple locations, it should be recognized that each location may be associated with different metadata for the same data object. For example, an object stored in region 1 may be downloaded earlier than the same object stored in region 2. As another example, the number of access requests for an object may vary from one location to the next. Thus, metadata from an uploaded object may be recorded separately at each location where the object is ultimately stored. Figure 1In the example of , each of data centers 110a, 110b, 120a, 120b, 130a, 130b, 140a, 140b may include its own access log (except for data center 140b not shown), which may store a log of data objects or files that have been uploaded to that data center, including metadata for those objects.
[0053] The metadata aggregator 160 may be used to track the metadata of each uploaded object across multiple regions. The aggregator 160 may be able to collect metadata from the metadata log of each data center on a regular basis, such as according to an aggregation schedule. Aggregated metadata may be timestamped so that changes in the metadata can be tracked over time. For example, the aggregated logs collected by the metadata log aggregator may be classified according to the duration represented by each aggregated log, such as metadata from the previous week, metadata from the previous month, metadata from the previous three months, metadata from a time period longer than the previous three months, and the like. Differences in the metadata across the classified logs may indicate changes in the uploaded data over time, such as an increase or decrease in interest in accessing the uploaded data. In addition, the metadata of a given uploaded data object may be tracked as a whole, or may be tracked according to the storage location. Therefore, the aggregated log may indicate overall changes in the metadata of a given uploaded data object, as well as regional-specific or data center-specific changes in the uploaded data object.
[0054] The aggregated data may then be fed to the prediction model 170 in order to train the model 170. The trained prediction model may better predict where future uploaded data is most likely to be accessed, thereby improving the placement of future uploaded data, making access to future uploaded data more efficient. The prediction model 170 may be a machine learning algorithm stored in the system 100, such as at one of the system's data centers. The prediction model may be a decision tree model, whereby the aggregated data may include information about how often and from where previously uploaded data was accessed, and thus may associate costs with placement of previously uploaded data in the system. Based on this information, the prediction model 170 may determine strategic placement of future uploaded data based on access patterns of past uploaded data.
[0055] In other examples, other types of machine learning algorithms may be applied in order to build a predictive model. In addition, more heuristic approaches are possible whereby certain information may determine the placement of future uploaded data. For example, if a number of previously uploaded data objects uploaded by a given user have a threshold number of downloads in certain areas, this may ensure that future data objects uploaded by that user are sent to and stored in areas with the threshold number of downloads.
[0056] The predictive model 170 can be dynamic. For example, the aggregation of metadata and access logs by the aggregator 160 can occur on a consistent and possibly predetermined basis, such as weekly, biweekly, monthly, or less or more frequently. The frequency of model updates can further depend on the nature of the data stored and shared on the particular network to which the system 100 is applied. That is, for some products that change relatively slowly, once a month may be sufficient. However, for other platforms or products where user tendencies are constantly changing and access patterns are constantly switching, monthly updates may not be sufficient, and more regular updates to the model, such as weekly, may be preferred.
[0057] Once the prediction model 170 has been trained based on previously uploaded and stored data, it can be used to predict where current or future uploaded data will most likely be accessed. Figure 1 In the example of , when a given unit of upload data 101 is received at data center 140b, the data is initially stored in data server 145 while being processed by access location predictor 180. Access location predictor 180 can determine whether the data is likely to be accessed in the same area as it was uploaded, whether the data is likely to be accessed in another area of the system. Access location predictor 180 can also predict the amount of time between when upload data 101 is uploaded and when it is downloaded. Based on these determinations, upload data 101 can remain stored in data server 145, can be stored at one or more migration files 190, can be stored in cache server 195, or any combination thereof. Figure 5 The storage determination operation is discussed in more detail.
[0058] Storage in data servers 145 is typically permanent, meaning that the data is expected to be stored there and not moved, and thus may remain stored indefinitely or until manually deleted. In contrast, storage in migration files 190 and cache servers 195 is typically temporary, meaning that the data is expected to be transferred to another location and may be automatically deleted sometime after the expected transfer. Figure 6 The data transfer operation is discussed in more detail.
[0059] Despite Figure 1The migration file 190 is shown as being separate from the data server, but it should be recognized that the migration file can actually be stored at one or more servers of the data center 140b, and therefore can be stored at the data server. In addition, although the contents of the migration file may be deleted periodically, such as after migration, the file itself may be permanent. In addition, the file may include permanent information, such as header information, indicating the destination to which the file contents are to be sent. The migration file can be a file of a distributed file system of a network. It should also be recognized that the data stored in the data server can also be stored in a large file of the distributed file system. In fact, the large file of the data server can be a file with information indicating that the file contents are located at its intended destination. For example, a file written to the data server 145 of the data center 140b can have a header indicating a destination of area 4, from which it can be determined that the contents of the file do not need to be sent to a different area.
[0060] Cache server 195 may be used by a data center for predictive injection caching as described herein, as well as for on-demand caching of actual data requests (as opposed to speculative requests that trigger injection, as described herein).
[0061] For illustrative purposes, Figure 1 The metadata aggregator 160 and prediction model 170 of the system are shown as being separate from each of the geographic regions 110, 120, 130, 140 and the data center, and the access location predictor 180 is shown as being included in the data center 140b. However, it should be appreciated that the data stored at the aggregator 160, prediction model 170, and access location predictor 180 and the instructions executed by the aggregator 160, prediction model 170, and access location predictor 180 can be anywhere in the system, such as in the regions and data centers shown, or in other regions or data centers not shown, or any combination thereof, including being distributed in multiple data centers across geographic regions, or distributed in multiple geographic regions across the system.
[0062] Example Method
[0063] Figure 3 is a flow chart illustrating an example routine 300 for storing data in a distributed network. A network may include distributed in various geographic areas (such as Figure 1 multiple data centers in regions 110, 120, 130, and 140) such as Figure 1Some operations in the method may be performed by a processor in the data center from which the data is transferred, while some operations may be performed by processors in other data centers or by processors and servers independent of data centers or geographic regions.
[0064] At block 310, data may be uploaded to a data center belonging to a first region of the network. The uploaded data may include a data object and metadata of the uploaded data object, such as the upload time, the type of the data object, the location where the object was uploaded, and the like.
[0065] At block 320, access information about previously uploaded data previously stored in the network may be received and analyzed. The previously uploaded data may also include metadata when it was uploaded, and may also have additional metadata gathered after the upload, such as metadata indicating the time when the previously uploaded data was accessed, the location where the previously uploaded data was accessed, etc.
[0066] At box 330, a prediction may be made of the geographic area of the network where the uploaded data may be accessed. The prediction or determination may be based on metadata of the current uploaded data, as well as access information about previously uploaded data. In addition, the prediction may be made before the current uploaded data is accessed, whereby metadata of the current uploaded data may be sufficient for the prediction at the time of uploading. Patterns identified in information of previously uploaded data may indicate possible results of accessing the current uploaded data, and thus may be used to predict an ideal location for storing the current uploaded data. In many cases, it may be preferred to store the uploaded data at the data center to which it was originally uploaded. However, in other cases, additionally or alternatively, it may be preferred to store the uploaded data in different data centers or even different geographic areas.
[0067] At block 340, the uploaded data is directed from the originating data center to other data centers where the data may be accessed, which may be located in geographic regions different than the first geographic region, thereby making access to the uploaded data more efficient in those other regions. Efficiency may be a measure of accessing data faster, spending less total bandwidth, executing on a connection with more available bandwidth, or any combination of these and other factors.
[0068] Figure 4 It is shown Figure 3 Flowchart of an example subroutine 400 of routine 300 of FIG. Subroutine 400 illustrates example operations that may be performed to implement determining one or more destination regions for uploading a data object, and distributing the data object to one or more data centers of the determined destination regions.
[0069] At block 410, one or more geographic regions from which the current upload data will be accessed are predicted. This prediction may be made by an access prediction program, such as Figure 1 The access location predictor 180 shown may be based on output from a prediction model that has been trained based on access logs of previously uploaded data, such as Figure 1 Prediction model 170 is shown. The prediction model can recognize access patterns of previously uploaded data and can predict where the current uploaded data is likely to be accessed based on those patterns.
[0070] At block 420, the current upload data may be sent (such as copied or moved) to one or more files designated for migrating the upload data to a destination region different from the first region, such as Figure 1 Migration file 190. The file to which the current upload data is sent may be based on the prediction of block 410, whereby the data will be transferred to those areas that are expected to be visited.
[0071] At block 430, the amount of time until the current upload data will be accessed in the predicted geographic area is predicted. The prediction may also be based on output from a prediction model. In this case, the access log of the previously uploaded data fed to the prediction model as training data should include information from which the direction between the upload and the first access of the previously uploaded data can be determined or derived, such as the upload time, and a log of the first or each access to the data. The prediction model can recognize access patterns of previously uploaded data, and can predict the time when the current upload data is likely to be accessed based on those patterns.
[0072] At block 440, the prediction of when the current upload data is likely to be accessed may be compared to a threshold, such as an amount of time from the current time. If the predicted amount of time until the current upload data is accessed exceeds or equals the threshold amount, meaning that the current upload data is not expected to be accessed on a relatively immediate basis, the operation may end, and the data may be migrated at a relatively slow rate using the one or more migration files. Conversely, if the predicted amount of time until the current upload data is accessed is less than the threshold amount, meaning that the current upload data is expected to be accessed on a relatively immediate basis, the operation may continue at block 450, whereby the current upload data is sent (copied or moved) to a cache server, such as a cache server. Figure 1 The cache server 195 of the data center can be injected into the cache server of the remote data center in the same geographic area or one or more different geographic areas. As with the migration file, the data can be injected into which data center, region, or both based on the determination of the access location predictor at box 410.
[0073] exist Figure 4In the example of , the decision to send the upload data to the migration file is shown as being made before determining the urgency or priority of transferring the data. Thus, the determination to migrate the data is made regardless of whether the data is needed sooner or later, that is, regardless of whether the data is also injected into the destination. However, in other examples, there may be data that may only be accessed on an immediate basis and not accessed at a later time, such as a streaming live video with a relatively short shelf life. In such cases, the system may determine to inject the upload data into a cache server at the remote data center without adding the upload data to the migration file to avoid unnecessary permanent storage of the data at the remote data center.
[0074] Figure 5 is an operational block diagram 500 illustrating example operations of an access location predictor 520 that predicts a network geographic area where uploaded data is likely to be accessed and directs uploaded data to be transmitted to the predicted geographic area, such as Figure 3 As shown in blocks 330 and 340 of routine 300, the access location predictor 520 selectively uploads data objects 5011-501 to the data center in region 4. N The access location predictor 520 may determine the location of each uploaded data object 5011-5012 based on the metadata of the object and the output of the prediction model for predicting one or more locations where the uploaded data may be accessed. N placement.
[0075] exist Figure 5 In the example, upload data objects 5011-501 N Each of the access locations has different metadata. Therefore, the access location predictor 520 is used to upload data objects 5011-501 N Each of them determines a different placement strategy.
[0076] In the example of uploading Object 1 (5011), the access location predictor 520 determines that the object is likely to be accessed at Region 2. Therefore, Object 1 is moved from the data server to a migration file 534, which may be a file dedicated to objects to be migrated from the data center where Object 1 was uploaded to Region 2.
[0077] like Figure 5As shown, each migration file may include common metadata 542, such as a header of the file, which may indicate the destination of the file. Presenting common metadata avoids the need to write this metadata separately to the migration file for each additional object 544, which may save space in the migration file and may further reduce the processing requirements for moving objects from the data server to the migration file. In the case of the destination zone, there is also no need to rewrite this metadata to the object after migration, because the destination zone will maintain the common metadata of the object and all other objects stored at the zone. In addition to the common metadata such as the destination zone, the remaining metadata of each object may be moved or copied with the object so that the object metadata is preserved during the migration.
[0078] In some examples, the migration file may have a predetermined capacity, whereby when moving or copying objects to the migration file causes the migration file to reach or exceed the predetermined capacity, the migration file may be transferred to one or more data centers in the destination region. Additionally or alternatively, the migration file may be transferred to one or more data centers in the destination region after a predetermined amount of time has passed since the migration file was created. Figure 4 The operations for transferring objects are described in more detail.
[0079] If it is determined that the object may be accessed from only one area, then the object may be moved to a single destination, such as in the case of (5011) in the example of object 1. However, in other cases, it may be determined that the object may be accessed from more than one area, including or excluding the area where the object was uploaded. In such cases, the access location predictor 520 may determine to move or copy the object to more than one migration file.
[0080] In the example of uploading Object 2 (5012), the access location predictor 520 determines that the object is likely to be accessed at each of regions 1, 3, and 4. Therefore, Object 2 is copied from the data server to each of migration files 532 and 536, which may be files dedicated to objects to be migrated from the data center where Object 2 was uploaded to regions 1 and 3. The object may also remain permanently stored at the data server so that it can be accessed at region 4.
[0081] As described above, the migration of objects from one region to another can begin after the file has been filled with several data objects. Since the migration file may be large, it may take some time to fill the migration file. However, in some cases, data objects may be in high demand in remote areas of the network, but only for a period of time before the data migration occurs. In this case, the slow-paced strategic relocation of data objects using the migration file will destroy the ability of users in other regions to efficiently access the data object when the demand for the data object is high. For example, if a user is streaming live video in Europe, and several users in the United States want to access the streaming video immediately, it is not helpful to wait for the video file to be migrated from a data center in Europe to a data center in the United States, even after the streaming video is stored in the data center in the United States, it may no longer be needed.
[0082] To address this challenge, object files can include metadata that can be used to determine how soon after the data is uploaded it will be accessed. Figure 1 In a system of FIG. 1 , metadata aggregator 160 may collect metadata showing the upload time and earliest download time for each stored object. This data may then be used to train a prediction model 170 to predict whether future uploaded data objects will be accessed soon or much later after the object is uploaded to the system. In turn, access location predictor 180 may use this information to determine, for any given uploaded data object, the amount of time until the object is likely to be downloaded. The system may also store a threshold time value, whereby if the predicted amount of time until the object is likely to be downloaded is equal to or less than the threshold amount, the object may bypass the usual data migration scheme via a migration file and be moved or copied to cache server 350 to transfer the uploaded data object to other areas of the system relatively quickly.
[0083] In the example of uploading object 3 (5013), the access location predictor 520 determines that the object may be in high demand within a threshold amount of time. Therefore, object 3 is copied from the data server to the cache server 550, so that the file can be injected into the cache server of the remote data center in region 4 and other regions (including but not limited to region 1, region 2 and region 3) based on the determination of the access location predictor 520. Object 3 can remain stored at the data server in the data center.
[0084] In many cases, it may be determined that an uploaded object may only be accessed from the region in which it was uploaded. This may be the case for files that are not shared between users in a personal store, or for files that are shared between a group of users that are geographically very close to each other. In such cases, the access location predictor 520 may determine that the uploaded object should not be copied to the migration files 532, 534, 536 or the cache server 550. Instead, the object remains stored at the data server of the data center where the object was uploaded. In the example of uploading object 4 (5014), the access location predictor 520 determines that the object may only be accessed in the original region 4. Therefore, object 4 will not be copied from the data server.
[0085] It should be noted that keeping a file on a data server, copying a file to a migration file, or copying a file to a cache server can be considered independent operations. Therefore, the decision to perform one operation does not exclude the performance of any other operation. For example, broadcast data such as streaming video files may be in high demand across multiple regions, either immediately or at a later time. In such cases, it may be determined that the file should be kept in the original data server and copied to both the migration file and the cache server to address long-term and short-term needs. This determination may be based at least in part on whether the uploaded data is broadcast data, as well as on other access patterns detected by the predictive model.
[0086] When uploading object N(501 N ), the access location predictor 520 determines that the object is likely to be in high demand at both regions 3 and 4 within a threshold amount of time and at a later time. Therefore, object N is copied from the data server to both the migration file 536 and the cache server 550 so that the object can be injected into the cache server of region 3 to address immediate demand, and migrated to the permanent storage of region 3 to address long-term demand.
[0087] Figure 6 is a graph showing the effect of a visit location predictor (such as Figure 1 The access location predictor 180 or Figure 5 A block diagram of example operations for allocating uploaded data objects based on a determination made by an access location predictor 520. Figure 6The example of shows uploaded objects being migrated (long term) and injected (short term) from region 4 (640) of the distributed network system 600 to both region 1 (610) and region 2 (620). Each region may include one or more data centers 610a, 620a, 640a, whereby each data center may include one or more corresponding processors 612, 622, 642, data servers 614, 624 (not shown for region 4), and cache servers 616, 626, 646. For purposes of illustration, the migration file 644 is shown as being stored at the data center 640 of region 4, but the migration file may also be stored at data centers of other regions to facilitate the transmission of objects uploaded at those other regions as well over the network.
[0088] In operation, moving the contents of the migration file 644 of the data center 640a of region 4 can begin with the data migration controller 650 of the system 600 executing a program, thereby querying the data center 610a for data files stored in its servers. The processor 642 can receive the query and, in response, can provide information indicating the destination of each file. In the case where the file is to be stored at region 4 (whereby the destination is region 4), the data migration controller 650 can take no further action. On the contrary, in the case where the file is to be stored at other regions such as regions 1 and 2 (whereby the destination indicated by the file is not region 4), the data migration controller 650 can determine to initiate the migration of data from the file to the identified destination. The data migration controller 650 can transmit instructions to one or more processors of each identified destination, such as processors 612, 622. Based on these instructions, these processors can instruct the data servers 614, 624 at their respective locations to perform data migration, whereby each data server 614, 624 can access and read the corresponding migration file 644 at region 4. After reading the migration file 644, the contents of the migration file 644 can be deleted. This process may be repeated for future data objects uploaded to data center 610a in region 4 (640).
[0089] The contents of the cache server 646 of the mobile region 4 data center 640a can begin with the server 646 pre-fetching the data to be transmitted and injecting it into the cache servers 616, 626 of the remote data center according to the instructions of the access location predictor. The injection can be performed on a relatively instant scale, which means that there are no further steps to be performed before initiating the data injection. This can make the data available in other data centers as soon as possible. Unlike data migration (in data migration, data can be stored at a fixed and limited number of data centers and a fixed and limited number of locations in the destination area), cache injection can place copies of the uploaded and transmitted objects in multiple data centers in any given area to which it is sent, including but not limited to caches of all data centers in the destination area. This can allow many users to access the required data in a short period of time. Compared with synchronous replication, synchronous cache injection is relatively fast and efficient.
[0090] The above examples refer to data distribution schemes according to data centers and regions. However, those skilled in the art will readily recognize that the same principles can be applied to other systems that organize data in different ways. The basic principle is that some parts of a large-scale network, such as the global network, may be closer to any given user than other parts of the network, and to the extent that it can be predicted from which locations the data will be accessed, it may be advantageous to move the data closer to those locations from which it will be accessed after the data has been uploaded. In this regard, if it can be predicted that a particular one or a group of users may access the data in the future, the data can be moved to a data center, server, or other unit storage that is closer to the predicted accessing user.
[0091] Furthermore, the above examples generally describe a location as being closer to a user when the location is geographically closer. However, those skilled in the art will recognize that the "closeness" of data is not necessarily a measure of geographic distance, but rather a measure of the cost of accessing the data. Thus, when data is located "closer" to a user or group of users, or "closer" to a location where the data is predicted to be accessed, the location may be selected to reduce the overall cost of accessing the data, such as bandwidth, time, bandwidth usage fees as a contracting party, or any combination of these or other factors.
[0092] Although the technology herein has been described with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the present technology. Therefore, it should be understood that various modifications may be made to the illustrative embodiments and other arrangements may be designed without departing from the spirit and scope of the present technology as defined by the appended claims.
Claims
1. A method for storing data in a distributed network having a plurality of data centers distributed over a plurality of geographic regions and accessed by a plurality of users, the data comprising metadata and accessible by the plurality of users, the metadata indicating one or more storage locations of the data within the distributed network and the corresponding geographic locations from which the data was uploaded, wherein: Each geographic area includes at least one user from the plurality of users, the method comprising: receiving, by one or more processors, data uploaded by a first user among the plurality of users to a first data center of the distributed network; receiving, by the one or more processors, access information regarding access to previously uploaded data by the plurality of users; predicting, by the one or more processors, before the uploaded data is accessed by any of the plurality of users, one or more of the plurality of geographic areas from which the uploaded data will be accessed by one or more of the plurality of users based at least in part on: a storage location of the previously uploaded data within the distributed network, a geographic location from which the previously uploaded data was uploaded, and the access information regarding access to the previously uploaded data by one or more of the plurality of users; and The one or more processors are directed to transmit the upload data from the first data center to one or more second data centers located at each of the one or more predicted geographic regions.
2. The method according to claim 1, wherein: The access information is derived from a prediction model trained based on metadata of the previously uploaded data.
3. The method according to claim 2, wherein: The prediction model is a decision tree model.
4. The method according to claim 2, wherein: The metadata of the previously uploaded data includes an identification of a user who uploaded the previously uploaded data, and wherein the one or more geographic areas of the plurality of geographic areas from which the uploaded data will be accessed by one or more of the plurality of users is predicted based at least in part on the identification of the user who uploaded the previously uploaded data.
5. The method according to claim 1, wherein: The metadata includes an identification of a user that uploaded the uploaded data, and wherein the one or more geographic areas of the plurality of geographic areas from which the uploaded data will be accessed by one or more of the plurality of users is predicted based at least in part on the identification of the user that uploaded the previously uploaded data.
6. The method according to claim 5, wherein: The previously uploaded data comprises one or more files, wherein the metadata of the previously uploaded data comprises file characteristic data of the one or more files, wherein the file characteristic data comprises at least one of: a name of the corresponding file; a size of the corresponding file; or an identification of a directory or file path where the corresponding file is stored, and wherein the one or more geographic areas of the multiple geographic areas from which the uploaded data will be accessed by one or more of the multiple users are predicted based at least in part on the file characteristic data.
7. The method according to claim 1, wherein: The access information indicates an amount of time between an initial upload of the previously uploaded data and a first download of the previously uploaded data, wherein the method comprises: predicting, by the one or more processors, an amount of time until the upload data is first downloaded based on the metadata and the access information; and Transmission of the upload data to one or more cache servers located at the one or more predicted geographic areas is directed by the one or more processors based on the predicted amount of time.
8. The method according to claim 7, wherein: Instructing the upload data to be transmitted to one or more cache servers located at the one or more predicted geographic areas further comprises: determining, by the one or more processors, that the uploaded data is broadcast data based on the metadata and the access information; and For each given predicted geographic area, the one or more processors instruct the upload data to be transmitted to at least one cache server in each data center of the given predicted geographic area.
9. The method according to claim 1, further comprising: directing, by the one or more processors, to include the uploaded data in a file, the file including previously uploaded data having a common predicted geographic area; as well as The one or more processors direct the file to be transmitted to one or more second data centers located at the commonly predicted geographic area.
10. The method according to claim 9, wherein: The file is initially stored at one or more source servers located at a first data center, and wherein instructing the file to be transferred includes instructing, by the one or more processors, data servers of the one or more second data centers located at the commonly predicted geographic area to retrieve the file from the one or more source servers.
11. A system for storing data in a distributed network having a plurality of data centers distributed over a plurality of geographic regions and accessed by a plurality of users, the data comprising metadata and accessible by the plurality of users, the metadata indicating one or more storage locations of the data within the distributed network and the corresponding geographic locations from which the data was uploaded, wherein: Each geographic area includes at least one user from the plurality of users, the system comprising: one or more storage devices at a first data center of the distributed network, the one or more storage devices being configured to store data uploaded to the first data center by a first user of the plurality of users, the uploaded data including metadata and accessible by the plurality of users; and One or more processors in communication with the one or more storage devices, the one or more processors configured to: receiving access information regarding access by the plurality of users to previously uploaded data previously stored in the plurality of data centers of the distributed network; predicting, before the uploaded data is accessed by any of the plurality of users, one or more of the plurality of geographic areas from which the uploaded data will be accessed by one or more of the plurality of users based at least in part on: a storage location of the previously uploaded data within the distributed network, a geographic location from which the previously uploaded data was uploaded, and the access information regarding access to the previously uploaded data by one or more of the plurality of users; and Instructing to transmit the upload data from the first data center to one or more second data centers located at each of the one or more predicted geographic regions.
12. The system according to claim 11, wherein: The access information is derived from a prediction model trained based on metadata of the previously uploaded data.
13. The system according to claim 12, wherein: The prediction model is a decision tree model.
14. The system according to claim 12, wherein: The metadata of the previously uploaded data includes an identification of a user who uploaded the previously uploaded data, and wherein the one or more processors are configured to predict, based at least in part on the identification of the user who uploaded the previously uploaded data, the one or more geographic locations of the plurality of geographic locations from which the uploaded data will be accessed by one or more of the plurality of users.
15. The system according to claim 11, wherein: The metadata comprises an identification of a user that uploaded the uploaded data, and wherein the one or more processors are configured to predict, based at least in part on the identification of the user that uploaded the previously uploaded data, the one or more geographic locations of the plurality of geographic locations from which the uploaded data will be accessed by one or more of the plurality of users.
16. The system of claim 15, wherein: The previously uploaded data comprises one or more files, wherein the metadata of the previously uploaded data comprises file characteristic data of the one or more files, wherein the file characteristic data comprises at least one of: a name of the corresponding file; a size of the corresponding file; or an identification of a directory or file path where the previously uploaded data is stored, and wherein the one or more processors are configured to predict, based at least in part on the file characteristic data, the one or more geographic areas of the multiple geographic areas from which the uploaded data will be accessed by one or more of the multiple users.
17. The system of claim 11, wherein: The access information indicates an amount of time between an initial upload of the previously uploaded data and a first download of the previously uploaded data, wherein the one or more processors are configured to: predicting an amount of time until the upload data is first downloaded based on the metadata and the access information; as well as Based on the predicted amount of time, transmitting the upload data to one or more cache servers located at the one or more predicted geographic areas is directed.
18. The system of claim 17, wherein: The one or more processors are configured to instruct transmission of the uploaded data to one or more cache servers located at the one or more predicted geographic areas based on a determination that the uploaded data is broadcast data based on the metadata, and Wherein for each given predicted geographic area, the one or more processors are configured to instruct the uploaded data to be transmitted to at least one cache server of each data center of the given predicted geographic area.
19. The system of claim 11, wherein: The one or more processors are configured to: instructing that the uploaded data be included in a file including previously uploaded data for a geographic area having a common prediction; and The file is directed to be transmitted to one or more second data centers located at the commonly predicted geographic area.
20. The system of claim 19, wherein: The file is initially stored at one or more source servers located at a first data center, and wherein the one or more processors are configured to instruct data servers of the one or more second data centers located at the commonly predicted geographic area to extract the file from the one or more source servers.
