Object pushing method and apparatus, and related device

Abstract

The present application provides an object pushing method and apparatus, and a related device. The method is applied to a service layer of an object storage system, and comprises: receiving an object subscription request of a first client for a target storage bucket, wherein the object subscription request carries a subscription topic; in response to the object subscription request, establishing a first transmission channel with the first client; then, on the basis of the subscription topic, obtaining a plurality of newly-added objects in the target storage bucket, wherein the plurality of newly-added objects are stored in at least one object storage device in a distributed mode; and finally, pushing the plurality of newly-added objects in the target storage bucket to the first client by means of the first transmission channel. In this way, a service layer can actively push newly-added objects in a storage bucket to a client subscribing to the storage bucket, and the client does not need to repeatedly send an object acquisition request to the service layer to obtain the newly-added objects, thereby reducing the network overhead. The service layer actively pushes the newly-added objects to the client by means of a transmission channel, thereby improving the real-time performance of object acquisition.

Description

Object push method, device and related equipment

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 17, 2023, with application number 202310876361.1 and application name “A method, device and other equipment for object access”, and claims priority to the Chinese patent application filed with the China Patent Office on November 15, 2023, with application number 202311524185.1 and application name “A method, device and related equipment for object push”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of storage technology, and in particular to an object push method, apparatus, and related equipment. Background Art

[0003] With the development of computer technology, cloud storage services have gained widespread adoption. Object storage services (OBS) are a type of cloud storage service. In an OBS, users can establish a connection with the OBS layer through an object storage client. Based on this connection, users can store objects in the OBS layer and retrieve objects already stored there.

[0004] In some scenarios, if objects stored in the object storage service layer change, the object storage client may need to retrieve the changed objects. To retrieve the changed objects, the object storage client can periodically establish a connection with the object storage service layer and send object retrieval requests through the connection. The object storage service layer can then send the changed objects to the object storage client based on the object retrieval requests. However, this method relies on the client sending requests to the service layer, which leads to high network overhead and the real-time nature of newly acquired objects.

[0005] Summary of the Invention

[0006] In view of this, embodiments of the present application provide an object push method for reducing client network overhead and improving the efficiency of client data acquisition. The present application also provides a corresponding apparatus, a computing device cluster, a computer-readable storage medium, and a computer program product.

[0007] In a first aspect, the present application provides an object push method that can be applied to the service layer of an object storage system. The object storage system also includes a client and at least one storage device. The service layer of the object storage system can access the at least one object storage device. Different object storage devices can be located in multiple data centers in different regions, and each data center can include multiple servers. When executing the object push method, the service layer can receive an object subscription request. The object subscription request is a request sent by a first client for a target bucket and can also include a subscription topic, indicating that the user of the first client wants to obtain newly added objects in the target bucket related to the subscription topic. In response to the object subscription request, the service layer can establish a first transmission channel with the first client. The first transmission channel can be used to push the object to the first client after the service layer detects an object uploaded to the target bucket. After establishing the first transmission channel, the service layer can monitor the target bucket for newly added objects that match the subscription topic. If so, the service layer can obtain the newly added objects in the target bucket based on the subscription topic. These newly added objects can be distributed and stored in the at least one object storage device. The service layer then pushes the obtained multiple newly added objects to the first client via the first transmission channel. In other words, if a user wants to retrieve the latest objects added to a bucket related to a subscribed topic, they can send an object subscription request for that bucket through the client. The service layer establishes a transmission channel based on the object subscription request and, when objects corresponding to the subscribed topic are added to the bucket, pushes them to the client through the transmission channel. This allows the client to retrieve objects in the subscribed topic from the bucket by simply sending an object subscription request, eliminating the need for the client to repeatedly send object retrieval requests to the service layer, thus reducing network overhead. Furthermore, the service layer proactively pushes newly added objects to the client through the transmission channel, allowing the client to retrieve newly added objects in the target bucket more quickly, improving the real-time nature of newly added object retrieval.

[0008] In some possible implementations, the first transmission channel can be established based on a persistent connection between the service layer and the first client. Specifically, the first client can first establish a short connection with the service layer, and then send an object subscription request to the service layer based on the short connection. The service layer can then establish a persistent connection based on the short connection to establish the first transmission channel. For example, the first transmission channel can be established based on the Websocket protocol. The object subscription request can be sent based on a Hypertext Transport Protocol (HTTP) connection between the first client and the service layer, or based on a Hypertext Transfer Protocol Secure (HTTPS) connection between the first client and the service layer. Accordingly, after receiving the object subscription request, the service layer can switch the HTTP connection or HTTPS connection between the first client and the server to a Websocket connection. In this way, object push is implemented based on a persistent connection, so that the first transmission channel will not be disconnected due to no data transmission, avoiding the first client from frequently establishing transmission channels with the service layer, and reducing network overhead.

[0009] In some possible implementations, the service layer can push either the newly added object itself or the access domain name of the newly added object to the first client. The access domain name of the newly added object is used to access or download the newly added object from the object storage device. Thus, after obtaining the access domain name of the newly added object, the first client can view or download the newly added object using the access domain name. Transmitting the domain name of the newly added object rather than the object itself reduces the amount of data required for transmission, improves data transmission efficiency, and reduces network overhead.

[0010] In some possible implementations, the subscription topic may include at least one of the following: the client identifier to which the multiple newly added objects belong, the category to which the multiple newly added objects belong, the directory where the multiple newly added objects are located, the save time of the multiple newly added objects, and the storage type of the multiple newly added objects. Among them, the client identifier to which the newly added objects belong is used to identify the client that uploads the newly added objects to the object storage system. The category described in the newly added object can be used to identify the data type of the newly added object, or to identify other types of newly added objects. The directory where the newly added object is located can be determined based on the storage address of the newly added object in the object storage system. The save time of the newly added object can be used to indicate the time when the newly added object is uploaded to the object storage system. The storage type of the newly added object can be used to indicate the access frequency of the multiple newly added objects. In this way, the service layer supports pushing new objects corresponding to one or more subscription topics, thereby improving the flexibility of object push.

[0011] In some possible implementations, the service layer can determine whether the target bucket has permission to open the first transmission channel before establishing the first transmission channel. Specifically, the service layer can determine whether the target bucket has permission to open the first transmission channel based on the configuration information of the first client. In this way, bucket management can be better implemented based on the client's configuration information.

[0012] In some possible implementations, multiple newly added objects in a target bucket may originate from different storage devices in different regions. Before pushing the newly added objects to a client via a transmission channel, the service layer may first synchronize the target bucket so that the target bucket stores the same newly added objects across different storage devices in different regions. Specifically, assume that the newly added objects in the target bucket include a first object and a second object. The first object is stored on a first object storage device located in a first region, and the second object is stored on a second object storage device located in a second region. If the service layer pushes the newly added objects to a first client via a first transmission channel, the service layer may first synchronize the second object from the second object storage device to the first object storage device, and then push the first and second objects in the first storage device to the first client via the first transmission channel, thereby pushing the multiple newly added objects in the target bucket to the first client. If the service layer pushes the newly added objects to a second client located in a second region via a second transmission channel, the service layer may first synchronize the first object from the first object storage device to the second object storage device, and then push the first and second objects in the second storage device to the second client via the first transmission channel, thereby pushing the multiple newly added objects in the target bucket to the second client. By synchronizing objects in the object storage devices corresponding to the target buckets in different regions, the target buckets can store the same objects on the object storage devices in each region. Clients in each region can retrieve the same objects from the object storage devices in their region. This enables cross-region transfer and synchronization of objects.

[0013] In a second aspect, the present application provides an object push device, which is applied to a service layer of an object storage system, the service layer being used to access at least one object storage device, the at least one object storage device being located in multiple data centers in different regions, each data center including multiple servers, the device comprising:

[0014] A receiving module, configured to receive an object subscription request from a first client for a target storage bucket, where the object subscription request carries a subscription topic;

[0015] a channel establishment module, configured to establish a first transmission channel with the first client in response to the object subscription request, wherein the first transmission channel is configured to push the object to the first client after the service layer detects that the target storage bucket has uploaded the object;

[0016] An object acquisition module, configured to acquire a plurality of newly added objects in a target storage bucket according to a subscription topic, wherein the plurality of newly added objects are distributed and stored in at least one object storage device;

[0017] The push module is configured to push the multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

[0018] In some possible implementations, the first transmission channel is established based on the WebSocket protocol, and the channel establishment module is specifically used to switch the HTTP connection established between the first client and the service layer to a WebSocket connection in response to the object subscription request, so as to establish a WebSocket transmission channel with the first client.

[0019] In some possible implementations, the push module is specifically configured to push the access domain names of the multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

[0020] In some possible implementations, the subscription topic includes at least one of the following: a client identifier to which the multiple newly added objects belong, a category to which the multiple newly added objects belong, a directory where the multiple newly added objects are located, a storage time of the multiple newly added objects, or a storage type of the multiple newly added objects, where the storage type is used to indicate an access frequency of the multiple newly added objects.

[0021] In some possible implementations, the channel establishing module is further configured to determine a target storage bucket and open the first transmission channel according to the configuration information of the first client.

[0022] In some possible implementations, the multiple newly added objects in the target storage bucket include a first object and a second object, the first object is stored in a first object storage device located in the first area, and the second object is stored in a second object storage device located in the second area. The device also includes a synchronization module, which is used to synchronize the first object from the first object storage device to the second object storage device; the push module is also used to push the multiple newly added objects in the target storage bucket to a second client through a second transmission channel, and the second client is set in the second area.

[0023] In a third aspect, the present application provides a computing device cluster, wherein the computing device includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory; the at least one memory is used to store instructions, and the at least one processor executes the instructions stored in the at least one memory, so that the computing device cluster executes the object push method in the above-mentioned first aspect or any possible implementation of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The at least one computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on at least one computing device, the at least one computing device executes the method described in the first aspect or any implementation of the first aspect.

[0025] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0026] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0028] FIG1 is a schematic diagram of at least one application scenario provided by an embodiment of the present application;

[0029] FIG2 is a schematic diagram of an architecture of an object storage system provided in an embodiment of the present application;

[0030] FIG3a is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0031] FIG3 b is an interactive diagram of the object push method provided in an embodiment of the present application;

[0032] FIG3c is a schematic diagram of another application scenario provided by an embodiment of the present application;

[0033] FIG4 is a schematic diagram of an implementation method of the first client provided in an embodiment of the present application;

[0034] FIG5 is a schematic diagram of an implementation method for establishing a first transmission channel provided in an embodiment of the present application;

[0035] FIG6a is a schematic diagram of another application scenario provided in an embodiment of the present application;

[0036] FIG6 b is another interactive diagram of the object push method provided in an embodiment of the present application;

[0037] FIG7 is a schematic structural diagram of an object pushing device provided in an embodiment of the present application;

[0038] FIG8 is a schematic diagram of a structure of a computing device provided in an embodiment of the present application;

[0039] FIG9 is a schematic diagram of a structure of a computing device cluster provided in an embodiment of the present application;

[0040] FIG10 is a schematic diagram of an implementation method of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will describe the solutions in the embodiments provided in this application in conjunction with the drawings in this application.

[0042] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate and are merely used to describe the manner in which objects with the same attributes are described in the embodiments of this application.

[0043] First, some of the terms involved in the embodiments of this application and the application scenarios of the object storage service are introduced.

[0044] (1) Object Storage Service (OBS): The object storage service is an object-based storage service that offers the advantages of massive capacity, security, high reliability, and low cost. OBS is an Internet-accessible service. Tenants can establish a connection with the object storage service layer (a computing node that supports OBS) through the object storage client, create buckets in the storage nodes managed by the object storage service layer, and then access and manage the objects in the buckets. By default, the object storage service layer uses a sequential distribution method to store objects in buckets. The sequential distribution method can also be called a lexicographic distribution or a range distribution.

[0045] (2) Object: An object is the basic unit of data storage in an object storage system. An object is actually a collection of a file's data and its related attribute information (metadata). Data uploaded by tenants to OBS is stored in buckets as objects. Objects consist of three parts: a key, metadata, and data. The key, or object name, is a UTF-8-encoded character sequence with a length greater than 0 and no more than 1024. Each object in a bucket has a unique object key.

[0046] (3) Bucket: A bucket is a container for storing objects in OBS and can also be called a storage bucket or data bucket. Object storage provides a flat storage method based on buckets and objects. All objects in a bucket are at the same logical level, eliminating the multi-level tree directory structure in the file system. Each bucket has its own storage category, access rights, region, and other attributes. Tenants can create buckets with different storage categories and access rights and configure more advanced attributes to meet the storage requirements of different scenarios.

[0047] (4) Object storage device (OSD): OSD is the basic storage unit of the object storage system, which is set on the physical disk. Specifically, it is a fixed-size storage space of the physical disk. The object storage system manages the physical disks of multiple computing devices in the form of OSD.

[0048] Tenants can log in to the cloud management platform on the public cloud access page using a pre-registered account and password, and after successful login, select and purchase corresponding public cloud services on the public cloud access page, such as OBS, virtual machine services, container services, etc. For OBS, tenants can further create multiple buckets through the configuration interface or application programming interface (API) on the public cloud access page provided by the cloud platform. There is no limit on the number and total size of objects stored in each bucket, and tenants do not need to consider the scalability of data. OBS is a service based on the Representational State Transfer API (REST API) and the Hypertext Transport Protocol (HTTP) and Hypertext Transfer Protocol Secure (HTTPS). Tenants can locate bucket resources through the Uniform Resource Locator (URL), also referred to as the access domain name in this application, or simply the domain name. Among them, in OBS, the bucket name is globally unique and cannot be modified, that is, the bucket created by a tenant cannot have the same name as other buckets created by the tenant, nor can it have the same name as buckets created by other tenants.

[0049] Figure 1 shows a schematic diagram of at least one application scenario in an embodiment of the present application. As shown in Figure 1, each bucket can include multiple objects, and the objects between buckets are isolated from each other. The tenant logs in to the cloud management platform through the object storage client, selects and purchases the cloud service of the object storage service on the cloud management platform. After the purchase, the tenant can store objects based on the functions provided by the object storage service. Among them, the cloud management platform is mainly used to manage the infrastructure for running the object storage service. Exemplarily, the infrastructure for running the object storage service may include multiple data centers set up in different regions, and each data center includes multiple servers. The data center can provide basic resources for the object storage service, such as computing resources, storage resources, etc. Therefore, when purchasing and using the object storage service, the tenant mainly pays for the resources used. Specifically, the object storage service provides the domain name of the bucket, and the tenant can access the domain name of the bucket through the object storage client to upload data to the bucket or download data from the bucket, wherein the uploaded data is stored in the bucket in the form of objects.

[0050] Figure 2 shows a schematic diagram of the architecture of an object storage system in an embodiment of the present application. The object storage system includes an object storage client, an object storage service layer, and an object storage device.

[0051] The object storage client can be installed on a terminal device, such as a mobile phone, laptop, tablet, PDA, wireless terminal in a smart city, or wireless device in a smart home. Tenants log in to the cloud management platform through the client and select and purchase object storage cloud services on the cloud management platform.

[0052] The object storage service layer manages one or more object storage devices and includes one or more servers, which can be located in the service devices of the service layer. Exemplarily, the service devices connect to the storage devices via a switch. In another exemplary embodiment, the service devices connect to the terminal devices via a remote connection gateway.

[0053] Tenants log in to the object storage service layer using their accounts on the client. In the service layer, they select and purchase object storage cloud services, such as creating buckets and configuring bucket names. After the service layer detects the tenant's operation (such as creating a bucket), it can send a creation command to the object storage device. The creation command includes information such as the bucket name and domain name. The creation command is used to notify the storage device to create a bucket and save the bucket name, bucket domain name, and other information. After the bucket is created, the tenant can access the bucket domain name through the object storage client, locate the bucket on the storage device, and upload, download, and delete objects from the bucket.

[0054] The service layer includes a control unit and an operating system. The control unit runs on the operating system, which includes a disk driver and a physical network card driver. The control unit controls the disk controller through the disk driver to set the physical disk as multiple persistent storage units (persistence log, PLOG).

[0055] The infrastructure for running the object storage service can be configured with multiple data centers in different regions. For example, object storage devices are configured in multiple data centers in different regions, with each data center including multiple object storage devices. Each object storage device includes at least one physical disk. For example, the disk controller configures physical disk 1 and physical disk 2 in region 1 as four PLOGs.

[0056] After receiving the creation instruction for bucket 1, the service layer notifies the control unit to create bucket 1. The control unit creates bucket 1 on physical disk 1 and physical disk 2 in the object storage device through the operating system. The PLOG of bucket 1 is distributed on physical disk 1 and physical disk 2, and the bucket name, bucket domain name and other information of bucket 1 are saved.

[0057] When a tenant needs to upload an object to bucket 1 through a client, the tenant can trigger the object storage device through the client to write the object to bucket 1. For example, the control unit in the object storage service layer can write the object to bucket 1.

[0058] When a tenant needs to download an object from bucket 1 through a client, the tenant can trigger the object storage device through the client to read the object from bucket 1. For example, a control unit in the service layer can obtain the object from bucket 1 and feed it back to the client.

[0059] In some scenarios, clients may need to retrieve newly added objects in the service layer. These newly added objects are referred to as newly added objects. Optionally, these newly added objects can come from a specific bucket. Users can retrieve newly added objects from a specific bucket. For example, in scenarios such as data subscriptions, live broadcasts, and maps, objects stored in a bucket may change, and users may need to retrieve the latest updated objects in the bucket.

[0060] To meet this requirement, the client can actively send a request to the service layer to obtain the newly added objects in the bucket. Specifically, a polling mechanism can be deployed in the client. The client with the polling mechanism deployed can periodically establish a connection with the service layer and send an object query request to the service layer. The object query request can include a bucket identifier, which is used to query whether there is a newly added object in the bucket corresponding to the bucket identifier. If there is a newly added object in the bucket, the service layer can obtain the newly added object and send the newly added object to the client through the connection with the client. In this way, if an object is added to the bucket, the client can obtain the newly added object in the bucket after sending the object query request next time.

[0061] However, although the above method can obtain newly added objects, the object acquisition request for obtaining the newly added objects is actively sent by the client, and the connection for transmitting the newly added objects is also actively requested to be established by the client. Therefore, in order to obtain the newly added objects in a timely manner, the client needs to frequently establish a connection with the service layer and send object acquisition requests to the service layer. In this way, frequently establishing connections and sending requests requires more network resources, and there is a problem of high network overhead. In particular, if the service layer is used to provide services to multiple server terminals, and multiple server terminals have the need to obtain newly added objects, then the service layer may need to establish connections with multiple clients and receive multiple object query requests within a unit of time. This will significantly increase the pressure on the service layer and may affect the normal operation of the service layer.

[0062] In addition, since the action of obtaining newly added objects is initiated by the client, and the client cannot determine when the object is added to the bucket, the client may not be able to obtain the newly added objects in a timely manner, resulting in poor timeliness of object acquisition.

[0063] For example, suppose a new object is added to bucket 1 at time t1. The client establishes a connection with the service layer at time t2 (time t2 is later than time t1) and sends a request to the service layer to retrieve the object in bucket 1. The service layer can then receive the object request at the earliest possible time t2 and send the new object in bucket 1 to the client. The client can then retrieve the new object in bucket 1 at the earliest possible time t2. In this process, there is a time difference of at least (t2 - t1) between the appearance of the new object and the client's retrieval of the new object.

[0064] From the above introduction, it can be seen that since the service layer lacks the means to actively push new objects to the client, the client will incur a large network overhead to obtain the new objects and cannot obtain the new objects in a timely manner.

[0065] Based on this, an embodiment of the present application provides an object push method. This method can be implemented by an object storage service layer in an object storage system. By proactively pushing newly added objects, the method improves the efficiency of client access to newly added objects and reduces network overhead. Specifically, when executing the object push method, the service layer can first receive an object subscription request from a first client for a target bucket. The object subscription request carries a subscription topic, which is used to subscribe to newly added objects in the target bucket that match the subscription topic. Based on the object subscription request, a first transmission channel can be established between the service layer and the first client. The first transmission channel is used to push the object to the first client after the service layer detects an object uploaded to the target bucket. After establishing the first transmission channel, the service layer can retrieve multiple newly added objects in the target bucket based on the subscription topic. The multiple newly added objects can be distributed and stored in at least one object storage device in the object storage system. After retrieving the multiple newly added objects, the service layer can push the multiple newly added objects to the first client via the first transmission channel. In other words, if a user wants to obtain the latest newly added objects in a bucket related to the subscription topic, the user can send an object subscription request for that bucket through the client. The service layer establishes a transmission channel based on object definition requests and, after adding objects corresponding to a subscribed topic to a bucket, pushes them to the client via the transmission channel. This allows the client to retrieve objects in the subscribed topic from the bucket simply by sending an object subscription request, eliminating the need for the client to repeatedly send object retrieval requests to the service layer, thus reducing network overhead. Furthermore, the service layer proactively pushes newly added objects to the client via the transmission channel, allowing the client to retrieve newly added objects in the target bucket more quickly and improving the real-time nature of newly added object retrieval.

[0066] Next, various non-limiting specific implementations of the object push process are described in detail.

[0067] First, the application scenarios of the object push method are introduced.

[0068] Refer to Figure 3a, which is a schematic diagram illustrating an application scenario of the object push method provided in an embodiment of the present application. In the application scenario shown in Figure 3a, the object storage system includes a service layer, an object storage device, client A, and client B. Client A can upload objects to the service layer. The service layer can proactively push the objects uploaded by client A to client B. In addition, client B can also send an object subscription request to the service layer to establish a first transmission channel between the service layer and client B. The first transmission channel can be used to transmit objects proactively pushed by the service layer to client B.

[0069] Refer to Figure 3b, which is a flow chart of an object push method provided in an embodiment of the present application. This method can be applied to the application scenario shown in Figure 3a above, or it can also be applied to other applicable application scenarios. The following description uses the application scenario shown in Figure 3a as an example. Optionally, the object push method shown in Figure 3b can be executed by the service layer in Figure 3a.

[0070] Specifically, the object push method shown in FIG3b may include:

[0071] Step S301: The first client sends an object subscription request for a target bucket to the service layer.

[0072] In embodiments of the present application, the service layer can proactively push newly added objects to the client. In some implementations, this proactive push functionality can be enabled by the client. Specifically, if a user using a first client desires that the service layer proactively push newly added objects to the client, the user can send an object subscription request to the service layer through the first client. The object subscription request is used to request that the service layer proactively push the newly added objects to the client. Optionally, the first client can be client B in the user scenario shown in FIG3a.

[0073] As previously described, the service layer of an object storage system may include one or more servers. If the service layer of the object storage system includes multiple servers, the first client may send an object subscription request to one of the multiple servers. For ease of description, the server to which the first client sends the object subscription request is referred to as the first server.

[0074] Optionally, the first client may send an object subscription request to the server according to the region in which the first client is located. That is, the first client and the first server may be in the same region.

[0075] In a real-world scenario, an object storage system may include multiple buckets, each capable of storing objects. The newly added objects that a client desires to obtain may be from one or more specific buckets. Therefore, the object subscription request sent by the client to the service layer may be a request for a specific bucket, intended to retrieve newly added objects in that specific bucket. In the implementation illustrated in FIG1 , a user using a first client desires to retrieve newly added objects in a target bucket. The first client may send an object subscription request for the target bucket to the service layer.

[0076] In some implementations, a client may need to retrieve newly added objects from multiple buckets. Accordingly, the client can send multiple object subscription requests, each corresponding to a specific bucket. Alternatively, an object subscription request can be sent to multiple buckets to subscribe to newly added objects in multiple buckets. For ease of explanation, the following example uses an object subscription request sent by a first client to subscribe to newly added objects in a target bucket.

[0077] In an embodiment of the present application, the target bucket can be a logical bucket. Specifically, objects in the target bucket can be stored in multiple object storage devices. Furthermore, the target storage devices storing the objects in the target bucket can be located in the same region or in different regions. For example, the target bucket can store object A, object B, and object C, where object A can be stored in a first storage device in a first region, object B can be stored in a second storage device in a second region, and object C can be stored in a third storage device in the second region.

[0078] In an embodiment of the present application, the object subscription request may include an identifier of the target bucket, indicating that the object subscription request is a request for the target bucket, and is used to request to obtain the newly added information in the object bucket. The identifier of the target bucket may include, for example, the bucket name of the target bucket. Optionally, the identifier of the target bucket may also include information such as the access domain name of the target bucket. The object subscription request may also include a subscription topic. The subscription topic is used to indicate the newly added objects that the first client needs to obtain. That is, the target bucket may include a variety of objects with different attributes, and the first client may need to obtain one or more types of newly added objects. Accordingly, the subscription topic may be used to describe the relevant information of the newly added objects that the first client needs to obtain.

[0079] Optionally, a subscription topic can include at least one of the following: the client identifier of the newly added objects, the category to which the newly added objects belong, the directory where the newly added objects reside, the storage time of the newly added objects, and the storage type of the newly added objects. This allows the service layer to push newly added objects corresponding to one or more subscription topics, enhancing the flexibility of object push. These are described below.

[0080] (1) The client ID to which the new object belongs.

[0081] In real-world scenarios, multiple clients may share the same bucket. Consequently, multiple clients may have permission to upload objects to the target bucket. Consequently, newly added objects to the target bucket may originate from multiple clients.

[0082] The client ID of a newly added object identifies the client to which the newly added object belongs. The client to which the newly added object belongs is the client that uploaded the newly added object to the target bucket. For example, if client A uploaded object a to bucket 1, and client B uploaded object b to bucket 1, the client ID of newly added object a identifies client A, and the client ID of newly added object b identifies client B.

[0083] If the subscription topic includes the client identifier of the newly added object, it indicates that the user of the first client wants to retrieve newly added objects uploaded by a specific client. For example, if the user wants to retrieve newly added objects uploaded by client A, the subscription topic can include the corresponding identifier of client A. In this way, based on the subscription topic, the service layer can send object a to the first client. Since the subscription topic does not include the client identifier of object b, the service layer does not send object b to the first client.

[0084] (2) The category to which the new object belongs.

[0085] In practical scenarios, a bucket can be used to store objects of various categories. The object category can refer to the object type, for example, the object's data format. Accordingly, a user using a first client may want to obtain newly added objects of a specific category. Accordingly, the subscription topic in the object subscription request sent by the first client can include information related to the specific category.

[0086] For example, suppose bucket 1 can store objects in text format, audio format, and video format. However, client A may only want to retrieve newly added text-format objects in bucket 1. Accordingly, the subscription topic in the object subscription request sent by client A to the service layer can include the category to which the newly added object belongs, and this category is used to indicate text-format objects.

[0087] (3) The directory where the new object is located.

[0088] In real-world scenarios, a bucket may contain multiple objects. Users can create directories within the bucket to manage stored objects. Accordingly, when storing objects in a bucket, users can specify specific directories. For example, users can store multiple content-related objects in the same directory. Optionally, the directory containing the newly added object can include the directory where the newly added object is stored in the target bucket.

[0089] If the subscription topic includes the directory where the newly added object is located, it indicates that the user using the first client wants to obtain newly added objects under a specific directory in the target bucket. For example, if client A uploads objects a1 and a2 to bucket 1, object a1 is uploaded to directory M1 in bucket 1, and object a2 is uploaded to directory M2 in bucket 1. The subscription topic in the object subscription request sent by the first client can include information about directory M1. Based on the subscription topic, the service layer can then send object a1 to the first client. Since the subscription topic does not include information about directory M2, the service layer does not send object a2 to the first client.

[0090] (4) The storage time of the newly added object.

[0091] In real-world scenarios, objects in a bucket can have varying storage durations. A user using the first client may want to retrieve data stored in the target bucket within a specific time period. Accordingly, the subscription topic in the object subscription request sent by the first client can include the storage duration of newly added objects. The storage duration indicates the user-selected time period, indicating that the first client wants to retrieve newly added objects stored in the target bucket within that time period.

[0092] (5) Add storage type of new objects.

[0093] An object's storage type can be used to indicate its access frequency. Specifically, in real-world scenarios, multiple objects stored in a bucket may be accessed at different frequencies. For example, some objects may be accessed more frequently, others less frequently, or some objects may be used for backup or archiving and are less likely to be accessed. Alternatively, an object's storage type can be determined based on its access frequency. Accordingly, a subscription topic can include the storage type of newly added objects, which indicates their access frequency.

[0094] It is understandable that the above five types of subscription topics are shown only for reference. In actual scenarios, the object subscription request may also include other types of subscription topics. For example, the subscription topic may also include a select-all topic, indicating that the user needs to obtain all newly added objects in the target bucket. When the subscription topic of the object subscription request is a select-all topic, the service layer may push each newly added object in the target bucket to the first client. Optionally, the above select-all topic may be the default value of the subscription topic. That is, if the user does not set the subscription topic of the object subscription request on the first client, the subscription topic in the object subscription request sent by the first client may be a select-all topic.

[0095] The above describes the target bucket and subscription topic. The following describes how to determine the target bucket and subscription topic.

[0096] In an embodiment of the present application, the object subscription request sent by the first client is for a target bucket in the object storage system and also includes a subscription topic. Optionally, the target bucket and subscription topic can be selected by the user on the first client.

[0097] For example, see Figure 4, which is a schematic diagram of an implementation of a first client provided in an embodiment of the present application. Specifically, in the implementation corresponding to Figure 4, if a user using the first client wishes to send an object subscription request to the service layer through the first client, the first client may display a display area 400 as shown in Figure 4. Display area 400 includes a push control 410, a bucket setting control 420, a subscription topic setting control 430, and an confirmation control 440.

[0098] Among them, push control control 410 can be used to enable the active push function of the service layer. In the implementation corresponding to Figure 4, push control control 410 includes sub-control 411. When sub-control 411 is selected, it indicates that the user has enabled the active push function of the service layer. The first client can send an object subscription request to the service layer. It is understood that if sub-control 411 is not selected, the request sent by the first client to the service layer may not be an object subscription request.

[0099] Bucket settings control 420 is used to obtain a bucket specified by the user, i.e., a target bucket. Optionally, bucket settings control 420 may include an input box in which the user can enter relevant information about the target bucket, such as the target bucket name. Alternatively, bucket settings control 420 may include a drop-down menu from which the user can select the target bucket.

[0100] The subscription topic setting control 430 is used to obtain the subscription topic determined by the user. Optionally, the subscription topic setting control 430 may include an input box in which the user can enter the subscription topic. Alternatively, the subscription topic setting control 430 may also include a drop-down menu in which the user can select the subscription topic.

[0101] The confirmation control 440 is used to trigger the request sending operation, that is, to trigger the first client to send an object subscription request to the service layer. In other words, after the user determines the target bucket and subscription topic, the user can trigger the confirmation control 440 to control the first client to send the object subscription request to the service layer.

[0102] In an embodiment of the present application, the service layer can establish a transmission channel with the first client based on the object subscription request and proactively push newly added objects in the target bucket to the first client. In some possible implementations, the service layer may only support proactively pushing newly added objects in certain buckets to the client. Accordingly, the service layer can support proactively pushing newly added objects in the target bucket to the client.

[0103] Optionally, the attribute information of a bucket in the object storage system may include push attribute information. The push attribute information is used to indicate whether the service layer can proactively push newly added objects in the bucket to the client. Accordingly, the push attribute information of the target bucket can indicate that the service layer can proactively push newly added objects in the target bucket to the client. It is understandable that, assuming that the push attribute information of bucket 1 indicates that the service layer cannot proactively push newly added objects in bucket 1 to the client, then if the service layer receives an object subscription request for bucket 1, the service layer can reject the object subscription request.

[0104] Optionally, push attribute information can be configured when creating a bucket. For example, a user can enable the target bucket to proactively push newly added objects to the client on the target bucket creation page. Alternatively, push attribute information can be modified by the client. For example, a user with permission to modify the target bucket can modify the target bucket's push attribute information through the service layer.

[0105] In an embodiment of the present application, the service layer may establish a first transmission channel with the first client according to the object subscription request. Before establishing the first transmission channel, the first client may first establish a connection with the service layer so as to send the object subscription request to the service layer through the connection.

[0106] Optionally, the first transmission channel can be established based on a persistent connection between the first client and the service layer. The first client can first establish a short connection with the service layer to send an object subscription request to the service layer via the short connection. Optionally, the short connection between the first client and the service layer can include, for example, a short connection established over HTTP or HTTPS.

[0107] It is understandable that the connection between the first client and the service layer may be a connection between the first client and the first server.

[0108] Step S302: The service layer establishes a first transmission channel with the first client in response to the object subscription request.

[0109] After receiving the object subscription request sent by the first client, the service layer may establish a first transmission channel with the first client according to the object subscription request, wherein the first transmission channel is used to transmit newly added objects related to the subscription topic in the target bucket.

[0110] Before establishing the first transmission channel, the service layer may first determine whether the first transmission channel can be established with the client. Optionally, the service layer may determine whether the first transmission channel can be established based on the configuration information of the target bucket and / or the configuration information of the first client. For ease of subsequent description, in an embodiment of the present application, the service layer may determine whether the target bucket can open the first transmission channel based on the configuration information of the target bucket and / or determine whether the target bucket can open the first transmission channel based on the configuration information of the first client.

[0111] Optionally, the service layer can determine whether a transmission channel for actively pushing new objects to the first client can be established based on the configuration information of the target bucket. Specifically, the object storage system may include multiple buckets, some of which may support pushing new objects to the client, and some may not support pushing new objects to the client. Before establishing the first transmission channel, the service layer can first determine whether the target bucket supports pushing new objects to the client. For example, if the configuration information of the target bucket can be used to indicate that the new objects in the target bucket can be actively pushed to the client through the transmission channel, the service layer can establish the first transmission channel; if the configuration information of the target bucket can be used to indicate that the new objects in the target bucket cannot be actively pushed to the client through the transmission channel, the service layer can refuse to establish the first transmission channel.

[0112] Optionally, the service layer can also determine whether the first transmission channel can be established based on the configuration information of the first client. Specifically, some clients may not be able to establish a transmission channel for the service layer to actively push new objects, so the service layer can refuse to establish a transmission channel for such clients. For example, the service layer can determine whether the first client has subscribed to the cloud service that actively pushes new objects. If the first client has not subscribed to the cloud service that actively pushes new objects, the client can refuse to establish the first transmission channel. For another example, the service layer can also determine whether the network conditions of the first client are sufficient to establish the first transmission channel. If the network conditions of the first service layer are not sufficient to establish the first transmission channel, the service layer can refuse to establish the first transmission channel.

[0113] As described above, the first client can first establish a connection with the service layer and send an object subscription request to the service layer through the connection. Optionally, the first transmission channel can be implemented based on the connection used to transmit the object subscription request. For example, the first transmission channel can be a channel between the first client and the service layer for transmitting the object subscription request. Alternatively, the first client can first establish a short connection with the service layer, and the first transmission channel can be a long connection established based on the short connection. That is, after obtaining the object subscription request, the service layer can switch the short connection for transmitting the object subscription request to a long connection, and the first transmission channel can be implemented based on the long connection. Optionally, the above-mentioned long connection can be established based on the Websocket protocol.

[0114] The following description will be made with reference to FIG5 , using an example in which the short connection between the first client and the service layer is an HTTP connection and the long connection is a WebSocket connection. Referring to FIG5 , this figure is a schematic diagram of an implementation method for establishing a first transmission channel provided in an embodiment of the present application. Specifically, the method for establishing the first transmission channel shown in FIG5 may include:

[0115] Step S501: An HTTP connection is established between the first client and the service layer.

[0116] Before establishing the first transmission channel, an HTTP connection may be established between the first client and the service layer. Specifically, the first client may send a connection establishment request to the first server, where the connection establishment request is used to request establishment of an HTTP connection. The first server may send a connection establishment response to the first client based on the connection establishment request, thereby establishing the HTTP connection between the first client and the first server.

[0117] Optionally, the connection may be established by the first client before sending the object subscription request.

[0118] Step S502: The first client sends a connection upgrade request to the service layer.

[0119] The first client can send a connection upgrade request to the service layer via an HTTP connection between the first client segment and the service layer. Alternatively, the first client can send a connection upgrade request to the first server in the service layer. The connection upgrade request is used to request an upgrade of the HTTP connection to a WebSocket connection. Optionally, the connection upgrade request can include an upgrade flag and a WebSocket flag. The connection upgrade request flag is used to request an upgrade of the current connection to a WebSocket connection.

[0120] Optionally, the connection upgrade request can be sent based on the object subscription response of the service layer. Specifically, after the first client sends the object subscription request to the service layer, the service layer can generate an object subscription response and send the object subscription response to the first client. The first client can generate a connection upgrade request based on the object subscription response and send it to the service layer. Optionally, if the first client does not have permission to subscribe to new objects of the subscription topic in the target bucket, and / or the service layer does not include the target bucket, and / or the target bucket is not used to store new objects corresponding to the subscription topic, the service layer may not send the object subscription response to the first client.

[0121] Step S503: The service layer sends a connection upgrade response to the first client.

[0122] After receiving the connection upgrade request, the service layer may send a connection upgrade response to the first client. Optionally, the first server in the service layer may send a connection upgrade response to the first client. The connection upgrade response may include upgrade status information, which indicates the status of the connection between the first client and the service layer, that is, indicating that the connection between the first client and the service layer has been switched from an HTTP connection to a WebSocket connection. Optionally, the upgrade status information may include status code 101. Optionally, before sending the connection upgrade response, the service layer may make a judgment. For example, it may determine whether the first client has permission to establish a WebSocket connection with the service layer.

[0123] It is understood that the short connection between the first client and the service layer can be a short connection between the first client and the first server, and the long connection between the first client and the service layer can be a long connection between the first client and the first server. The service layer in Figure 4 can be the first server.

[0124] After obtaining the connection establishment response, the first client completes the establishment of the Websocket connection with the service layer, that is, the establishment of the first transmission channel is completed.

[0125] Step S303: The service layer obtains multiple newly added objects in the target bucket according to the subscription topic.

[0126] After receiving the object subscription request, the service layer can monitor the newly added objects in the target bucket according to the subscription topic. If a new object appears in the target bucket, the service layer can obtain the newly added object according to the subscription request so as to send the multiple newly added objects to the first client through the first transmission channel. In the embodiment of the present application, the newly added object refers to the object stored in the service layer by the client. That is, the newly added object obtained by the service layer can be data stored in the target bucket by other users through clients other than the first client. Optionally, in the implementation corresponding to Figure 3a, the first client can be client B, and the newly added object can be the newly added object uploaded to the service layer by client A.

[0127] Optionally, the service layer obtaining the newly added object may include obtaining the newly added object itself from the target bucket, or may include obtaining the access domain name corresponding to the newly added object from the target bucket. For an introduction to the access domain name for the newly added object, please refer to the following section and will not be repeated here.

[0128] Specifically, after an object is added to the target bucket, the service layer can determine whether the newly added object matches the subscription topic. If the newly added object matches the subscription topic, the service layer can obtain the newly added object. If the newly added object does not match the subscription topic, the service layer may not obtain the newly added object. Optionally, the newly added objects obtained by the service layer can be distributed and stored in a single object storage device or in multiple object storage devices.

[0129] Optionally, if the first transmission channel is a transmission channel between the first client and the first server, the first server can obtain the target object in the target bucket, and the first server can obtain the newly added object from the object storage device in the area to which the first client belongs.

[0130] It is understood that as the object storage system operates, multiple newly added objects matching the subscription topic may appear in the target bucket, and therefore the service layer may retrieve multiple newly added objects. Each time an object matching the subscription topic is stored in the target bucket, the service layer may retrieve the newly added object from the target bucket based on the subscription topic. Alternatively, the service layer may retrieve multiple newly added objects at once, or a single newly added object at a time.

[0131] Step S304: The service layer pushes the multiple newly added objects in the target bucket to the first client through the first transmission channel.

[0132] After establishing the first transmission channel and obtaining the newly added objects, the service layer may push the multiple newly added objects in the target bucket to the client through the first transmission channel.

[0133] In an embodiment of the present application, the service layer can push the newly added object itself to the first client, and can also push the access domain name of the newly added object to the first client. The access domain name of the newly added object is used to indicate the storage address of the newly added object in the object storage system, for example, it can indicate the storage address of the newly added object in the object storage device. The access domain name of the newly added object is used to access or download the newly added object in the object storage device. In this way, after obtaining the access domain name of the newly added object, the first client can view or download the newly added object through the access domain name. In this way, the domain name of the newly added object is transmitted instead of the newly added object itself, which reduces the amount of data that needs to be transmitted, improves data transmission efficiency, and reduces network overhead.

[0134] Optionally, as the object storage system operates, one or more clients may store objects in the target bucket at different points in time. Accordingly, each time a new object is stored in the target bucket, the service layer can determine whether the newly added object matches the subscribed topic. If so, the service layer can retrieve the newly added object and send it to the first client via the first transmission channel. Specifically, the service layer can proactively push the newly added object to the first client via the first server.

[0135] Optionally, the first transmission channel can be implemented based on a persistent connection between the service layer and the first client. This persistent connection can persist after data transmission. Thus, after the server pushes one or more newly added objects to the first client, the first transmission channel can persist. If other clients store objects matching the subscribed topic in the target bucket, the service layer can also use the first transmission channel to push these newly added objects to the first client. This allows the first transmission channel to be continuously used to transmit newly added objects, reducing the network overhead associated with frequent connection establishment.

[0136] In an embodiment of the present application, the service layer can establish a transmission channel with the client based on an object creation request sent by the client, and proactively push the newly added objects required by the client to the client through this transmission channel. Thus, by establishing a transmission channel, the service layer of the object storage system has the ability to proactively push objects to the client of the object storage system. This not only eliminates the need for the client to frequently send requests to the service layer and query whether new objects exist, saving network resources, but also enables the service layer to push new objects to the client as soon as they appear, eliminating the need for the service layer to wait for client polling and improving the efficiency of obtaining new objects.

[0137] The object push method provided in the embodiment of the present application can also be applied to the field of big data, for example, it can be used to push data streams to a big data cluster that consumes streaming data.

[0138] For big data clusters consuming streaming data, message middleware can currently be used to input streaming data from data sources into the big data cluster for consumption. However, message middleware's ability to forward data is limited, which restricts the consumption capacity of the big data cluster and also poses a high pressure problem.

[0139] To this end, in an embodiment of the present application, streaming data can be uploaded to the service layer as an object, and a client can be configured for the big data cluster. In this way, the service layer can actively push new objects to the client to achieve the effect of actively pushing data streams to the big data cluster.

[0140] Specifically, refer to Figure 3c, which is a scenario diagram of an application scenario of the object push method provided in an embodiment of the present application. In the application scenario shown in Figure 3c, it includes a data source, a service layer of an object storage system, a server of an object storage system, and a big data cluster. Among them, the big data cluster includes a streaming consumption interface and an operator. The data source can upload the data stream to the service layer of the object storage system. The service layer of the object storage system can actively push the newly added objects to the client through the transmission channel between the client and the data source. In this way, the data from the data source will be pushed to the client as a new addition. The client can forward the newly added object to the streaming consumption interface of the big data cluster. Therefore, the big data cluster can obtain the data stream from the data source through the streaming consumption interface, so that the operator inside the big data cluster can perform corresponding data processing according to the data stream. In this way, the demand for big data cluster to consume streaming data is realized by replacing the traditional message middleware with the object storage system.

[0141] As previously explained, the service layer of an object storage system can include multiple servers and manage multiple object storage devices. Different object storage devices can be located in different regions or the same region. The object storage devices corresponding to target buckets can be located in the same region or different regions.

[0142] If the object storage device corresponding to the target bucket is located in a different region, the region where the first client is located and the region where the object storage device storing the newly added object is located can be the same region or different regions.

[0143] In some possible implementations, if the client storing the newly added object is in the same region as the first client, the newly added object is stored on the object storage device in the region where the first client is located. Since the storage device in the region where the first client is located stores the newly added object, the first server can obtain the newly added object from the object storage device in its region and send the newly added object to the first client.

[0144] In some other possible implementations, if the client storing the newly added object is in a different region than the first client, the newly added object is stored on an object storage device in a region other than the region where the first client resides. In this case, the object storage device in the region where the first client resides does not store the newly added object, making it inconvenient for the first client to retrieve the newly added object.

[0145] To this end, in an embodiment of the present application, if the objects in the target bucket can be stored in different object storage devices in different areas, the service layer can first synchronize the target bucket after the new object appears in the target bucket. Synchronizing the target bucket refers to synchronizing the contents of the target bucket stored in the object storage devices in different areas. For example, assuming that the object storage system has object storage devices deployed in both the first area and the second area, and the objects in the target bucket can be stored in both the first object storage device in the first area and the second object storage device in the second area. Then synchronizing the target bucket can include synchronizing the objects in the target bucket stored in the first object storage device to the second object storage device, and synchronizing the objects in the target bucket stored in the second object storage device to the first object storage device.

[0146] For example, suppose client A in Region 1 stores object X in a target bucket. Because client A is in Region 1, client A can interact with the service layer's server in Region 1. The server in Region 1 then stores object X in the target bucket's corresponding object storage device in Region 1, i.e., the first object storage device.

[0147] After object X is stored in the first object storage device, the service layer can synchronize the target bucket. Specifically, the service layer can first determine the object storage devices in other regions of the target bucket, and then determine whether object X is stored in the object storage devices in these other regions. That is, the service layer can determine the second object storage device based on the target bucket, and determine whether object X is stored in the second object storage device. If object X is not stored in the second object storage device, the service layer can store object X in the second storage device. In this way, object X stored in the target bucket in the first region will be synchronized to the second region by the service layer. Then the client in the second region can also obtain object X from the object storage device corresponding to the target bucket in the second region. In this way, by synchronizing the target bucket, it can be ensured that the target bucket stores the same objects in the object storage device in each region. Clients in each region can obtain objects in the target bucket from the object storage device in this region. Optionally, if the service layer synchronizes the object storage devices corresponding to the target bucket in different regions, the target bucket can also be called a global bucket, indicating that the target bucket stores the same objects in the object storage devices in each region.

[0148] After synchronizing the target bucket, the service layer can retrieve newly added objects from the target bucket. Specifically, the first server can retrieve the newly added objects from the object storage device corresponding to the target bucket in the region where the first client resides. In other words, if a client in another region stores an object in the target bucket, the service layer can synchronize the newly added objects to the object storage device in the region where the first client resides, and then the first server can retrieve the newly added objects from the object storage device.

[0149] The following is a detailed description with reference to the accompanying drawings.

[0150] First, the application scenario is introduced. Refer to Figure 6a, which is a scenario diagram of an application scenario of the object push method provided in an embodiment of the present application. In the application scenario shown in Figure 6a, the object storage system includes a first client, a second client, a third client, a service layer, a first object storage device, and a second object storage device. Among them, the service layer includes a first server and a second server. The first client, the third client, the first server, and the first object storage device are located in the first area. The second client, the second server, and the second object storage device are located in the second area. Objects in the target bucket can be stored in the first object storage device and the second object storage device. Both the first client and the second client send object subscription requests for the target bucket to the service layer.

[0151] Refer to Figure 6b, which is a flow chart of a method for pushing an object provided in an embodiment of the present application. This method can be applied to the application scenario shown in Figure 6a above for example. Specifically, the method for pushing an object shown in Figure 6b may include:

[0152] Step S601: The third client sends object X to the first server.

[0153] During the operation of the object storage system, a user can store objects in the object storage system through a client. A user using a third client can also use the third client to store object X in a target bucket. Because the third client is located in the first region, the third client can send object X to the object storage system server in this region. In other words, the third client can send object X to the first server.

[0154] Specifically, the third client can send an object upload request to the first server. The first server can verify the object upload request, for example, by authenticating the third client to determine whether the third client has permission to upload new objects to the target bucket. If the verification is successful, the first server can send an object upload response to the third client. The third client can then send object X to the first server based on the object upload response.

[0155] Step S602: The first server stores object X in a first object storage device.

[0156] After obtaining object X, the first server can store object X on the object storage device. In the first region, objects in the target bucket can be stored on the first storage device. Therefore, the first server can store object X on the first object storage device.

[0157] It is understandable that, because the first server and the second object storage device belong to different zones, the first server may not be able to directly store the object in the second object storage device. Therefore, the first server may not store object X in the second object storage device.

[0158] Step S603: In response to object X being stored in the first object storage device, object X is synchronously stored in the second object storage device.

[0159] After object X is stored on the first object storage device, the service layer can synchronize the target bucket. Specifically, the service layer can synchronize object X to the second object storage device. This way, object X is also stored on the second object storage device, and clients in the second region can retrieve object X from the second object storage device. Optionally, this synchronized storage can be implemented based on information exchange between the first and second object storage devices, which is not detailed here.

[0160] It is understood that if the object storage system further includes a third object storage device located in a third region, after object X is stored in the first object storage device, object X may be synchronously stored in the third object storage device. Furthermore, if the object storage system includes object storage devices located in N regions (N is a positive integer greater than 1), object X may be synchronously stored in all of the N regions except the first region.

[0161] Step S604: The first server obtains object X from the first object storage device.

[0162] Step S605: The first server pushes object X to the first client through the first transmission channel.

[0163] After storing object X in the first object storage device, the first server can determine that a new object has appeared in the target bucket. Assuming that object X matches the first client's subscription topic, the first server can push object X to the first client via the first transmission channel. For details on this, please refer to the embodiment corresponding to Figure 3b and will not be repeated here.

[0164] Step S606: The second server obtains object X from the second object storage device.

[0165] After storing object X in the second object storage system, the second server can determine that a new object has appeared in the target bucket. Assuming that object X matches the second client's subscription topic, the second server can proactively push object X to the second client. Specifically, the second server can retrieve object X from the second object storage device.

[0166] Step S607: The second server pushes object X to the second client through the second transmission channel.

[0167] After obtaining object X from the second object storage system, the second server can push object X to the second client via the second transmission channel. In this way, even if a third client that is no longer in the same region as the second client uploads a new object to the target bucket, the first object storage device and the second object storage device can be synchronized so that the target bucket stores the newly added object in the corresponding object storage device in the second region, allowing the server in the second region to proactively push the newly added object to the second client.

[0168] By synchronizing objects in the object storage devices corresponding to the target buckets in different regions, the target buckets can store the same objects on the object storage devices in each region. Clients in each region can retrieve the same objects from the object storage devices in their region. This enables cross-region transfer and synchronization of objects.

[0169] The present application also provides an object push device, wherein the object push device can be applied to the object storage service layer in the implementation shown in FIG2 to implement the functions of the service layer in the implementation shown in FIG3b or FIG6b. Specifically, as shown in FIG7, the object push device includes:

[0170] The receiving module 710 is configured to receive an object subscription request from a first client for a target storage bucket, where the object subscription request carries a subscription topic.

[0171] a channel establishing module 720 configured to establish a first transmission channel with the first client in response to the object subscription request, wherein the first transmission channel is configured to push the object to the first client after the service layer detects that the object has been uploaded to the target bucket;

[0172] The object acquisition module 730 is configured to acquire, according to the subscription topic, a plurality of newly added objects in the target bucket, where the plurality of newly added objects are distributed and stored in the at least one object storage device.

[0173] The push module 740 is configured to push the multiple newly added objects in the target bucket to the first client through the first transmission channel.

[0174] The receiving module 710, the channel establishment module 720, the object acquisition module 730, and the push module 740 can all be implemented by software or hardware. For example, the implementation of the receiving module 710 will be described below using the receiving module 710 as an example. Similarly, the implementation of the channel establishment module 720, the object acquisition module 730, and the push module 740 can be similar to the implementation of the receiving module 710.

[0175] As an example of a software functional unit, the receiving module 710 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the receiving module 710 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0176] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0177] As an example of a hardware functional unit, receiving module 710 may include at least one computing device, such as a server. Alternatively, receiving module 710 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0178] The multiple computing devices included in the receiving module 710 can be distributed in the same region or in different regions. The multiple computing devices included in the receiving module 710 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the receiving module 710 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0179] It should be noted that, in other embodiments, the receiving module 710 can be used to execute any step in the object push method, the channel establishment module 720 can be used to execute any step in the object push method, the object acquisition module 730 can be used to execute any step in the object push method, and the push module 740 can be used to execute any step in the object push method. The steps that the receiving module 710, the channel establishment module 720, the object acquisition module 730 and the push module 740 are responsible for implementing can be specified as needed, and the object is implemented by respectively implementing different steps in the object push method through the receiving module 710, the channel establishment module 720, the object acquisition module 730 and the push module 740. Full functionality of the device.

[0180] This application also provides a computing device 800. As shown in Figure 8, computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. Processor 104, memory 106, and communication interface 108 communicate with each other via bus 102. Computing device 800 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 100.

[0181] Bus 102 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG8 shows a single bus line, but this does not imply a single bus or type of bus. Bus 104 may include a path for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, and communication interface 108).

[0182] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0183] The memory 106 may include volatile memory, such as random access memory (RAM). The processor 104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0184] The memory 106 stores executable program code. The processor 104 executes the executable program code to implement the functions of the aforementioned receiving module 110, channel establishment module 720, object acquisition module 730, and push module 740, thereby implementing the object push method. In other words, the memory 106 stores instructions for executing this stored method.

[0185] Alternatively, the memory 106 stores executable code, and the processor 104 executes the executable code to respectively implement the functions of the aforementioned object storage client, object storage service layer, and object storage device, thereby implementing the object push method. In other words, the memory 106 stores instructions for executing the object push method.

[0186] The communication interface 103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0187] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0188] As shown in Figure 9, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the object push method.

[0189] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the object push method. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for executing the object push method.

[0190] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions, each for executing a portion of the functions of the object storage service layer. In other words, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more of the receiving module 710, the channel establishment module 720, the object acquisition module 730, and the push module 740.

[0191] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG10 shows a possible implementation. As shown in FIG10 , two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the receiving module 710, the channel establishing module 720, and the pushing module 740. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the object acquisition module 730.

[0192] The connection method between the computing device clusters shown in Figure 10 may be to consider that the object push method provided in this application needs to interact with the object storage device, and the interaction may bring greater pressure, so it is considered to hand over the function of the object acquisition module 630 to the computing device 100B for execution.

[0193] It should be understood that the functions of the computing device 100A shown in FIG10 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0194] The present application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similar to the connection method of the computing device cluster described in Figures 9 and 10. However, the memory 106 of one or more computing devices 100A in this computing device cluster can store the same instructions for executing the object push method.

[0195] In some possible implementations, the memory of one or more computing devices 100B in the computing device cluster may also store partial instructions for executing the object push method. In other words, the combination of one or more computing devices can jointly execute the instructions for executing the object push method.

[0196] It should be noted that the memory 106 in different computing devices 100A in the computing device cluster may store different instructions for executing portions of the object storage functionality. That is, the instructions stored in the memory 106 in different computing devices 100A may implement the functionality of one or more devices in the object storage client, the object storage service layer, and the object storage device.

[0197] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the object push method.

[0198] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the object push method, or instruct the computing device to execute the object push method.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An object push method, characterized in that: The method is applied to a service layer of an object storage system, the service layer is used to access at least one object storage device, the at least one object storage device is located in a plurality of data centers in different regions, each data center includes a plurality of servers, the method includes: Receive an object subscription request from a first client for a target storage bucket, where the object subscription request carries a subscription topic; In response to the object subscription request, establishing a first transmission channel with the first client, wherein the first transmission channel is used to push the object to the first client after the service layer detects that the target storage bucket has uploaded the object; According to the subscription topic, a plurality of newly added objects in the target storage bucket are acquired, where the plurality of newly added objects are distributed and stored in the at least one object storage device; The multiple newly added objects in the target storage bucket are pushed to the first client through the first transmission channel.

2. The method according to claim 1, characterized in that The first transmission channel is established based on a WebSocket protocol, and the step of establishing the first transmission channel with the first client in response to the object subscription request includes: In response to the object subscription request, the Hypertext Transfer Protocol (HTTP) connection established between the first client and the service layer is switched to a WebSocket connection to establish a WebSocket transmission channel with the first client.

3. The method according to claim 1 or 2, characterized in that: The step of pushing the multiple newly added objects in the target storage bucket to the first client through the first transmission channel includes: The access domain names of the multiple newly added objects in the target storage bucket are pushed to the first client through the first transmission channel.

4. The method according to any one of claims 1 to 3, characterized in that: The subscription topic includes at least one of the following: a client identifier to which the multiple newly added objects belong, a category to which the multiple newly added objects belong, a directory where the multiple newly added objects are located, a storage time of the multiple newly added objects, or a storage type of the multiple newly added objects, wherein the storage type is used to indicate an access frequency of the multiple newly added objects.

5. The method according to any one of claims 1 to 4, characterized in that: Before establishing a first transmission channel with the first client in response to the object subscription request, the method includes: According to the configuration information of the first client, it is determined that the target storage bucket opens the first transmission channel.

6. The method according to any one of claims 1 to 5, characterized in that: The multiple newly added objects in the target bucket include a first object and a second object, the first object is stored in a first object storage device located in a first area, and the second object is stored in a second object storage device located in a second area, and the method includes: synchronizing the first object from the first object storage device to the second object storage device; The multiple newly added objects in the target storage bucket are pushed to a second client through a second transmission channel, where the second client is set in the second region.

7. An object pushing device, characterized in that: The device is applied to a service layer of an object storage system, the service layer is used to access at least one object storage device, the at least one object storage device is located in multiple data centers in different regions, each data center includes multiple servers, and the device includes: A receiving module, configured to receive an object subscription request from a first client for a target storage bucket, wherein the object subscription request carries a subscription topic; a channel establishment module, configured to establish a first transmission channel with the first client in response to the object subscription request, wherein the first transmission channel is used to push the object to the first client after the service layer detects that the target storage bucket has uploaded the object; An acquisition module, configured to acquire, according to the subscription topic, a plurality of newly added objects in the target storage bucket, wherein the plurality of newly added objects are distributed and stored in the at least one object storage device; A push module is used to push the multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

8. The device according to claim 7, characterized in that The first transmission channel is established based on the WebSocket protocol, and the channel establishment module is specifically used to: In response to the object subscription request, the Hypertext Transfer Protocol (HTTP) connection established between the first client and the service layer is switched to a WebSocket connection to establish a WebSocket transmission channel with the first client.

9. The device according to claim 7 or 8, characterized in that The push module is specifically used for: The access domain names of the multiple newly added objects in the target storage bucket are pushed to the first client through the first transmission channel.

10. The device according to any one of claims 7 to 9, characterized in that The subscription topic includes at least one of the following: a client identifier to which the multiple newly added objects belong, a category to which the multiple newly added objects belong, a directory where the multiple newly added objects are located, a storage time of the multiple newly added objects, or a storage type of the multiple newly added objects, wherein the storage type is used to indicate an access frequency of the multiple newly added objects.

11. The device according to any one of claims 7 to 10, characterized in that The channel establishment module is also used for: According to the configuration information of the first client, it is determined that the target storage bucket opens the first transmission channel.

12. The device according to any one of claims 7 to 11, characterized in that The multiple newly added objects in the target storage bucket include a first object and a second object, the first object is stored in a first object storage device located in a first area, and the second object is stored in a second object storage device located in a second area, and the apparatus further includes a synchronization module: The synchronization module is used to synchronize the first object from the first object storage device to the second object storage device; The push module is further configured to push the multiple newly added objects in the target storage bucket to a second client via a second transmission channel, where the second client is set in the second region.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images