Object pushing method and device and related equipment

Abstract

The invention provides an object pushing method and device and related equipment, the method is applied to a service layer of an object storage system, and the method comprises the steps that an object subscription request of a first client for a target storage bucket is received, and the object subscription request carries a subscription theme; and in response to the object subscription request, establishing a first transmission channel with the first client. Then, according to the subscription theme, multiple newly-added objects in the target storage bucket are obtained, and the multiple newly-added objects are distributed and stored in at least one object storage device; and finally, pushing the plurality of newly added objects in the target storage bucket to the first client through the first transmission channel. Thus, the service layer can actively push the newly added object in the storage bucket to the client subscribing the storage bucket, the client does not need to repeatedly send an object acquisition request to the service layer to acquire the newly added object, and the network overhead is reduced. And the service layer actively pushes the newly added object to the client through the transmission channel, so that the real-time performance of object acquisition is improved.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on July 17, 2023, with application number 202310876361.1 and invention name “A method, device and other equipment for object access”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of storage technology, and in particular to an object push method, device and related equipment. Background Art

[0003] With the development of computer technology, cloud storage services have been widely used. Object storage service (OBS) is a type of cloud storage service. In the object storage service, users can establish a connection with the object storage service layer through the object storage client. Based on the connection between the object storage client and the object storage service layer, users can store objects in the object storage service layer or obtain objects stored in the object storage service layer.

[0004] In some scenarios, if the objects stored in the object storage service layer have changed, the object storage client may need to obtain the changed objects. In order to obtain the changed objects, the object storage client can periodically establish a connection with the object storage service layer and send object acquisition requests through the connection. The object storage service layer can then send the changed objects to the object storage client based on the object acquisition request.

[0005] However, the above method relies on the client to send a request to the service layer, which has the problems of high network overhead and poor real-time performance of acquiring new objects. Summary of the invention

[0006] In view of this, the embodiment of the present application provides an object push method for reducing the network overhead of the client and improving the efficiency of the client in obtaining data. The present application also provides a corresponding apparatus, a computing device cluster, a computer-readable storage medium, and a computer program product.

[0007] In a first aspect, the present application provides an object push method, which can be applied to a service layer of an object storage system. The object storage system also includes a client and at least one storage device. The service layer of the object storage system can access at least one object storage device. Different object storage devices can be located in multiple data centers in different regions, and each data center can include multiple servers. When executing the object push method, the service layer can receive an object subscription request. The object subscription request is a request sent by a first client for a target storage bucket, and can also carry a subscription topic, indicating that a user using the first client needs to obtain a newly added object related to the subscription topic in the target storage bucket. According to the object subscription request, the service layer can establish a first transmission channel with the first client. The first transmission channel can be used to push the object to the first client after the service layer detects that the target storage bucket has uploaded an object. After the first transmission channel is established, the service layer can monitor whether a newly added object matching the subscription topic appears in the target storage bucket. If so, the service layer can obtain the newly added object in the target storage bucket according to the subscription topic. These newly added objects can be distributed and stored in at least one object storage device. Then, the service layer pushes the obtained multiple newly added objects to the first client through the first transmission channel. That is to say, if the user wants to obtain the latest newly added objects related to the subscription topic in a bucket, the user can send an object subscription request for the bucket through the client. The service layer can establish a transmission channel based on the object subscription request, and push the newly added objects corresponding to the subscription topic to the client through the transmission channel after the objects corresponding to the subscription topic are added to the bucket. In this way, the client only needs to send an object subscription request to obtain the objects of the subscription topic in the bucket, without the client repeatedly sending object acquisition requests to the service layer, thereby reducing network overhead. In addition, the service layer actively pushes the newly added objects to the client through the transmission channel, so that the client can obtain the newly added objects in the target bucket more quickly, improving the real-time acquisition of the newly added objects.

[0008] In some possible implementations, the first transmission channel can be established based on a long connection between the service layer and the first client. Specifically, the first client can first establish a short connection with the service layer, and then send an object subscription request to the service layer based on the short connection. Then, the service layer can establish a long connection based on the short connection to achieve the establishment of the first transmission channel. For example, the first transmission channel can be established based on the Websocket protocol. The object subscription request can be sent based on a Hypertext Transport Protocol (HTTP) connection between the first client and the service layer, or based on a Hypertext Transfer Protocol Secure (HTTPS) connection between the first client and the service layer. Accordingly, after obtaining the object subscription request, the service layer can switch the HTTP connection or HTTPS connection between the first client and the server to a Websocket connection. In this way, object push is implemented based on a long connection, so that the first transmission channel will not be disconnected due to no data transmission, avoiding the first client from frequently establishing a transmission channel with the service layer, and reducing network overhead.

[0009] In some possible implementations, the service layer may push the newly added object itself to the first client, or may push the access domain name of the newly added object to the first client. The access domain name of the newly added object is used to access or download the newly added object in the object storage device. In this way, after the first client obtains the access domain name of the newly added object, it can view or download the newly added object through the access domain name. In this way, the domain name of the newly added object is transmitted instead of the newly added object itself, which reduces the amount of data that needs to be transmitted, improves data transmission efficiency, and reduces network overhead.

[0010] In some possible implementations, the subscription topic may include at least one of the following: the client identifier to which the multiple newly added objects belong, the category to which the multiple newly added objects belong, the directory to which the multiple newly added objects are located, the save time of the multiple newly added objects, and the storage type of the multiple newly added objects. Among them, the client identifier to which the newly added objects belong is used to identify the client that uploads the newly added objects to the object storage system. The category described in the newly added object can be used to identify the data type of the newly added object, or to identify other types of newly added objects. The directory where the newly added object is located can be determined based on the storage address of the newly added object in the object storage system. The save time of the newly added object can be used to indicate the time when the newly added object is uploaded to the object storage system. The storage type of the newly added object can be used to indicate the access frequency of the multiple newly added objects. In this way, the service layer supports pushing new objects corresponding to one or more subscription topics, thereby improving the flexibility of object push.

[0011] In some possible implementations, the service layer may determine whether the target storage bucket has the permission to open the first transmission channel before establishing the first transmission channel. Specifically, the service layer may determine that the target storage bucket opens the first transmission channel based on the configuration information of the first client. In this way, the management of the storage bucket can be better implemented based on the configuration information of the client.

[0012] In some possible implementations, multiple newly added objects of the target bucket may come from different storage devices in different regions. Before pushing to the client through the transmission channel, the service layer may first synchronize the target bucket so that the target bucket stores the same newly added objects in different storage devices in different regions. Specifically, it is assumed that the newly added objects of the target bucket include a first object and a second object. The first object is stored in a first object storage device located in the first region, and the second object is stored in a second object storage device located in the second region. If the service layer pushes the newly added object to the first client through the first transmission channel, the service layer may first synchronize the second object from the second object storage device to the first object storage device, and then push the first object and the second object in the first storage device to the first client through the first transmission channel, so as to push the multiple newly added objects in the target bucket to the first client. If the service layer pushes the newly added object to the second client set in the second region through the second transmission channel, the service layer may first synchronize the first object from the first object storage device to the second object storage device, and then push the first object and the second object in the second storage device to the second client through the first transmission channel, so as to push the multiple newly added objects in the target bucket to the second client. In this way, by synchronizing the objects in the object storage devices corresponding to the target buckets in different regions, the target buckets can store the same objects in the object storage devices in each region, and the clients in each region can obtain the same objects from the object storage devices in their region. In this way, cross-region transmission and synchronization of objects are achieved.

[0013] In a second aspect, the present application provides an object pushing device, which is applied to a service layer of an object storage system, the service layer is used to access at least one object storage device, the at least one object storage device is located in multiple data centers in different regions, each data center includes multiple servers, and the device includes:

[0014] A receiving module, configured to receive an object subscription request from a first client for a target storage bucket, wherein the object subscription request carries a subscription topic;

[0015] A channel establishment module, configured to establish a first transmission channel with the first client in response to an object subscription request, wherein the first transmission channel is used to push the object to the first client after the service layer detects that the target storage bucket has uploaded the object;

[0016] An object acquisition module, used to acquire multiple newly added objects in a target storage bucket according to a subscription topic, wherein the multiple newly added objects are distributed and stored in at least one object storage device;

[0017] The push module is used to push multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

[0018] In some possible implementations, the first transmission channel is established based on the WebSocket protocol, and the channel establishment module is specifically used to switch the HTTP connection established between the first client and the service layer to a WebSocket connection in response to the object subscription request, so as to establish a WebSocket transmission channel with the first client.

[0019] In some possible implementations, the push module is specifically configured to push the access domain names of the multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

[0020] In some possible implementations, the subscription topic includes at least one of the following: a client identifier to which the multiple newly added objects belong, a category to which the multiple newly added objects belong, a directory where the multiple newly added objects are located, a storage time of the multiple newly added objects, or a storage type of the multiple newly added objects, where the storage type is used to indicate an access frequency of the multiple newly added objects.

[0021] In some possible implementations, the channel establishing module is further used to determine the target storage bucket to open the first transmission channel according to the configuration information of the first client.

[0022] In some possible implementations, the multiple newly added objects in the target storage bucket include a first object and a second object, the first object is stored in a first object storage device located in the first area, and the second object is stored in a second object storage device located in the second area. The apparatus also includes a synchronization module, which is used to synchronize the first object from the first object storage device to the second object storage device; and a push module, which is also used to push the multiple newly added objects in the target storage bucket to a second client through a second transmission channel, and the second client is set in the second area.

[0023] In a third aspect, the present application provides a computing device cluster, wherein the computing device includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory; the at least one memory is used to store instructions, and the at least one processor executes the instructions stored in the at least one memory, so that the computing device cluster executes the object push method in the above-mentioned first aspect or any possible implementation of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The at least one computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on at least one computing device, the at least one computing device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0025] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0026] Based on the implementations provided in the above aspects, the present application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0028] Figure 1 A schematic diagram of at least one application scenario provided by an embodiment of the present application;

[0029] Figure 2 A schematic diagram of an architecture of an object storage system provided in an embodiment of the present application;

[0030] Figure 3a A schematic diagram of an application scenario provided in an embodiment of the present application;

[0031] Figure 3b An interactive schematic diagram of the object push method provided in an embodiment of the present application;

[0032] Figure 3c Another schematic diagram of an application scenario provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of an implementation method of the first client provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of an implementation method for establishing a first transmission channel provided in an embodiment of the present application;

[0035] Figure 6a A schematic diagram of another application scenario provided in an embodiment of the present application;

[0036] Figure 6b Another interactive schematic diagram of the object push method provided in the embodiment of the present application;

[0037] Figure 7 A schematic diagram of the structure of an object pushing device provided in an embodiment of the present application;

[0038] Figure 8 A schematic diagram of a structure of a computing device provided in an embodiment of the present application;

[0039] Fig. 9 A schematic diagram of a structure of a computing device cluster provided in an embodiment of the present application;

[0040] Fig.10 A schematic diagram of an implementation method of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The scheme in the embodiments provided in this application will be described below in conjunction with the drawings in this application.

[0042] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is just a way of distinguishing objects with the same attributes when describing the embodiments of this application.

[0043] First, some terms and application scenarios of object storage services involved in the embodiments of this application are introduced.

[0044] (1) Object storage service (OBS): The object storage service is an object-based storage service that has the advantages of massive, secure, highly reliable, and low-cost. OBS is a service accessible over the Internet. Tenants can establish a connection with the object storage service layer (computing nodes that support OBS) through the object storage client, create buckets in the storage nodes managed by the object storage service layer, and then access and manage objects in the buckets. The object storage service layer uses a sequential distribution method by default to store objects in buckets. The sequential distribution method can also be called a lexicographic distribution or a range distribution.

[0045] (2) Object: An object is the basic unit of data storage in an object storage system. An object is actually a collection of a file's data and its related attribute information (metadata). The data uploaded by tenants to OBS is stored in buckets in the form of objects. Objects consist of three parts: key, metadata, and data. The key, or object name, is a UTF-8-encoded character sequence with a length greater than 0 and less than 1024. Each object in a bucket has a unique object key.

[0046] (3) Bucket: A bucket is a container for storing objects in OBS, and can also be called a storage bucket or data bucket. Object storage provides a flat storage method based on buckets and objects. All objects in a bucket are at the same logical level, eliminating the multi-level tree directory structure in the file system. Each bucket has its own storage category, access rights, region, and other attributes. Tenants can create buckets with different storage categories and access rights, and configure more advanced attributes to meet the storage requirements of different scenarios.

[0047] (4) Object storage device (OSD): OSD is the basic storage unit of the object storage system. It is set on the physical disk and is specifically a fixed-size storage space of the physical disk. The object storage system manages the physical disks of multiple computing devices in the form of OSD.

[0048] Tenants can log in to the cloud management platform on the public cloud access page with a pre-registered account and password, and after successful login, select and purchase corresponding public cloud services on the public cloud access page, such as OBS, virtual machine services, container services, etc. For OBS, tenants can further create multiple buckets through the configuration interface or application programming interface (API) on the public cloud access page provided by the cloud platform. There is no limit on the total number and size of objects stored in each bucket. Tenants do not need to consider the scalability of data. OBS is a service based on the Representational State Transfer API (REST API) and Hypertext Transport Protocol (HTTP) and Hypertext Transfer Protocol Secure (HTTPS). Tenants can locate bucket resources through the Uniform Resource Locator (URL), which is also called the access domain name in this application, also referred to as the domain name. Among them, in OBS, the bucket name is globally unique and cannot be modified, that is, the bucket created by the tenant cannot be the same as the name of other buckets created by the tenant, nor can it be the same as the bucket name created by other tenants.

[0049] Figure 1 A schematic diagram showing at least one application scenario in an embodiment of the present application is shown. Figure 1 As shown, each bucket may include multiple objects, and the objects between buckets are isolated from each other. Tenants log in to the cloud management platform through the object storage client, select and purchase the cloud service of the object storage service on the cloud management platform. After the purchase, the tenant can store objects based on the functions provided by the object storage service. Among them, the cloud management platform is mainly used to manage the infrastructure for running the object storage service. Exemplarily, the infrastructure for running the object storage service may include multiple data centers set up in different regions, and each data center includes multiple servers. The data center can provide basic resources for the object storage service, such as computing resources, storage resources, etc. Therefore, when tenants purchase and use the object storage service, they mainly pay for the resources used. Specifically, the object storage service provides the domain name of the bucket, and the tenant can access the domain name of the bucket through the object storage client to upload data to the bucket or download data from the bucket, wherein the uploaded data is stored in the bucket in the form of objects.

[0050] Figure 2 The schematic diagram of the architecture of the object storage system in the embodiment of the present application is shown. The object storage system includes an object storage client, an object storage service layer and an object storage device.

[0051] The object storage client can be set in a terminal device, which can be a mobile phone, laptop, tablet computer, PDA, wireless terminal in a smart city, or wireless device in a smart home. The tenant logs in to the cloud management platform through the client, and selects and purchases the object storage cloud service on the cloud management platform.

[0052] The object storage service layer is used to manage one or more object storage devices. The object storage service layer includes one or more service terminals, which can be set in the service device of the service layer. Exemplarily, the service device is connected to the storage device through a switching device. Another exemplary embodiment, the service device is connected to the terminal device through a remote connection gateway.

[0053] Tenants log in to the object storage service layer using their accounts on the client, and select and purchase object storage cloud services in the service layer, such as creating buckets and configuring bucket names. After the service layer detects the tenant's operation (such as creating a bucket), it can send a creation command to the object storage device, where the creation command includes information such as the bucket name and domain name. The creation command is used to notify the storage device to create a bucket and save the bucket name, bucket domain name and other information. After creating the bucket, the tenant can access the bucket domain name through the object storage client, locate the bucket in the storage device, and upload, download, and delete objects from the bucket.

[0054] The service layer includes a control unit and an operating system. The control unit runs on the operating system. The operating system includes a disk driver and a physical network card driver. The control unit controls the disk controller through the disk driver to set the physical disk as multiple persistent storage units (persistence log, PLOG).

[0055] The infrastructure for running the object storage service can be set up in multiple data centers in different regions. For example, the object storage devices are set up in multiple data centers in different regions, and each data center includes multiple object storage devices. Each object storage device includes at least one physical disk. For example, the disk controller sets physical disk 1 and physical disk 2 in Region 1 as 4 PLOGs.

[0056] After receiving the creation instruction of bucket 1, the service layer notifies the control unit to create bucket 1. The control unit creates bucket 1 on physical disk 1 and physical disk 2 in the object storage device through the operating system. The PLOG of bucket 1 is distributed on physical disk 1 and physical disk 2, and the bucket name, bucket domain name and other information of bucket 1 are saved.

[0057] When the tenant needs to upload an object to bucket 1 through the client, the tenant can trigger the object storage device through the client to write the object into bucket 1. For example, the object can be written into bucket 1 by a control unit in the object storage service layer.

[0058] When the tenant needs to download an object in bucket 1 through the client, the tenant can trigger the object storage device through the client to read the object from bucket 1. For example, the control unit in the service layer can obtain the object from bucket 1 and feed it back to the client.

[0059] In some scenarios, the client may need to obtain newly added objects in the service layer. The newly added objects in the service layer may be referred to as newly added objects. Optionally, the newly added objects may come from a specific bucket. Users can obtain newly added objects from a specific bucket. For example, in scenarios such as data subscription, live broadcast, and map, the objects stored in the bucket may change, and users need to obtain the latest updated objects in the bucket.

[0060] To meet this requirement, the client can actively send a request to the service layer to obtain the newly added objects in the bucket. Specifically, a polling mechanism can be deployed in the client. The client deployed with the polling mechanism can periodically establish a connection with the service layer and send an object query request to the service layer. The object query request may include a bucket identifier, which is used to query whether there is a newly added object in the bucket corresponding to the bucket identifier. If there is a newly added object in the bucket, the service layer can obtain the newly added object and send the newly added object to the client through the connection with the client. In this way, if an object is added to the bucket, the client can obtain the newly added object in the bucket after sending the object query request next time.

[0061] However, although the above method can obtain the newly added object, the object acquisition request for obtaining the newly added object is actively sent by the client, and the connection for transmitting the newly added object is also actively requested to be established by the client. Therefore, in order to obtain the newly added object in a timely manner, the client needs to frequently establish a connection with the service layer and send an object acquisition request to the service layer. In this way, frequent establishment of connections and sending of requests require more network resources, and there is a problem of high network overhead. In particular, if the service layer is used to provide services for multiple service terminals, and multiple service terminals have the need to obtain newly added objects, then the service layer may need to establish connections with multiple clients within a unit time and receive multiple object query requests. In this way, the pressure on the service layer will be significantly increased, which may affect the normal operation of the service layer.

[0062] In addition, since the action of obtaining the newly added object is actively initiated by the client, and the client cannot determine when the object is added to the bucket, the client may not be able to obtain the newly added object in time, resulting in poor timeliness of object acquisition.

[0063] For example, suppose that a new object is added to bucket 1 at time t1, and the client establishes a connection with the service layer at time t2 (time t2 is later than time t1) and sends an object acquisition request for bucket 1 to the service layer. Then the service layer can obtain the object acquisition request at the earliest time t2 and send the new object in bucket 1 to the client, and the client can obtain the new object in bucket 1 at the earliest time t2. In the above process, there is at least a time difference of (t2-t1) between the appearance of the new object and the client obtaining the new object.

[0064] From the above introduction, it can be seen that since the service layer lacks the means to actively push new objects to the client, the client will incur a large network overhead to obtain the new objects, and it is impossible to obtain the new objects in time.

[0065] Based on this, an embodiment of the present application provides an object push method, which can be implemented by an object storage service layer in an object storage system system, and improves the efficiency of the client obtaining the newly added objects by actively pushing the newly added objects, and reduces the network overhead. Specifically, when executing the object push method, the service layer can first receive an object subscription request from a first client for a target bucket. The object subscription request carries a subscription topic, which is used to subscribe to the newly added objects matching the subscription topic in the target bucket. According to the object subscription request, a first transmission channel can be established between the service layer and the first client. The first transmission channel is used to push the object to the first client after the service layer detects that the target bucket has uploaded an object. After establishing the first transmission channel, the service layer can obtain multiple newly added objects in the target bucket according to the subscription topic. The multiple newly added objects can be distributed and stored in at least one object storage device of the object storage system. After obtaining the multiple newly added objects, the service layer can push the multiple newly added objects to the first client through the first transmission channel. That is, if a user wants to obtain the latest newly added objects related to the subscription topic in a bucket, the user can send an object subscription request for the bucket through the client. The service layer can establish a transmission channel based on the object definition request, and push the object corresponding to the subscription topic to the client through the transmission channel after adding it to the bucket. In this way, the client only needs to send an object subscription request to obtain the object of the subscription topic in the bucket, without the client repeatedly sending object acquisition requests to the service layer, thereby reducing network overhead. In addition, the service layer actively pushes the newly added object to the client through the transmission channel, so that the client can obtain the newly added object in the target bucket more quickly, improving the real-time acquisition of the newly added object.

[0066] Next, various non-limiting specific implementations of the object push process are described in detail.

[0067] First, the application scenarios of the object push method are introduced.

[0068] See also Figure 3a , Figure 3a A schematic diagram of an application scenario of the object push method provided in the embodiment of the present application. Figure 3a In the application scenario shown, the object storage system includes a service layer, an object storage device, a client A, and a client B. Among them, client A can upload an object to the service layer. The service layer can actively push the object uploaded by client A to client B. In addition, client B can also send an object subscription request to the service layer to establish a first transmission channel between the service layer and client B. The first transmission channel can be used to transmit the object that the service layer actively pushes to client B.

[0069] See also Figure 3b , Figure 3b A flow chart of the object push method provided in the embodiment of the present application. The method can be applied to the above Figure 3a The application scenario shown in FIG. 1 may also be applied to other applicable application scenarios. Figure 3a The application scenario shown is used as an example for explanation. Optionally, Figure 3b The object push method shown can be used by Figure 3a The service layer in .

[0070] Specifically, Figure 3b The object push method shown may specifically include:

[0071] Step S301: The first client sends an object subscription request for a target storage bucket to the service layer.

[0072] In an embodiment of the present application, the service layer can actively push new objects to the client. In some implementations, the active push function can be enabled by the client. Specifically, if a user using the first client wants the service layer to actively push new objects to the client, the user can send an object subscription request to the service layer through the first client. The object subscription request is used to request the service layer to actively push new objects to the client. Optionally, the first client can be Figure 3a Shown is client B in the user scenario.

[0073] As described above, the service layer of the object storage system may include one or more servers. If the service layer of the object storage system includes multiple servers, the first client may send an object subscription request to one of the multiple servers. For ease of introduction, the server to which the first client sends the object subscription request is referred to as the first server below.

[0074] Optionally, the first client may send an object subscription request to the server according to the region where the first client is located. That is, the first client and the first server may be in the same region.

[0075] In actual scenarios, an object storage system may include multiple buckets, each of which can be used to store objects. The newly added objects that the client wants to obtain may be from one or more specific buckets. Therefore, the object subscription request sent by the client to the service layer may be a request for a specific bucket to obtain the newly added objects in the specific bucket. Figure 1 In a corresponding implementation, a user using the first client needs to obtain a newly added object in the target bucket. The first client may send an object subscription request for the target bucket to the service layer.

[0076] In some implementations, the client may need to obtain newly added objects from multiple buckets. Accordingly, the client may send multiple object subscription requests, each of which may correspond to one bucket. Alternatively, the object subscription request may also be directed to multiple buckets to subscribe to newly added objects in multiple buckets. For ease of introduction, the following description will take the object subscription request sent by the first client to subscribe to newly added objects in the target storage bucket as an example.

[0077] In an embodiment of the present application, the target storage bucket may be a logical bucket. Specifically, the objects in the target storage bucket may be stored in multiple object storage devices. Furthermore, the target storage devices storing the objects in the target storage bucket may be located in the same region or in different regions. For example, the target storage bucket may store object A, object B, and object C, where object A may be stored in a first storage device in a first region, object B may be stored in a second storage device in a second region, and object C may be stored in a third storage device in the second region.

[0078] In an embodiment of the present application, the object subscription request may include an identifier of a target bucket, indicating that the object subscription request is a request for the target bucket, and is used to request to obtain newly added information in the object bucket. Among them, the identifier of the target bucket may include, for example, the bucket name of the target bucket. Optionally, the identifier of the target bucket may also include information such as the access domain name of the target bucket. The object subscription request may also include a subscription topic. The subscription topic is used to indicate the newly added objects that the first client needs to obtain. In other words, the target bucket may include a variety of objects with different attributes, and the first client may need to obtain one or more types of newly added objects. Accordingly, the subscription topic may be used to describe the relevant information of the newly added objects that the first client needs to obtain.

[0079] Optionally, the subscription topic may include at least one of the following: the client identifier to which the multiple newly added objects belong, the category to which the multiple newly added objects belong, the directory where the multiple newly added objects are located, the storage time of the multiple newly added objects, and the storage type of the multiple newly added objects. In this way, the service layer supports pushing the newly added objects corresponding to one or more subscription topics, which improves the flexibility of object push. They are introduced below.

[0080] (1) The client ID to which the new object belongs.

[0081] In actual scenarios, multiple clients may share the same bucket. In this way, there may be multiple clients with the permission to upload objects to the target bucket. Accordingly, the newly added objects in the target bucket may come from multiple clients.

[0082] The client ID of the newly added object is used to identify the client to which the newly added object belongs. The client to which the newly added object belongs refers to the client that uploaded the newly added object to the target bucket. In other words, if client A uploaded object a to bucket 1, and client B uploaded object b to bucket 1. Then the client ID of the newly added object a is used to indicate client A, and the client ID of the newly added object b is used to indicate client B.

[0083] If the subscription topic includes the client identifier to which the newly added object belongs, it means that the user using the first client wants to obtain the newly added object uploaded by a specific client. That is, if the user wants to obtain the newly added object uploaded by client A, the subscription topic can include the corresponding identifier of client A. In this way, according to the subscription topic, the service layer can send object a to the first client. Since the subscription topic does not include the client identifier to which object b belongs, the service layer does not send object b to the first client.

[0084] (2) The category to which the new object belongs.

[0085] In an actual scenario, a storage bucket may be used to store objects of various categories. The category of an object may refer to the type of the object, for example, the data format of the object. Accordingly, a user using the first client may want to obtain newly added objects of a specific category. Accordingly, the subscription topic in the object subscription request sent by the first client may include relevant information of the specific category.

[0086] For example, assume that bucket 1 can be used to store objects in text format, objects in audio format, and objects in video format. However, the user using client A may only want to obtain the newly added objects in text format in bucket 1. Accordingly, the subscription topic in the object subscription request sent by client A to the service layer may include the category to which the newly added object belongs, and the category is used to indicate the object in text format.

[0087] (3) The directory where the new object is located.

[0088] In actual scenarios, a bucket may store multiple objects. Users can build directories in the bucket to manage stored objects. Accordingly, when storing objects in a bucket, users can store objects in a specific directory. For example, users can store multiple objects that are related in content in the same directory. Optionally, the directory where the newly added object is located can include the storage directory of the newly added object in the target bucket.

[0089] If the subscription topic includes the directory where the newly added object is located, it means that the user using the first client wants to obtain the newly added object under a specific directory in the target storage bucket. For example, if client A uploads object a1 and object a2 to bucket 1, object a1 is uploaded to directory M1 in bucket 1, and object a2 is uploaded to directory M2 in bucket 1. The subscription topic in the object subscription request sent by the first client can include relevant information about directory M1. In this way, according to the subscription topic, the service layer can send object a1 to the first client. Since the subscription topic does not include relevant information about directory M2, the service layer does not send object a2 to the first client.

[0090] (4) The storage time of the newly added object.

[0091] In actual scenarios, the storage time of objects in a bucket may be different. The user using the first client may want data stored in the target bucket within a specific time period. Accordingly, the subscription topic in the object subscription request sent by the first client may include the storage time of the newly added object. The storage time is used to indicate the time period selected by the user, indicating that the first client wants to obtain the newly added objects stored in the target bucket within the time period.

[0092] (5) Add storage type for new objects.

[0093] The storage type of an object can be used to indicate the access frequency of the object. Specifically, in actual scenarios, the access frequencies of multiple objects stored in a bucket may be different. For example, some objects may be accessed more frequently, some objects may be accessed less frequently, and some objects may be used for backup or archiving and are less likely to be accessed. Optionally, the storage type of an object can be determined based on the access frequency of the object. Accordingly, the subscription topic can include the storage type of a newly added object, and the storage type of a newly added object is used to indicate the access frequency of the newly added object.

[0094] It is understandable that the above five types of subscription topics are shown only as a reference. In actual scenarios, the object subscription request may also include other types of subscription topics. For example, the subscription topic may also include a select all topic, indicating that the user needs to obtain all newly added objects in the target bucket. When the subscription topic of the object subscription request is a select all topic, the service layer may push each newly added object in the target bucket to the first client. Optionally, the above select all topic may be the default value of the subscription topic. That is, if the user does not set the subscription topic of the object subscription request on the first client, the subscription topic in the object subscription request sent by the first client may be a select all topic.

[0095] The above describes the target bucket and subscription topic. The following describes how to determine the target bucket and subscription topic.

[0096] In the embodiment of the present application, the object subscription request sent by the first client is for a target storage bucket in the object storage system, and also includes a subscription topic. Optionally, the target storage bucket and the subscription topic may be selected by the user on the first client.

[0097] For example, see Figure 4 , which is a schematic diagram of an implementation method of the first client provided in an embodiment of the present application. Figure 4 In a corresponding implementation, if a user using the first client wants to send an object subscription request to the service layer through the first client, the first client may display the following information: Figure 4 The display area 400 is shown. The display area 400 includes a push control control 410, a bucket setting control 420, a subscription topic setting control 430, and a confirmation control 440.

[0098] Among them, the push control control 410 can be used to enable the active push function of the service layer. Figure 4 In the corresponding implementation, the push control control 410 includes a sub-control 411. The sub-control 411 is in a selected state, indicating that the user has enabled the active push function of the service layer. The first client can send an object subscription request to the service layer. It is understandable that if the sub-control 411 is not selected, the request sent by the first client to the service layer may not be an object subscription request.

[0099] The bucket setting control 420 is used to obtain the bucket determined by the user, that is, the target bucket. Optionally, the bucket setting control 420 may include an input box, in which the user may enter relevant information of the target bucket, such as the bucket name of the target bucket. Alternatively, the bucket setting control 420 may also include a drop-down menu, in which the user may select the target bucket.

[0100] The subscription theme setting control 430 is used to obtain the subscription theme determined by the user. Optionally, the subscription theme setting control 430 may include an input box, in which the user may enter the subscription theme. Alternatively, the subscription theme setting control 430 may also include a drop-down menu, in which the user may select the subscription theme.

[0101] The confirmation control 440 is used to trigger the request sending operation, that is, to trigger the first client to send an object subscription request to the service layer. That is, after the user determines the target bucket and the subscription topic, the user can trigger the confirmation control 440 to control the first client to send an object subscription request to the service layer.

[0102] In the embodiment of the present application, the service layer can establish a transmission channel with the first client according to the object subscription request, and actively push the newly added objects in the target bucket to the first client. In some possible implementations, the service layer may only support actively pushing the newly added objects in some buckets to the client. Accordingly, the service layer can support actively pushing the newly added objects in the target bucket to the client.

[0103] Optionally, the attribute information of the bucket in the object storage system may include push attribute information. The push attribute information is used to indicate whether the service layer can actively push the newly added objects in the bucket to the client. Correspondingly, the push attribute information of the target bucket can indicate that the service layer can actively push the newly added objects in the target bucket to the client. It can be understood that, assuming that the push attribute information of bucket 1 indicates that the service layer cannot actively push the newly added objects in bucket 1 to the client, then if the service layer obtains an object subscription request for bucket 1, the service layer can reject the object subscription request.

[0104] Optionally, the push attribute information can be set when creating a bucket. For example, the user can set the target bucket to actively push the newly added objects in the bucket to the client on the page for creating the target bucket. Alternatively, the push attribute information can also be modified by the client. For example, a user with the permission to modify the target bucket can modify the push attribute information of the target bucket through the service layer.

[0105] In an embodiment of the present application, the service layer may establish a first transmission channel with the first client according to the object subscription request. Before establishing the first transmission channel, the first client may first establish a connection with the service layer to send the object subscription request to the service layer through the connection.

[0106] Optionally, the first transmission channel can be established based on a long connection between the first client and the service layer. The first client can first establish a short connection with the service layer so as to send an object subscription request to the service layer through the short connection. Optionally, the short connection between the first client and the service layer can include, for example, a short connection established based on HTTP, or a short connection established based on HTTPS.

[0107] It can be understood that the connection between the first client and the service layer may be a connection between the first client and the first server.

[0108] Step S302: The service layer establishes a first transmission channel with the first client in response to the object subscription request.

[0109] After obtaining the object subscription request sent by the first client, the service layer can establish a first transmission channel with the first client according to the object subscription request. The first transmission channel is used to transmit newly added objects related to the subscription topic in the target bucket.

[0110] Before establishing the first transmission channel, the service layer may first determine whether the first transmission channel can be established with the client. Optionally, the service layer may determine whether the first transmission channel can be established based on the configuration information of the target storage bucket and / or the configuration information of the first client. For the convenience of subsequent description, in an embodiment of the present application, the service layer may determine that the target storage bucket can open the first transmission channel based on the configuration information of the target storage bucket, and / or determine that the target storage bucket can open the first transmission channel based on the configuration information of the first client.

[0111] Optionally, the service layer can determine whether a transmission channel for actively pushing newly added objects to the first client can be established based on the configuration information of the target bucket. Specifically, the object storage system may include multiple buckets, some of which may support pushing newly added objects to the client, and some of which may not support pushing newly added objects to the client. Before establishing the first transmission channel, the service layer can first determine whether the target bucket supports pushing newly added objects to the client. For example, if the configuration information of the target bucket can be used to indicate that the newly added objects in the target bucket can be actively pushed to the client through the transmission channel, the service layer can establish the first transmission channel; if the configuration information of the target bucket can be used to indicate that the newly added objects in the target bucket cannot be actively pushed to the client through the transmission channel, the service layer can refuse to establish the first transmission channel.

[0112] Optionally, the service layer can also determine whether the first transmission channel can be established based on the configuration information of the first client. Specifically, some clients may not be able to establish a transmission channel for the service layer to actively push new objects, so the service layer can refuse to establish a transmission channel for such clients. For example, the service layer can determine whether the first client has subscribed to the cloud service that actively pushes new objects. If the first client has not subscribed to the cloud service that actively pushes new objects, the client can refuse to establish the first transmission channel. For another example, the service layer can also determine whether the network conditions of the first client are sufficient to establish the first transmission channel. If the network conditions of the first service layer are insufficient to establish the first transmission channel, the service layer can refuse to establish the first transmission channel.

[0113] According to the foregoing introduction, the first client can first establish a connection with the service layer, and send an object subscription request to the service layer through the connection. Optionally, the first transmission channel can be implemented based on the connection for transmitting the object subscription request. For example, the first transmission channel can be a channel between the first client and the service layer for transmitting the object subscription request. Alternatively, the first client can first establish a short connection with the service layer, and the first transmission channel can be a long connection established based on the short connection. That is, after obtaining the object subscription request, the service layer can switch the short connection for transmitting the object subscription request to a long connection, and the first transmission channel can be implemented based on the long connection. Optionally, the above-mentioned long connection can be established based on the Websocket protocol.

[0114] Combine the following Figure 5 , taking the short connection between the first client and the service layer as HTTP connection and the long connection as Websocket connection as an example for introduction. Figure 5 , which is a schematic diagram of an implementation method of establishing a first transmission channel provided in an embodiment of the present application. Specifically, Figure 5 The method for establishing the first transmission channel shown may specifically include:

[0115] Step S501: An HTTP connection is established between the first client and the service layer.

[0116] Before establishing the first transmission channel, an HTTP connection between the first client and the service layer may be established. Specifically, the first client may send a connection establishment request to the first server, and the connection establishment request is used to request to establish an HTTP connection. The first server may send a connection establishment response to the first client according to the connection establishment request to establish an HTTP connection between the first client and the first server.

[0117] Optionally, the connection may be established by the first client before sending the object subscription request.

[0118] Step S502: The first client sends a connection upgrade request to the service layer.

[0119] Through the HTTP connection between the first client segment and the service layer, the first client can send a connection upgrade request to the service layer. Optionally, the first client can send a connection upgrade request to the first server in the service layer. The connection upgrade request is used to request to upgrade the HTTP connection to a Websocket connection. Optionally, the connection upgrade request may include an upgrade (Upgrade) flag and a Websocket flag. The connection upgrade request flag is used to request to upgrade the current connection to a Websocket connection.

[0120] Optionally, the connection upgrade request can be sent based on the object subscription response of the service layer. Specifically, after the first client sends the object subscription request to the service layer, the service layer can generate an object subscription response and send the object subscription response to the first client. The first client can generate a connection upgrade request based on the object subscription response and send it to the service layer. Optionally, if the first client does not have the permission to subscribe to the newly added objects of the subscription topic in the target bucket, and / or the service layer does not include the target bucket, and / or the target bucket is not used to store the newly added objects corresponding to the subscription topic, the service layer may not send the object subscription response to the first client.

[0121] Step S503: the service layer sends a connection upgrade response to the first client.

[0122] After obtaining the connection upgrade request, the service layer may send a connection upgrade response to the first client. Optionally, the first server in the service layer may send a connection upgrade response to the first client. The connection upgrade response may include upgrade status information, which indicates the status of the connection between the first client and the service layer, that is, indicating that the connection between the first client and the service layer has been switched from an HTTP connection to a Websocket connection. Optionally, the upgrade status information may include a status code 101. Optionally, before sending the connection upgrade response, the service layer may make a judgment. For example, it may be determined whether the first client has the authority to establish a Websocket connection with the service layer.

[0123] It can be understood that the short connection between the first client and the service layer may be a short connection between the first client and the first server, and the long connection between the first client and the service layer may be a long connection between the first client and the first server. Figure 4 The service layer in can be the first server.

[0124] After obtaining the connection establishment response, the first client completes the establishment of the Websocket connection with the service layer, that is, the establishment of the first transmission channel is completed.

[0125] Step S303: the service layer obtains multiple newly added objects in the target bucket according to the subscription topic.

[0126] After receiving the object subscription request, the service layer can monitor the newly added objects in the target bucket according to the subscription topic. If a new object appears in the target bucket, the service layer can obtain the new object according to the subscription request so as to send the multiple new objects to the first client through the first transmission channel. In the embodiment of the present application, the new object refers to the object stored in the service layer by the client. That is, the new object obtained by the service layer can be data stored in the target bucket by other users through clients other than the first client. Optionally, in Figure 3a In a corresponding implementation, the first client may be client B, and the newly added object may be the newly added object uploaded by client A to the service layer.

[0127] Optionally, the service layer obtains the newly added object, which may include the service layer obtaining the newly added object itself from the target storage bucket, or the service layer obtaining the access domain name corresponding to the newly added object from the target storage bucket. For an introduction to the access domain name of the newly added object, please refer to the following, which will not be repeated here.

[0128] Specifically, after an object is added to the target bucket, the service layer can determine whether the newly added object matches the subscription topic. If the newly added object matches the subscription topic, the service layer can obtain the newly added object. If the newly added object does not match the subscription topic, the service layer may not obtain the newly added object. Optionally, the newly added object obtained by the service layer can be distributed and stored in one object storage device or in multiple object storage devices.

[0129] Optionally, if the first transmission channel is a transmission channel between the first client and the first server, the first server may obtain the target object in the target storage bucket, and the first server may obtain the newly added object from the object storage device in the area to which the first client belongs.

[0130] It is understandable that, as the object storage system runs, there may be multiple newly added objects matching the subscription topic in the target bucket, so the service layer can obtain multiple newly added objects. Each time an object matching the subscription topic is stored in the target bucket, the service layer can obtain the newly added object from the target bucket according to the subscription topic. Optionally, the service layer can obtain multiple newly added objects at a time, or a single newly added object at a time.

[0131] Step S304: the service layer pushes the multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

[0132] After the first transmission channel is established and the newly added objects are obtained, the service layer may push the multiple newly added objects in the target storage bucket to the client through the first transmission channel.

[0133] In an embodiment of the present application, the service layer can push the newly added object itself to the first client, and can also push the access domain name of the newly added object to the first client. The access domain name of the newly added object is used to indicate the storage address of the newly added object in the object storage system, for example, it can indicate the storage address of the newly added object in the object storage device. The access domain name of the newly added object is used to access or download the newly added object in the object storage device. In this way, after the first client obtains the access domain name of the newly added object, it can view or download the newly added object through the access domain name. In this way, the domain name of the newly added object is transmitted instead of the newly added object itself, which reduces the amount of data that needs to be transmitted, improves data transmission efficiency, and reduces network overhead.

[0134] Optionally, as the object storage system works, there may be one or more clients that store objects in the target bucket at different time points. Accordingly, whenever a new object is stored in the target bucket, the service layer can determine whether the newly added object matches the subscription topic. If so, the service layer can obtain the newly added object and send it to the first client through the first transmission channel. Specifically, the service layer can actively push the newly added object to the first client through the first server.

[0135] Optionally, the first transmission channel can be implemented based on a long connection between the service layer and the first client. The long connection may not be disconnected after transmitting data. In this way, after the service end pushes one or more newly added objects to the first client, the first transmission channel may not be disconnected. If other clients store objects matching the subscription topic in the target storage bucket, the service layer can also use the first transmission channel to push the newly added objects to the first client. In this way, the first transmission channel can be continuously used to transmit the newly added objects, saving the network overhead caused by frequent connection establishment.

[0136] In the embodiment of the present application, the service layer can establish a transmission channel with the client according to the object establishment request sent by the client, and actively push the newly added objects required by the client to the client through the transmission channel. In this way, by establishing the transmission channel, the service layer of the object storage system has the ability to actively push objects to the client of the object storage system. It does not require the client to frequently send requests to the service layer and query whether there are newly added objects, which saves network resources, and enables the service layer to push the newly added objects to the client as soon as possible after they appear, eliminating the link of the service layer waiting for the client to poll, and improving the efficiency of obtaining newly added objects.

[0137] The object push method provided in the embodiment of the present application can also be applied to the field of big data, for example, it can be used to push data streams to a big data cluster that consumes streaming data.

[0138] For big data clusters that consume streaming data, currently, the streaming data from the data source can be input into the big data cluster through the message middleware so that the big data cluster can consume the streaming data. However, the ability of the message middleware to forward data is limited, which restricts the consumption capacity of the big data cluster and also has the problem of high pressure.

[0139] To this end, in the embodiment of the present application, the streaming data can be uploaded to the service layer as an object, and the client is configured for the big data cluster. In this way, the service layer can actively push the newly added object to the client to achieve the effect of actively pushing the data stream to the big data cluster.

[0140] Specifically, see Figure 3c , Figure 3c A schematic diagram of an application scenario of the object push method provided in the embodiment of the present application. Figure 3c The application scenario shown includes a data source, a service layer of an object storage system, a server of the object storage system, and a big data cluster. Among them, the big data cluster includes a streaming consumption interface and an operator. The data source can upload the data stream to the service layer of the object storage system. The service layer of the object storage system can actively push the newly added objects to the client through the transmission channel between the client and the data source. In this way, the data from the data source will be pushed to the client as a new addition. The client can forward the newly added object to the streaming consumption interface of the big data cluster. Therefore, the big data cluster can obtain the data stream from the data source through the streaming consumption interface, so that the operator inside the big data cluster can perform corresponding data processing according to the data stream. In this way, the demand for big data clusters to consume streaming data is realized by replacing the traditional message middleware with the object storage system.

[0141] As mentioned above, the service layer of the object storage system can include multiple servers, and the service layer can be used to manage multiple object storage devices. Different object storage devices can be located in different regions or in the same region. The object storage devices corresponding to the target storage bucket can be located in the same region or in different regions.

[0142] If the object storage device corresponding to the target bucket is located in a different region, the region where the first client is located and the region where the object storage device storing the newly added object is located may be the same region or different regions.

[0143] In some possible implementations, if the client storing the newly added object is in the same region as the first client, the newly added object is stored in the object storage device in the region where the first client is located. If the storage device in the region where the first client is located stores the newly added object, the first server can obtain the newly added object from the object storage device in the region where it is located, thereby sending the newly added object to the first client.

[0144] In some other possible implementations, if the client storing the newly added object is in a different region from the first client, the newly added object is stored in an object storage device in a region other than the region where the first client is located. In this way, the object storage in the region where the first client is located does not store the newly added object, making it inconvenient for the first client to obtain the newly added object.

[0145] To this end, in an embodiment of the present application, if the objects in the target bucket can be stored in different object storage devices in different areas, the service layer can first synchronize the target bucket after the new object appears in the target bucket. Synchronizing the target bucket refers to synchronizing the contents stored in the target bucket in object storage devices in different areas. For example, suppose that the object storage system has object storage devices deployed in both the first area and the second area, and the objects in the target bucket can be stored in both the first object storage device in the first area and the second object storage device in the second area. Then synchronizing the target bucket can include synchronizing the objects in the target bucket stored in the first object storage device to the second object storage device, and synchronizing the objects in the target bucket stored in the second object storage device to the first object storage device.

[0146] For example, assume that client A in the first region stores object X in the target bucket. Since client A is located in the first region, client A can interact with the server of the service layer in the first region, and the server in the first region will store object X in the object storage device corresponding to the target bucket in the first region, that is, the first object storage device.

[0147] After the first object storage device stores object X, the service layer can synchronize the target bucket. Specifically, the service layer can first determine the object storage devices in other regions of the target bucket, and then determine whether object X is stored in the object storage devices in these other regions. That is, the service layer can determine the second object storage device based on the target bucket, and determine whether object X is stored in the second object storage device. If object X is not stored in the second object storage device, the service layer can store object X in the second storage device. In this way, object X stored in the target bucket in the first region will be synchronized to the second region by the service layer. Then the client in the second region can also obtain object X from the object storage device corresponding to the target bucket in the second region. In this way, by synchronizing the target bucket, it can be ensured that the target bucket stores the same objects in the object storage device in each region. The client in each region can obtain the objects in the target bucket from the object storage device in this region. Optionally, if the service layer synchronizes the object storage devices corresponding to the target bucket in different regions, the target bucket can also be called a global bucket, indicating that the objects stored in the target bucket in the object storage devices in each region are the same.

[0148] After synchronizing the target bucket, the service layer can obtain the newly added objects in the target bucket. Specifically, the first server can obtain the newly added objects from the object storage device corresponding to the target bucket in the area to which the first client belongs. In other words, if clients in other areas store objects in the target bucket, the service layer can synchronize the newly added objects to the object storage device in the area where the first client is located, and then the first server can obtain the newly added objects from the object storage device.

[0149] A detailed description is given below in conjunction with the accompanying drawings of the specification.

[0150] First, let's introduce the application scenario. Figure 6a , Figure 6a A schematic diagram of an application scenario of the object push method provided in the embodiment of the present application. Figure 6a In the application scenario shown, the object storage system includes a first client, a second client, a third client, a service layer, a first object storage device, and a second object storage device. The service layer includes a first server and a second server. The first client, the third client, the first server, and the first object storage device are located in a first area. The second client, the second server, and the second object storage device are located in a second area. Objects in a target storage bucket may be stored in the first object storage device and the second object storage device. Both the first client and the second client send an object subscription request for the target storage bucket to the service layer.

[0151] See also Figure 6b , Figure 6bA flow chart of the object push method provided in the embodiment of the present application. The method can be applied to the above Figure 6a The application scenario shown is used as an example for explanation. Specifically, Figure 6b The object push method shown may specifically include:

[0152] Step S601: The third client sends object X to the first server.

[0153] During the operation of the object storage system, the user can store objects in the object storage system through the client. The user using the third client can store object X in the target storage bucket through the third client. Since the third client is located in the first area, the third client can send object X to the server of the object storage system in this area. That is, the third client can send object X to the first server.

[0154] Specifically, the third client may send an object upload request to the first server. The first server may verify the object upload request, for example, it may authenticate the third client to determine whether the third client has the authority to upload the newly added object to the target storage bucket. After the verification is passed, the first server may send an object upload response to the third client. The third client may send object X to the first server according to the object upload response.

[0155] Step S602: The first server stores object X in a first object storage device.

[0156] After acquiring the object X, the first server may store the object X in the object storage device. In the first region, the objects in the target bucket may be stored in the first storage device. Therefore, the first server may store the object X in the first object storage device.

[0157] It is understandable that, since the first server and the second object storage device belong to different regions, the first server may not be able to directly store the object in the second object storage device. Therefore, the first server may not store the object X in the second object storage device.

[0158] Step S603: In response to object X being stored in the first object storage device, object X is synchronously stored in the second object storage device.

[0159] After object X is stored in the first object storage device, the service layer can synchronize the target storage bucket. Specifically, the service layer can synchronize and store object X in the second object storage device. In this way, object X is also stored in the second object storage device, and the client in the second area can obtain object X from the second object storage device. Optionally, the above-mentioned synchronous storage can be implemented based on the information interaction between the first object storage device and the second object storage device, which will not be described in detail here.

[0160] It is understandable that if the object storage system further includes a third object storage device located in a third region, after the object X is stored in the first object storage device, the object X may be synchronously stored in the third object storage device. Further, if the object storage system includes object storage devices located in N regions (N is a positive integer greater than 1), the object X may be synchronously stored in all regions of the N regions except the first region.

[0161] Step S604: The first server obtains object X from the first object storage device.

[0162] Step S605: The first server pushes the object X to the first client through the first transmission channel.

[0163] After storing object X in the first object storage device, the first server can determine that a new object has appeared in the target bucket. Assuming that object X matches the subscription topic of the first client, the first server can push object X to the first client through the first transmission channel. For an introduction to this part, please refer to Figure 3b The corresponding embodiments will not be described in detail here.

[0164] Step S606: The second server obtains object X from the second object storage device.

[0165] After storing object X in the second object storage system, the second server can determine that a new object has appeared in the target bucket. Assuming that object X matches the subscription topic of the second client, the second server can actively push object X to the second client. Specifically, the second server can obtain object X from the second object storage device.

[0166] Step S607: The second server pushes object X to the second client through the second transmission channel.

[0167] After obtaining object X from the second object storage system, the second server can push object X to the second client through the second transmission channel. In this way, even if a third client that is no longer in the same region as the second client uploads a new object to the target storage bucket, the target storage bucket can store the newly added object in the corresponding object storage device in the second region by synchronizing the first object storage device and the second object storage device, so that the server in the second region can actively push the newly added object to the second client.

[0168] In this way, by synchronizing the objects in the object storage devices corresponding to the target buckets in different regions, the target buckets can store the same objects in the object storage devices in each region, and the clients in each region can obtain the same objects from the object storage devices in their region. In this way, cross-region transmission and synchronization of objects are achieved.

[0169] The present application also provides an object pushing device, wherein the object pushing device can be applied to Figure 2 The object storage service layer in the implementation shown is used to implement Figure 3b or Figure 6b The functions of the service layer in the implementation shown are as follows. Figure 7 As shown, the object pushing device 700 includes:

[0170] The receiving module 710 is configured to receive an object subscription request from a first client for a target storage bucket, wherein the object subscription request carries a subscription topic.

[0171] A channel establishment module 720 is used to establish a first transmission channel with the first client in response to the object subscription request, wherein the first transmission channel is used to push the object to the first client after the service layer detects that the target storage bucket has uploaded the object;

[0172] The object acquisition module 730 is used to acquire a plurality of newly added objects in the target storage bucket according to the subscription topic, and the plurality of newly added objects are distributed and stored in the at least one object storage device.

[0173] The push module 740 is configured to push the multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

[0174] Among them, the receiving module 710, the channel establishment module 720, the object acquisition module 730 and the push module 740 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the receiving module 710 is introduced below by taking the receiving module 710 as an example. Similarly, the implementation of the channel establishment module 720, the object acquisition module 730 and the push module 740 can be the implementation of the receiving module 710.

[0175] As an example of a software functional unit, the receiving module 710 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the receiving module 710 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0176] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0177] As an example of a hardware functional unit, the receiving module 710 may include at least one computing device, such as a server, etc. Alternatively, the receiving module 710 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0178] The multiple computing devices included in the receiving module 710 can be distributed in the same region or in different regions. The multiple computing devices included in the receiving module 710 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the receiving module 710 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0179] It should be noted that, in other embodiments, the receiving module 710 can be used to execute any step in the object pushing method, the channel establishing module 720 can be used to execute any step in the object pushing method, the object acquiring module 730 can be used to execute any step in the object pushing method, and the pushing module 740 can be used to execute any step in the object pushing method. The steps implemented by the receiving module 710, the channel establishing module 720, the object acquiring module 730 and the pushing module 740 can be specified as needed, and the receiving module 710, the channel establishing module 720, the object acquiring module 730 and the pushing module 740 can respectively implement different steps in the object pushing method to implement the object. Full functionality of the device.

[0180] The present application also provides a computing device 800. Figure 8 As shown, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate through the bus 102. The computing device 800 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0181] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus 104 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 104 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0182] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0183] The memory 106 may include a volatile memory, such as a random access memory (RAM). The processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0184] The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned receiving module 110, channel establishment module 720, object acquisition module 730 and push module 740, thereby implementing the object push method. That is, the memory 106 stores instructions for executing this storage method.

[0185] Alternatively, the memory 106 stores executable codes, and the processor 104 executes the executable codes to respectively implement the functions of the aforementioned object storage client, object storage service layer, and object storage device, thereby implementing the object push method. That is, the memory 106 stores instructions for executing the object push method.

[0186] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0187] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0188] like Fig. 9 As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the object push method.

[0189] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the object push method. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the object push method.

[0190] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the object storage service layer. That is, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more modules of the receiving module 710, the channel establishment module 720, the object acquisition module 730 and the push module 740.

[0191] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.10 A possible implementation is shown. Fig.10 As shown, two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the receiving module 710, the channel establishment module 720, and the push module 740. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the object acquisition module 730.

[0192] Fig.10 The connection method between the computing device clusters shown may be based on the consideration that the object push method provided in the present application needs to interact with the object storage device, and the interaction may bring greater pressure, so it is considered to hand over the function of the object acquisition module 630 to the computing device 100B for execution.

[0193] It should be understood that Fig.10 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0194] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig. 9 and Fig.10 The connection mode of the computing device cluster is different in that the memory 106 in one or more computing devices 100A in the computing device cluster may store the same instructions for executing the object push method.

[0195] In some possible implementations, the memory of one or more computing devices 100B in the computing device cluster may also store partial instructions for executing the object push method. In other words, a combination of one or more computing devices may jointly execute instructions for executing the object push method.

[0196] It should be noted that the memory 106 in different computing devices 100A in the computing device cluster may store different instructions for executing part of the functions of the object storage. That is, the instructions stored in the memory 106 in different computing devices 100A may implement the functions of one or more devices in the object storage client, the object storage service layer, and the object storage device.

[0197] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the object push method.

[0198] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the object push method, or instruct the computing device to execute the object push method.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An object push method, characterized in that: The method is applied to a service layer of an object storage system, the service layer is used to access at least one object storage device, the at least one object storage device is located in a plurality of data centers in different regions, each data center includes a plurality of servers, the method includes: Receive an object subscription request from a first client for a target storage bucket, where the object subscription request carries a subscription topic; In response to the object subscription request, establishing a first transmission channel with the first client, wherein the first transmission channel is used to push the object to the first client after the service layer detects that the target storage bucket has uploaded the object; According to the subscription topic, a plurality of newly added objects in the target storage bucket are acquired, where the plurality of newly added objects are distributed and stored in the at least one object storage device; The multiple newly added objects in the target storage bucket are pushed to the first client through the first transmission channel.

2. The method according to claim 1, characterized in that The first transmission channel is established based on a WebSocket protocol, and the step of establishing the first transmission channel with the first client in response to the object subscription request includes: In response to the object subscription request, the Hypertext Transfer Protocol (HTTP) connection established between the first client and the service layer is switched to a WebSocket connection to establish a WebSocket transmission channel with the first client.

3. The method according to claim 1 or 2, characterized in that: The step of pushing the multiple newly added objects in the target storage bucket to the first client through the first transmission channel includes: The access domain names of the multiple newly added objects in the target storage bucket are pushed to the first client through the first transmission channel.

4. The method according to any one of claims 1 to 3, characterized in that: The subscription topic includes at least one of the following: a client identifier to which the multiple newly added objects belong, a category to which the multiple newly added objects belong, a directory where the multiple newly added objects are located, a storage time of the multiple newly added objects, or a storage type of the multiple newly added objects, wherein the storage type is used to indicate an access frequency of the multiple newly added objects.

5. The method according to any one of claims 1 to 4, characterized in that: Before establishing a first transmission channel with the first client in response to the object subscription request, the method includes: According to the configuration information of the first client, it is determined that the target storage bucket opens the first transmission channel.

6. The method according to any one of claims 1 to 5, characterized in that: The multiple newly added objects in the target bucket include a first object and a second object, the first object is stored in a first object storage device located in a first area, and the second object is stored in a second object storage device located in a second area, and the method includes: synchronizing the first object from the first object storage device to the second object storage device; The multiple newly added objects in the target storage bucket are pushed to a second client through a second transmission channel, where the second client is set in the second region.

7. An object pushing device, characterized in that: The device is applied to a service layer of an object storage system, the service layer is used to access at least one object storage device, the at least one object storage device is located in multiple data centers in different regions, each data center includes multiple servers, and the device includes: A receiving module, configured to receive an object subscription request from a first client for a target storage bucket, wherein the object subscription request carries a subscription topic; a channel establishment module, configured to establish a first transmission channel with the first client in response to the object subscription request, wherein the first transmission channel is used to push the object to the first client after the service layer detects that the target storage bucket has uploaded the object; An acquisition module, configured to acquire, according to the subscription topic, a plurality of newly added objects in the target storage bucket, wherein the plurality of newly added objects are distributed and stored in the at least one object storage device; A push module is used to push the multiple newly added objects in the target storage bucket to the first client through the first transmission channel.

8. The device according to claim 7, characterized in that The first transmission channel is established based on the WebSocket protocol, and the channel establishment module is specifically used to: In response to the object subscription request, the Hypertext Transfer Protocol (HTTP) connection established between the first client and the service layer is switched to a WebSocket connection to establish a WebSocket transmission channel with the first client.

9. The device according to claim 7 or 8, characterized in that The push module is specifically used for: The access domain names of the multiple newly added objects in the target storage bucket are pushed to the first client through the first transmission channel.

10. The device according to any one of claims 7 to 9, characterized in that The subscription topic includes at least one of the following: a client identifier to which the multiple newly added objects belong, a category to which the multiple newly added objects belong, a directory where the multiple newly added objects are located, a storage time of the multiple newly added objects, or a storage type of the multiple newly added objects, wherein the storage type is used to indicate an access frequency of the multiple newly added objects.

11. The device according to any one of claims 7 to 10, characterized in that The channel establishment module is also used for: According to the configuration information of the first client, it is determined that the target storage bucket opens the first transmission channel.

12. The device according to any one of claims 7 to 11, characterized in that The multiple newly added objects in the target storage bucket include a first object and a second object, the first object is stored in a first object storage device located in a first area, and the second object is stored in a second object storage device located in a second area, and the apparatus further includes a synchronization module: The synchronization module is used to synchronize the first object from the first object storage device to the second object storage device; The push module is further configured to push the multiple newly added objects in the target storage bucket to a second client via a second transmission channel, where the second client is set in the second region.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images