Metadata transmission optimization method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a metadata transmission optimization method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: when a metadata acquisition request from a client is received; analyzing the metadata acquisition request to obtain a metadata category required by the client; based on the metadata category, determining a target interface server storing metadata fragments of the metadata category from a preset number of interface servers; the fragmented metadata is stored in each interface server, and the fragmented metadata is obtained by fragmenting all the metadata stored in the distributed key value database; and obtaining a metadata fragment of the metadata category from the target interface server, and sending the metadata fragment to the client. By adopting the method, OOM caused by too large metadata on a single interface server component or concurrent processing of a plurality of acquisition requests can be avoided, and the stability is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud native technology, and in particular to a metadata transmission optimization method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In cloud-native containerization technology, building ultra-large-scale Kubernetes clusters has become the norm, with the number of nodes easily exceeding 10,000, and the number of deployed Pods (the smallest computing unit, which can contain one or more containers) can easily exceed 100,000. These large numbers of business Pods and the corresponding Operators (automated tools for managing applications) need to interact with the Kubernetes control plane. In particular, after starting, Kubelet (Kubernetes' node agent) will obtain a large amount of metadata from the interface server (the core component of the Kubernetes control plane).

[0003] As the scale of Kubernetes clusters expands, the amount of metadata also increases dramatically. The interface server needs to manage and package these metadata in memory. However, when the amount of metadata is large, it will occupy a lot of memory resources. In particular, when multiple requests access the interface server at the same time, memory overflow (OOM) may occur, which will seriously affect the stability of the Kubernetes cluster and may even cause a cluster avalanche. Summary of the invention

[0004] Based on this, it is necessary to provide a metadata transmission optimization method, device, computer equipment, computer-readable storage medium and computer program product that can store and transmit metadata according to the load of the interface server to address the above technical problems.

[0005] In a first aspect, the present application provides a metadata transmission optimization method, comprising:

[0006] When receiving a metadata acquisition request from a client; parsing the metadata acquisition request to obtain the metadata category required by the client;

[0007] Based on the metadata category, determining a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, wherein the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0008] In one embodiment, obtaining the metadata slices of the metadata category from the target interface server and sending the metadata slices to the client includes:

[0009] Acquire the metadata fragment of the metadata category and the resource version number of the metadata under the metadata category from the target interface server;

[0010] The metadata fragment and the resource version number are sent to the client.

[0011] In one embodiment, it also includes:

[0012] Obtaining the load status of all interface servers at first preset intervals;

[0013] Based on the load status, the metadata slices stored by the interface server are updated.

[0014] In one embodiment, it also includes:

[0015] Every second preset time period, obtain the health status and load status of all interface servers;

[0016] The metadata slice stored by the interface server whose health status is abnormal is used as the missing slice;

[0017] Based on the load status and the health status, an expected interface server is determined, and the missing slices are allocated to the expected interface server for storage.

[0018] In one embodiment, obtaining the health status of all interface servers every second preset time period includes:

[0019] Every second preset time period, sending a health query request to all interface servers to query the health status of each interface server;

[0020] If a query reply is received within a third preset time period, determining that the health status of the interface server corresponding to the query reply is normal, and determining the load status of the corresponding interface server based on the query reply; the third preset time period is less than the second preset time period;

[0021] The health status of the interface servers other than the interface servers in the normal status is marked as abnormal status.

[0022] In one embodiment, determining the desired interface server based on the load status and the health status includes:

[0023] Based on the load status, it is determined that among the interface servers whose health status is normal, an expected interface server that needs to take over the missing slice.

[0024] In a second aspect, the present application also provides a metadata transmission optimization device, comprising:

[0025] The receiving module is used for, when receiving a metadata acquisition request from a client, parsing the metadata acquisition request to obtain the metadata category required by the client;

[0026] A determination module is used to determine, based on the metadata category, a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, and the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0027] The transmission module is used to obtain the metadata slices of the metadata category from the target interface server and send the metadata slices to the client.

[0028] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0029] When receiving a metadata acquisition request from a client; parsing the metadata acquisition request to obtain the metadata category required by the client;

[0030] Based on the metadata category, determining a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, wherein the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0031] The metadata slices of the metadata category are obtained from the target interface server, and the metadata slices are sent to the client.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0033] When receiving a metadata acquisition request from a client; parsing the metadata acquisition request to obtain the metadata category required by the client;

[0034] Based on the metadata category, determining a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, wherein the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0035] The metadata slices of the metadata category are obtained from the target interface server, and the metadata slices are sent to the client.

[0036] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0037] When receiving a metadata acquisition request from a client; parsing the metadata acquisition request to obtain the metadata category required by the client;

[0038] Based on the metadata category, determining a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, wherein the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0039] The metadata slices of the metadata category are obtained from the target interface server, and the metadata slices are sent to the client.

[0040] The above-mentioned metadata transmission optimization method, device, computer equipment, computer-readable storage medium and computer program product, when receiving a metadata acquisition request from a client; parse the metadata acquisition request to obtain the metadata category required by the client; based on the metadata category, determine the target interface server storing the metadata shards of the metadata category from a preset number of interface servers; each interface server stores the sharded metadata, and the sharded metadata is obtained by sharding all metadata stored in the distributed key-value database; obtain the metadata shards of the metadata category from the target interface server, and send the metadata shards to the client. Through the load conditions of all interface servers, the metadata shards stored in the interface servers are dynamically balanced, and when a request for data acquisition from the client is received, metadata is acquired from the corresponding interface servers and sent to the client respectively. It can avoid the metadata on a single interface server component being too large or OOM caused by concurrent processing of multiple acquisition requests, greatly improving the stability of ultra-large-scale Kubernetes clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0042] FIG1 is a diagram showing an application environment of a metadata transmission optimization method according to an embodiment;

[0043] FIG2 is a schematic diagram of a flow chart of a metadata transmission optimization method according to an embodiment;

[0044] FIG3 is a schematic diagram of checking the health status of an interface server in one embodiment;

[0045] FIG4 is a schematic diagram of a method for optimizing metadata transmission in one embodiment;

[0046] FIG5 is a structural block diagram of a metadata transmission optimization device in one embodiment;

[0047] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The metadata transmission optimization method provided in the embodiment of the present application can be applied to the application environment shown in FIG. 1. The client 102 communicates with the interface server 106 through the gateway 104. The distributed key-value database can store the data that the gateway 104 needs to process. The distributed key-value database can be placed on the cloud or other network servers. When the gateway 104 receives a metadata acquisition request from the client 102; the metadata acquisition request is parsed to obtain the metadata category required by the client 102; based on the metadata category, the target interface server 106 storing the metadata shards of the metadata category is determined from a preset number of interface servers 106; each interface server 106 stores the sharded metadata, and the sharded metadata is obtained by sharding all the metadata stored in the distributed key-value database; the metadata shards of the metadata category are obtained from the target interface server 106, and the metadata shards are sent to the client 102. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The interface server 106 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0050] In an exemplary embodiment, as shown in FIG2 , a metadata transmission optimization method is provided, which is described by taking the method applied to the gateway 104 in FIG1 as an example, and includes the following steps 202 to 206. Among them:

[0051] Step 202: When a metadata acquisition request is received from a client, the metadata acquisition request is parsed to obtain the metadata category required by the client.

[0052] Among them, in Kubernetes, metadata is an important part of API objects, which is used to describe the basic information and properties of objects. Each API object in Kubernetes (such as Pod, Deployment, Service, etc.) contains metadata fields.

[0053] Optionally, the client may be an application on a device such as a computer. The acquisition request may be a List request, which is used to acquire a series of key-value pairs stored in a distributed key-value database.

[0054] Exemplarily, when a metadata acquisition request is received from a client, the metadata acquisition request is parsed to obtain the metadata category required by the client.

[0055] Step 204: Based on the metadata category, determine the target interface server storing the metadata shards of the metadata category from a preset number of interface servers; each interface server stores the sharded metadata, and the sharded metadata is obtained by sharding all the metadata stored in the distributed key-value database.

[0056] Optionally, the interface server may be a Kubernetes API Server (Kube-Apiserver); the distributed key-value database may be ETCD.

[0057] Optionally, the preset number may be determined according to the load of the cluster. The greater the load, the greater the number of interface servers.

[0058] Exemplarily, based on the metadata category, a target interface server storing metadata shards of the corresponding metadata category is determined from a preset number of interface servers; each interface server stores sharded metadata, and the sharded metadata is obtained by sharding all metadata stored in the distributed key-value database.

[0059] Step 206: Obtain metadata slices of the metadata category from the target interface server, and send the metadata slices to the client.

[0060] Optionally, after obtaining the metadata fragments, you can package them together or send them separately.

[0061] Exemplarily, metadata slices of corresponding metadata categories are obtained from the target interface server, and the metadata slices are sent to the client one by one.

[0062] The above-mentioned metadata transmission optimization method, device, computer equipment, computer-readable storage medium and computer program product, when receiving a metadata acquisition request from a client; parse the metadata acquisition request to obtain the metadata category required by the client; based on the metadata category, determine the target interface server storing the metadata shards of the metadata category from a preset number of interface servers; each interface server stores the sharded metadata, and the sharded metadata is obtained by sharding all metadata stored in the distributed key-value database; obtain the metadata shards of the metadata category from the target interface server, and send the metadata shards to the client. Through the load conditions of all interface servers, dynamically balance the metadata shards stored in the interface servers, and when receiving a request from the client to obtain data, obtain metadata from the corresponding interface servers and send them to the client respectively. It can avoid the metadata on a single interface server component being too large or OOM caused by concurrent processing of multiple acquisition requests, greatly improving stability.

[0063] In an exemplary embodiment, acquiring the metadata slices of the metadata category from the target interface server and sending the metadata slices to the client includes:

[0064] Obtaining metadata fragments of the metadata category and resource version numbers of metadata under the metadata category from the target interface server; and sending the metadata fragments and the resource version numbers to the client.

[0065] The resource version number is Resource Version, which represents the current metadata version.

[0066] Exemplarily, metadata fragments of a corresponding metadata category and resource version numbers of metadata under the corresponding metadata category are obtained from the target interface server; the metadata fragments are first sent to the client one by one, and finally the resource version number is sent to the client.

[0067] In this embodiment, by sending the resource version number to the client, the client can confirm the version of the currently acquired metadata, ensuring that the metadata read by the client is in the latest state.

[0068] In an exemplary embodiment, the method further comprises:

[0069] The load status of all interface servers is obtained at first preset time intervals; and based on the load status, the metadata slices stored in the interface servers are updated.

[0070] The first preset time length may be 10 seconds.

[0071] Among them, the metadata in ETCD is constantly changing, and the metadata in the interface server comes from ETCD, so it also changes according to ETCD.

[0072] Exemplarily, the load status of all interface servers is obtained every 10 seconds, and according to the load status, the metadata stored in the interface servers with greater load pressure are sharded and distributed to the interface servers with less load pressure.

[0073] In this embodiment, the metadata slices stored in the interface server are dynamically adjusted according to the load status of the interface server, which can effectively prevent a certain interface server from having a memory overflow due to excessive load.

[0074] In an exemplary embodiment, the method further comprises:

[0075] At every second preset time interval, the health status and load status of all interface servers are obtained; the metadata shards stored in the interface server whose health status is abnormal are used as missing shards; based on the load status and the health status, the expected interface server is determined, and the missing shards are allocated to the expected interface server for storage.

[0076] Optionally, the second preset duration may be 60 seconds.

[0077] Exemplarily, as shown in FIG3 , Kube-Gateway is the gateway applied by this method, which obtains the health status and load status of all interface servers (Kube-Apiserver) every 60 seconds; the metadata shard Pod-2 stored in the interface server whose health status is abnormal is used as the missing shard; based on the load status and the health status, the expected interface server is determined, and the missing shard Pod-2 is allocated to the expected interface server for storage.

[0078] In this embodiment, the stability of metadata transmission can be ensured by allocating the missing shards in the interface server in the abnormal state to the interface server in the normal state.

[0079] In an exemplary embodiment, obtaining the health status of all interface servers every second preset time period includes:

[0080] Every second preset time period, a health query request is sent to all interface servers to query the health status of each interface server; if a query reply is received within a third preset time period, the health status of the interface server corresponding to the query reply is determined to be normal, and based on the query reply, the load status of the corresponding interface server is determined; the third preset time period is less than the second preset time period; the health status of interface servers other than the interface servers in the normal state is marked as abnormal.

[0081] Optionally, the third preset time length may be 5 seconds.

[0082] Exemplarily, every 60 seconds, a health query request is sent to all interface servers to query the health status of each interface server; if a query reply is received within 5 seconds, the health status of the interface server corresponding to the query reply is determined to be normal, and based on the query reply, the load status of the corresponding interface server is determined; all interface servers other than the interface servers in normal status are regarded as interface servers in abnormal status.

[0083] In this embodiment, by periodically acquiring the health status of the interface server, abnormal interface servers are promptly checked to prevent the transmission of metadata from being affected by abnormalities in the interface servers.

[0084] In an exemplary embodiment, determining the desired interface server based on the load status and the health status includes:

[0085] Based on the load status, it is determined that among the interface servers whose health status is normal, an expected interface server that needs to take over the missing slice.

[0086] Exemplarily, based on the load status and the missing slices, it is determined which interface servers in the normal health state can bear the missing slices, and these interface servers are used as expected servers.

[0087] In this embodiment, the expected interface server is determined by the load status, so that the interface server that can bear the missing slice can be determined, thereby preventing a new interface server from being in an abnormal state.

[0088] In an exemplary embodiment, as shown in FIG4 , when a metadata acquisition request, i.e., a List request, is received from a client (Client); the metadata acquisition request is parsed to obtain the metadata category required by the client. Based on the metadata category, the target interface server storing the metadata shards (Pod-0, Pod-1, and Pod-2) of the corresponding metadata Pod category is determined from a preset number of interface servers (Kube-Apiserver); each interface server stores the sharded metadata, which is obtained by sharding all metadata stored in the distributed key-value database. The metadata shards of the corresponding metadata category and the resource version number of the metadata under the corresponding metadata category are obtained from the target interface server; the metadata shards are sent to the client one by one, and the resource version number ResourceVersion is sent to the client. Every 10 seconds, the load status of all interface servers is obtained, and according to the load status, the metadata shards stored in the interface servers with greater load pressure are distributed to the interface servers with less load. Every 60 seconds, a health query request is sent to all interface servers to query the health status of each interface server; if a query reply is received within 5 seconds, the health status of the interface server corresponding to the query reply is determined to be normal, and the load status of the corresponding interface server is determined based on the query reply; all interface servers other than the interface servers in normal status are regarded as interface servers in abnormal status. The metadata shards stored by the interface servers with abnormal health status are regarded as missing shards; based on the load status and missing shards, determine which interface servers with normal health status can bear the missing shards, regard these interface servers as expected servers, and allocate the missing shards to the expected interface servers for storage.

[0089] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0090] In an exemplary embodiment, as shown in FIG5 , a metadata transmission optimization device is provided, including: a receiving module, a determining module and a transmitting module, wherein:

[0091] The receiving module is used for, when receiving a metadata acquisition request from a client, parsing the metadata acquisition request to obtain the metadata category required by the client;

[0092] A determination module is used to determine, based on the metadata category, a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, and the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0093] The transmission module is used to obtain the metadata slices of the metadata category from the target interface server and send the metadata slices to the client.

[0094] In one embodiment, the transmission module is further configured to:

[0095] Obtaining metadata fragments of the metadata category and resource version numbers of metadata under the metadata category from the target interface server; and sending the metadata fragments and the resource version numbers to the client.

[0096] In one embodiment, the system further comprises: an adjustment module for:

[0097] The load status of all interface servers is obtained at first preset time intervals; and based on the load status, the metadata slices stored in the interface servers are updated.

[0098] In one embodiment, the invention further comprises: a checking module, which is used to:

[0099] At every second preset time interval, the health status and load status of all interface servers are obtained; the metadata shards stored in the interface server whose health status is abnormal are used as missing shards; based on the load status and the health status, the expected interface server is determined, and the missing shards are allocated to the expected interface server for storage.

[0100] In one embodiment, the inspection module is further configured to:

[0101] Every second preset time period, a health query request is sent to all interface servers to query the health status of each interface server; if a query reply is received within a third preset time period, the health status of the interface server corresponding to the query reply is determined to be normal, and based on the query reply, the load status of the corresponding interface server is determined; the third preset time period is less than the second preset time period; the health status of interface servers other than the interface servers in the normal state is marked as abnormal.

[0102] In one embodiment, the inspection module is further configured to:

[0103] Based on the load status, it is determined that among the interface servers whose health status is normal, an expected interface server that needs to take over the missing slice.

[0104] Each module in the above-mentioned metadata transmission optimization device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0105] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store metadata. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a metadata transmission optimization method is implemented.

[0106] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0107] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0108] When receiving a metadata acquisition request from a client; parsing the metadata acquisition request to obtain the metadata category required by the client;

[0109] Based on the metadata category, determining a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, wherein the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0110] The metadata slices of the metadata category are obtained from the target interface server, and the metadata slices are sent to the client.

[0111] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0112] Acquire the metadata fragment of the metadata category and the resource version number of the metadata under the metadata category from the target interface server;

[0113] The metadata fragment and the resource version number are sent to the client.

[0114] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0115] The load status of all interface servers is obtained at first preset time intervals; and based on the load status, the metadata slices stored in the interface servers are updated.

[0116] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0117] At every second preset time interval, the health status and load status of all interface servers are obtained; the metadata shards stored in the interface server whose health status is abnormal are used as missing shards; based on the load status and the health status, the expected interface server is determined, and the missing shards are allocated to the expected interface server for storage.

[0118] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0119] Every second preset time period, a health query request is sent to all interface servers to query the health status of each interface server; if a query reply is received within a third preset time period, the health status of the interface server corresponding to the query reply is determined to be normal, and based on the query reply, the load status of the corresponding interface server is determined; the third preset time period is less than the second preset time period; the health status of interface servers other than the interface servers in the normal state is marked as abnormal.

[0120] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0121] Based on the load status, it is determined that among the interface servers whose health status is normal, an expected interface server that needs to take over the missing slice.

[0122] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0123] When receiving a metadata acquisition request from a client; parsing the metadata acquisition request to obtain the metadata category required by the client;

[0124] Based on the metadata category, determining a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, wherein the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0125] The metadata slices of the metadata category are obtained from the target interface server, and the metadata slices are sent to the client.

[0126] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0127] Acquire the metadata fragment of the metadata category and the resource version number of the metadata under the metadata category from the target interface server;

[0128] The metadata fragment and the resource version number are sent to the client.

[0129] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0130] The load status of all interface servers is obtained at first preset time intervals; and based on the load status, the metadata slices stored in the interface servers are updated.

[0131] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0132] At every second preset time interval, the health status and load status of all interface servers are obtained; the metadata shards stored in the interface server whose health status is abnormal are used as missing shards; based on the load status and the health status, the expected interface server is determined, and the missing shards are allocated to the expected interface server for storage.

[0133] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0134] Every second preset time period, a health query request is sent to all interface servers to query the health status of each interface server; if a query reply is received within a third preset time period, the health status of the interface server corresponding to the query reply is determined to be normal, and based on the query reply, the load status of the corresponding interface server is determined; the third preset time period is less than the second preset time period; the health status of interface servers other than the interface servers in the normal state is marked as abnormal.

[0135] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0136] Based on the load status, it is determined that among the interface servers whose health status is normal, an expected interface server that needs to take over the missing slice.

[0137] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0138] When receiving a metadata acquisition request from a client; parsing the metadata acquisition request to obtain the metadata category required by the client;

[0139] Based on the metadata category, determining a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, wherein the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database;

[0140] The metadata slices of the metadata category are obtained from the target interface server, and the metadata slices are sent to the client.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0142] Acquire the metadata fragment of the metadata category and the resource version number of the metadata under the metadata category from the target interface server;

[0143] The metadata fragment and the resource version number are sent to the client.

[0144] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0145] The load status of all interface servers is obtained at first preset time intervals; and based on the load status, the metadata slices stored in the interface servers are updated.

[0146] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0147] At every second preset time interval, the health status and load status of all interface servers are obtained; the metadata shards stored in the interface server whose health status is abnormal are used as missing shards; based on the load status and the health status, the expected interface server is determined, and the missing shards are allocated to the expected interface server for storage.

[0148] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0149] Every second preset time period, a health query request is sent to all interface servers to query the health status of each interface server; if a query reply is received within a third preset time period, the health status of the interface server corresponding to the query reply is determined to be normal, and based on the query reply, the load status of the corresponding interface server is determined; the third preset time period is less than the second preset time period; the health status of interface servers other than the interface servers in the normal state is marked as abnormal.

[0150] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0151] Based on the load status, it is determined that among the interface servers whose health status is normal, an expected interface server that needs to take over the missing slice.

[0152] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0153] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0154] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A metadata transmission optimization method, characterized in that: include: When receiving a metadata acquisition request from a client; Parsing the metadata acquisition request to obtain the metadata category required by the client; Based on the metadata category, determining a target interface server storing metadata slices of the metadata category from a preset number of interface servers; Each interface server stores sharded metadata, which is obtained by sharding all metadata stored in the distributed key-value database; The metadata slices of the metadata category are obtained from the target interface server, and the metadata slices are sent to the client.

2. The method according to claim 1, characterized in that The acquiring the metadata slices of the metadata category from the target interface server and sending the metadata slices to the client comprises: Acquire the metadata fragment of the metadata category and the resource version number of the metadata under the metadata category from the target interface server; The metadata fragment and the resource version number are sent to the client.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining the load status of all interface servers at first preset intervals; Based on the load status, the metadata slices stored by the interface server are updated.

4. The method according to claim 1, characterized in that The method further comprises: Every second preset time period, obtain the health status and load status of all interface servers; The metadata slice stored by the interface server whose health status is abnormal is used as the missing slice; Based on the load status and the health status, an expected interface server is determined, and the missing slices are allocated to the expected interface server for storage.

5. The method according to claim 4, characterized in that The step of obtaining the health status of all interface servers at every second preset time period includes: Every second preset time period, sending a health query request to all interface servers to query the health status of each interface server; If a query reply is received within a third preset time period, determining that the health status of the interface server corresponding to the query reply is normal, and determining the load status of the corresponding interface server based on the query reply; the third preset time period is less than the second preset time period; The health status of the interface servers other than the interface servers in the normal status is marked as abnormal status.

6. The method according to claim 4, characterized in that The determining of the desired interface server based on the load status and the health status includes: Based on the load status, it is determined that among the interface servers whose health status is normal, an expected interface server that needs to take over the missing slice.

7. A metadata transmission optimization device, characterized in that: The device comprises: The receiving module is used for, when receiving a metadata acquisition request from a client, parsing the metadata acquisition request to obtain the metadata category required by the client; A determination module is used to determine, based on the metadata category, a target interface server storing metadata shards of the metadata category from a preset number of interface servers; each interface server stores sharded metadata, and the sharded metadata is obtained by sharding all metadata stored in a distributed key-value database; The transmission module is used to obtain the metadata slices of the metadata category from the target interface server and send the metadata slices to the client.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images