Multi-cluster resource processing method and device

Abstract

The invention discloses a multi-cluster resource processing method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the steps that in response to a resource processing request sent by a client, at least one target cluster is determined according to the resource processing request, and the resource processing request comprises a resource type and an operation type; determining a request forwarding path according to the resource type and the operation type; according to the request forwarding path, allocating the resource processing request to at least one target cluster for processing to obtain a processing result of each target cluster; and aggregating the processing results of the target clusters to obtain a resource processing result. According to the embodiment, resource aggregation processing across multiple clusters can be realized in a small-scale and simple multi-cluster scene or a complex multi-cluster scene, the scene adaptability is high, and the implementation is simple; and meanwhile, different resource aggregation processing is carried out on different resource types and operation types, so that the flexibility of resource aggregation processing is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for processing multi-cluster resources. Background Technology

[0002] In modern distributed systems, enterprises typically use multiple Kubernetes clusters (a type of application management system for containerized applications across multiple hosts in a cloud platform) to manage and deploy applications. These clusters may be distributed across different geographical locations or different cloud service providers to improve system availability, scalability, and disaster recovery capabilities. Currently, when performing resource operations (such as fetching, updating, and sorting) across multiple Kubernetes clusters, a common solution is to use a multi-cluster management framework for unified multi-cluster management and scheduling.

[0003] While existing solutions can provide unified multi-cluster management and scheduling, the multi-cluster management framework requires the deployment and maintenance of a complex control plane and agent system, which increases the complexity and operation and maintenance costs of the system and is not suitable for some simple business scenarios. Furthermore, when dealing with complex business operation requirements, additional custom development is required, lacking flexible customization capabilities. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method and apparatus for multi-cluster resource processing, which can determine the request forwarding path based on resource type and operation type to distribute the request to multiple clusters for processing, and aggregate the processing results of multiple clusters. Whether it is a small-scale and simple multi-cluster scenario or a complex multi-cluster scenario, resource aggregation processing across multiple clusters can be realized. It has strong scenario adaptability and is simple to implement. At the same time, different resource aggregation processing is performed on different resource types and operation types, which improves the flexibility of resource aggregation processing.

[0005] To achieve the above objectives, according to one aspect of the present invention, a method for multi-cluster resource processing is provided, comprising: In response to a resource processing request from a client, at least one target cluster is determined based on the resource processing request, wherein the resource processing request includes a resource type and an operation type; Determine the request forwarding path based on the resource type and the operation type; Based on the request forwarding path, the resource processing request is allocated to the at least one target cluster for processing, and the processing results of each target cluster are obtained. The processing results of each target cluster are aggregated to obtain the resource processing result.

[0006] Optionally, the resource processing request further includes a cluster and resource identifier specified by the client; determining at least one target cluster based on the resource processing request includes: determining the target cluster from the configured clusters based on the resource types included in the resource processing request and the resource types that each configured cluster can provide; or, determining the target cluster based on the cluster specified by the client included in the resource processing request; or, determining the target cluster from the configured clusters based on the resource identifier included in the resource processing request and the resource types that each configured cluster can provide.

[0007] Optionally, before allocating the resource processing request to the at least one target cluster for processing according to the request forwarding path, the method further includes: determining request encapsulation logic based on the resource type and the operation type; encapsulating the resource processing request to obtain a standard request according to the request encapsulation logic; and allocating the resource processing request to the at least one target cluster for processing according to the request forwarding path, including: allocating the standard request to the at least one target cluster for processing according to the request forwarding path.

[0008] Optionally, encapsulating the resource processing request into a standard request according to the request encapsulation logic includes: converting the resource processing request into a standard internal operation object according to the request encapsulation logic, and converting the general operations in the internal operation object into specific operations of the target cluster to obtain the standard request.

[0009] Optionally, determining the request forwarding path based on the resource type and the operation type includes: determining a plugin configuration unit based on the resource type and the operation type, and determining the request forwarding path corresponding to the resource type through the plugin configuration unit.

[0010] Optionally, the method further includes: in response to the addition of a new resource type, generating and registering a new resource type plugin configuration unit so as to load the new resource type plugin configuration unit when the system starts.

[0011] Optionally, allocating the resource processing request to the at least one target cluster for processing includes: determining a request allocation method based on the health status, load of each target cluster and the priority of the resource processing request; and allocating the resource processing request to the at least one target cluster for processing according to the request allocation method.

[0012] Optionally, before aggregating the processing results of each target cluster, the method further includes: performing error detection on the processing results of each target cluster, and determining whether a retry is needed; if a retry is needed, retrying the execution and updating the processing results, determining the execution success rate, the list of successfully executed clusters, the list of failed clusters, and the reasons for failure based on the updated processing results, to generate partially successful information; aggregating the processing results of each target cluster to obtain resource processing results, including: aggregating the processing results of successfully executed target clusters to obtain an aggregation result, converting the aggregation result into a format conforming to the cluster interface specification, and adding aggregation-related metadata and the partially successful information to obtain resource processing results.

[0013] Optionally, the processing results of each target cluster are aggregated, including: deduplicating the processing results of each target cluster according to the resource identifier, and merging the deduplicated processing results according to the merging rules, wherein the merging rules include: in the case of resource conflicts, selecting the resource with the highest resource version number, the resource with the latest timestamp, or the resource with the highest cluster priority.

[0014] According to another aspect of the present invention, an apparatus for multi-cluster resource processing is provided, comprising: The target cluster determination module is used to determine at least one target cluster in response to a resource processing request sent by a client, wherein the resource processing request includes a resource type and an operation type. A forwarding path determination module is used to determine a request forwarding path based on the resource type and the operation type; The request allocation processing module is used to allocate the resource processing request to the at least one target cluster for processing according to the request forwarding path, and obtain the processing results of each target cluster. The processing result aggregation module is used to aggregate the processing results of each target cluster to obtain the resource processing results.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multi-cluster resource processing method provided in the embodiments of the present invention.

[0016] According to another aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method for multi-cluster resource processing provided in the embodiments of the present invention.

[0017] According to another aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the multi-cluster resource processing method provided in the embodiments of the present invention.

[0018] One embodiment of the above invention has the following advantages or beneficial effects: By responding to a resource processing request sent by a client, determining at least one target cluster based on the resource processing request, the resource processing request including resource type and operation type; determining a request forwarding path based on the resource type and operation type; allocating the resource processing request to at least one target cluster for processing according to the request forwarding path, obtaining the processing results of each target cluster; and aggregating the processing results of each target cluster to obtain the resource processing result, this technical solution, by determining the request forwarding path based on resource type and operation type to allocate the request to multiple clusters for processing, and aggregating the processing results of multiple clusters, can achieve resource aggregation processing across multiple clusters, whether in small-scale, simple multi-cluster scenarios or complex multi-cluster scenarios. It has strong scenario adaptability and is simple to implement; at the same time, it performs different resource aggregation processing on different resource types and operation types, improving the flexibility of resource aggregation processing.

[0019] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0020] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram of the main steps of a multi-cluster resource processing method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the aggregation process of the processing results according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the multi-cluster resource processing flow according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the main modules of a multi-cluster resource processing device according to an embodiment of the present invention; Figure 5 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied; Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0021] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] It should be noted that the technical solutions disclosed in this invention, regarding the collection, updating, analysis, processing, use, transmission, and storage of user personal information, all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0023] To address the technical problems existing in the prior art, this invention provides a method for multi-cluster resource processing. Based on a custom interface aggregation service, it can determine the request forwarding path to distribute requests to multiple clusters for processing and aggregate the processing results of multiple clusters, thereby realizing resource aggregation query and operation across multiple clusters. This provides a simple, flexible, and easy-to-maintain solution.

[0024] Figure 1 This is a schematic diagram illustrating the main steps of a multi-cluster resource processing method according to an embodiment of the present invention. Figure 1 As shown, the multi-cluster resource processing method of this invention mainly includes the following steps S101 to S104.

[0025] Step S101: In response to a resource processing request from a client, determine at least one target cluster based on the resource processing request. The resource processing request includes a resource type and an operation type. Examples of resource processing requests include resource query requests and resource sorting requests. In this embodiment of the invention, an interface router is used to route the resource processing request to the target cluster for processing. The interface router, as the system's entry point, is responsible for receiving all requests from clients and forwarding these requests to a custom interface aggregation service on the server side according to certain rules. This design pattern allows the system to centrally handle cross-cutting concerns such as authentication, authorization, and traffic control, while providing a unified access interface for clients.

[0026] According to one embodiment of the present invention, the resource processing request further includes a cluster and a resource identifier specified by the client. The resource identifier may include, for example, a resource name, a resource namespace, etc. Determining at least one target cluster based on the resource processing request may specifically include: determining the target cluster from the configured clusters based on the resource types included in the resource processing request and the resource types that each configured cluster can provide; or, determining the target cluster based on the cluster specified by the client included in the resource processing request; or, determining the target cluster from the configured clusters based on the resource identifier included in the resource processing request and the resource types that each configured cluster can provide.

[0027] According to embodiments of the present invention, the interface router determines how to forward requests based on the request path of the resource processing request. Common forwarding rules include: matching the target cluster and forwarding the request based on the resource type of the Kubernetes cluster; forwarding the request based on the target cluster or cluster group specified by the client, wherein the target cluster or cluster group specified by the client can be obtained through query parameters, request headers, or path parameters; filtering resources according to the resource namespace and further determining the target cluster for request forwarding, etc.

[0028] According to an embodiment of the present invention, the system maintains a cluster connection pool. The connection information for each cluster includes: interface server address, authentication information, connection status (active / inactive), etc. When a user initiates a resource processing request (such as retrieving a list of all containers), the system sends the request in parallel to all connected Kubernetes clusters and uses a thread pool or asynchronous processing mechanism to process the requests from multiple clusters simultaneously. Finally, the system collects the results returned by all clusters and merges them.

[0029] The resource processing request includes information such as resource type, operation type, resource identifier, query parameters, and target cluster. Resource type includes core resource types such as containers, services, and deployments; user-defined extended resource types; and resources managed by third-party controllers. Operation type includes GET (single resource retrieval), LIST (list query), CREATE (create), UPDATE (update), and DELETE (delete). Resource identifier includes resource name and resource namespace. Query parameters include tag selectors and field selectors. The target cluster refers to the cluster the request is targeting.

[0030] For example, a resource processing request is: GET / api / v1 / namespaces / default / pods / nginx-pod?cluster=prod,staging. Parsing it reveals that the resource type is Pod, the operation type is GET (single resource retrieval), the resource name in the resource identifier is nginx-pod, the resource namespace is default, and the target cluster is prod,staging.

[0031] Step S102: Determine the request forwarding path based on the resource type and operation type. After determining at least one target cluster to be forwarded based on the resource processing request, the interface aggregation service can be used to perform the specific forwarding of the request.

[0032] According to one embodiment of the present invention, determining a request forwarding path based on resource type and operation type may specifically include: determining a plugin configuration unit based on resource type and operation type, and determining the request forwarding path corresponding to the resource type through the plugin configuration unit. In specific implementation, the plugin configuration unit may be, for example, a developed plugin or a configuration file, as long as it allows for flexible expansion of the aggregation service function corresponding to the resource type. The plugin or configuration file implements the association between resource type and request forwarding path.

[0033] In embodiments of the present invention, to flexibly perform resource aggregation processing, specific plugins or configuration files can be generated based on resource type and operation type. When processing resource processing requests, a specific plugin configuration unit can be determined based on the resource type and operation type, and the resource aggregation logic corresponding to that plugin configuration unit can then be used for further resource aggregation processing.

[0034] In practical implementation, the resource aggregation logic of this plugin configuration unit includes determining the target cluster to which the resource processing request should be forwarded for request forwarding, obtaining the processing results of each target cluster, and performing result aggregation processing.

[0035] In one embodiment, the plugin configuration unit is, for example, a plugin. When determining a plugin, a precise match is first performed based on the resource type and operation type to attempt to find a plugin that matches both the resource type and operation type. If no precise match is found, a general plugin for processing that resource type is searched. If no matching plugin is found, the default plugin is used. Based on the functions implemented by the plugins, the plugins supported by the plugin system can be categorized as follows: resource processing plugins (plugins for processing operations of specific resource types), operation processing plugins (plugins for processing specific operation types), conversion plugins (plugins for performing data format conversion), aggregation plugins (plugins for implementing aggregation logic for multi-cluster results), and filtering plugins (plugins for users to filter resources based on conditions).

[0036] According to one embodiment of the present invention, in response to the addition of a new resource type, a plugin configuration unit for the new resource type is generated and registered so that the plugin configuration unit for the new resource type is loaded when the system starts. Through a plugin mechanism or configuration file, new resource types can be quickly supported without modifying the core code. The system can flexibly support resource aggregation operations for various resource types while maintaining code maintainability and scalability.

[0037] Step S103: Based on the request forwarding path, distribute the resource processing request to at least one target cluster for processing, and obtain the processing results of each target cluster. After obtaining the request forwarding path, the resource processing request can be distributed to at least one target cluster for processing.

[0038] In one embodiment of the present invention, before allocating a resource processing request to at least one target cluster for processing according to a request forwarding path, the method further includes: determining request encapsulation logic based on the resource type and operation type; and encapsulating the resource processing request according to the request encapsulation logic to obtain a standard request. Furthermore, allocating the resource processing request to at least one target cluster for processing according to the request forwarding path specifically includes: allocating the standard request to at least one target cluster for processing according to the request forwarding path. Before request allocation, the resource processing request can also be encapsulated into a standard request that can be recognized and processed by the internal system to achieve standardized processing.

[0039] According to one embodiment of the present invention, a standard request is obtained by encapsulating a resource processing request according to request encapsulation logic. Specifically, this may include: converting the resource processing request into a standard internal operation object according to the request encapsulation logic, and converting the general operations in the internal operation object into specific operations of the target cluster to obtain the standard request. Furthermore, when generating the standard request, the internal operation object may also be validated to check whether the operations in the internal operation object are valid and whether the parameters are complete, etc.

[0040] According to one embodiment of the present invention, allocating resource processing requests to at least one target cluster for processing may specifically include: determining a request allocation method based on the health status, load condition, and priority of the resource processing requests of each target cluster; and allocating the resource processing requests to at least one target cluster for processing according to the request allocation method. When allocating requests, a load balancing strategy can be used to determine how to allocate requests to each cluster based on the health status, load condition, and request characteristics of each cluster. For clusters with good health and low current load, more concurrent requests can be allocated; for clusters with good health and moderate current load, a moderate number of concurrent requests can be allocated; for clusters with slow response and potential problems, lower timeout times and retry strategies can be set.

[0041] According to embodiments of the present invention, dividing thread pools by cluster dimension is an efficient resource management strategy in a multi-Kubernetes cluster management system. This method allocates an independent thread pool to each cluster, allowing for fine-grained configuration based on cluster characteristics and avoiding the "one-size-fits-all" problem caused by a single thread pool.

[0042] The thread pool configuration for each cluster can primarily set the following parameters: core thread count, maximum thread count, wait queue capacity, thread lifespan, and handling strategy when the queue is full. Cluster health status can be assessed by considering performance factors including: average response time of the cluster interface server, number of nodes and resource objects in the cluster, and network connection quality. Cluster load includes factors such as current CPU utilization and memory utilization. Additionally, rate limiting can be considered, such as the query limit per second for the cluster interface server, the maximum allowed concurrent connections, error response frequency, and token bucket status (this parameter should be considered in scenarios where the cluster uses the token bucket algorithm for rate limiting).

[0043] By combining cluster health status, load conditions, and rate limiting settings, thread pool configurations can be dynamically adjusted to achieve efficient resource utilization. Simultaneously, the dynamic adjustment mechanism ensures that the thread pool configuration adapts to changes in cluster status, maintaining system stability and high performance.

[0044] Step S104: Aggregate the processing results of each target cluster to obtain the resource processing result. Resource aggregation processing refers to the process of deduplicating, merging, and integrating similar resources obtained from multiple clusters.

[0045] According to one embodiment of the present invention, the processing results of various target clusters are aggregated, which specifically includes: deduplicating the processing results of each target cluster based on resource identifiers, and merging the deduplicated processing results according to merging rules. The merging rules include: in the case of resource conflicts, selecting the resource with the highest resource version number, the resource with the latest timestamp, or the resource with the highest cluster priority. In specific implementation, the processing results of each target cluster are deduplicated based on the resource identifiers (usually a combination of resource namespace and resource name) in the processing results. For resources with the same name, a cluster prefix can be added for differentiation. After resource deduplication, the deduplicated processing results are merged. This can be a simple merge or a smart merge based on special processing according to resource characteristics. When merging, if resource conflicts exist, the main solutions include: version priority (selecting the resource with the higher version number), timestamp priority (selecting the latest resource), and cluster priority selection (selecting resources according to predefined cluster priorities), etc. Thus, the interface achieves the aggregation of resources of the same type returned by multiple clusters to obtain resource processing results.

[0046] According to one embodiment of the present invention, before aggregating the processing results of each target cluster, the method may further include: performing error detection on the processing results of each target cluster, and determining whether a retry is needed; if a retry is needed, retrying the execution and updating the processing results, and determining the execution success rate, the list of successfully executed clusters, the list of failed clusters, and the reasons for failure based on the updated processing results, so as to generate partially successful information. Furthermore, aggregating the processing results of each target cluster to obtain resource processing results may specifically include: aggregating the processing results of successfully executed target clusters to obtain an aggregated result, converting the aggregated result into a format conforming to the cluster interface specification, and adding aggregation-related metadata and partially successful information to obtain the resource processing result.

[0047] In practice, the processing result of a failed target cluster may be empty or an error value. By performing error detection on the processing results of each target cluster, clusters with empty or error results can be identified as failed clusters. For failed target clusters, it can be determined whether to retry the execution. Specifically, the type of execution failure can be determined based on the reasons for the failure, and it can be determined whether the failure is temporary or permanent to decide whether to retry. If a retry is required, the execution is retried a certain number of times. If the execution is successful before the set retry limit is reached, the processing result is updated. If the execution is still unsuccessful after the set retry limit is reached, the reason for the failure is recorded, and it can also be determined whether to degrade the processing, retrieving pre-stored processing results to update the processing result of the corresponding target cluster. If a retry is not required, degrade the processing directly, retrieving pre-stored processing results to update the processing result of the corresponding target cluster to ensure system availability.

[0048] In other embodiments, after resource aggregation, the aggregated resource processing results can be converted into a response format that conforms to the interface specification, and aggregation-related metadata and partial success information can be added to obtain the resource processing results.

[0049] Upon receiving processing results or error messages from various target clusters, the results or messages are stored in a shared data structure. A lock is used to protect this shared data structure, preventing data races and ensuring thread-safe access to shared resources in concurrent scenarios. Furthermore, atomic classes are used to handle calculators and statistics, and tools are used to coordinate multi-threaded operations. These safety mechanisms ensure the consistency and integrity of the shared data structure when processing multiple cluster requests in parallel, avoiding data races and inconsistencies, thus guaranteeing the system's reliability and correctness.

[0050] Figure 2 This is a schematic diagram of the aggregation process of the processing results according to an embodiment of the present invention. Figure 2 As shown, in one embodiment of the present invention, after receiving the processing results from each cluster, error detection is performed on the processing results of each cluster, and it is determined whether any clusters have failed to execute. If not, it means that all clusters have executed successfully, and the system returns a result indicating that all clusters have executed successfully, confirming the successful processing results, and then aggregating the successful processing results to generate an aggregated result.

[0051] If any clusters fail to execute, analyze the reasons for these failures to determine the type of failure. Determine if the failure is temporary; if not, it's a permanent failure, and error information is recorded. If it's temporary, retry the execution and check for success. If successful, update the cluster's processing result; otherwise, record error information. Then, construct partially successful information based on the error information. Finally, generate an aggregated result based on the updated successful processing result and the partially successful information.

[0052] Figure 3 This is a schematic diagram of a multi-cluster resource processing flow according to an embodiment of the present invention. Figure 3 As shown, in one embodiment of the present invention, after receiving a resource processing request from a client, the target clusters are first determined through an interface router to route the resource processing request to these target clusters for processing. After determining the target clusters, a suitable plugin is determined based on the resource type and operation type of the resource processing request to perform specific request forwarding and result aggregation. Specifically, after determining a suitable plugin, request encapsulation logic is determined through the plugin to encapsulate the resource processing request into a standard request; then, the request forwarding path is determined based on the request type, and the resource processing request is forwarded to each target cluster based on a load balancing strategy; finally, the processing results of each target cluster are collected.

[0053] Then, error detection is performed on the processing results of each target cluster, and it is determined whether to retry. If it is determined that a retry should be performed, the processing results are retried and updated, and partially successful information is generated. If it is determined that a retry should not be performed, degradation processing is performed and the processing results are updated, and partially successful information is generated.

[0054] Finally, based on the updated execution results and partial success information, an aggregated result is generated, and the aggregated result is formatted and returned to the client.

[0055] Figure 4 This is a schematic diagram of the main modules of a multi-cluster resource processing apparatus according to an embodiment of the present invention. Figure 4 As shown, the multi-cluster resource processing device 400 of this embodiment mainly includes a target cluster determination module 401, a forwarding path determination module 402, a request allocation processing module 403, and a processing result aggregation module 404.

[0056] The target cluster determination module 401 is used to respond to a resource processing request sent by the client and determine at least one target cluster based on the resource processing request. The resource processing request includes resource type and operation type. Forwarding path determination module 402 is used to determine the request forwarding path based on resource type and operation type; The request allocation processing module 403 is used to allocate resource processing requests to at least one target cluster for processing according to the request forwarding path, and obtain the processing results of each target cluster. The processing result aggregation module 404 is used to aggregate the processing results of each target cluster to obtain the resource processing results.

[0057] According to one embodiment of the present invention, the resource processing request further includes a cluster specified by the client and a resource identifier; the target cluster determination module 401 may be specifically used to: determine a target cluster from the configured clusters based on the resource type included in the resource processing request and the resource types that the configured clusters can provide; or, determine a target cluster based on the cluster specified by the client included in the resource processing request; or, determine a target cluster from the configured clusters based on the resource identifier included in the resource processing request and the resource types that the configured clusters can provide.

[0058] According to one embodiment of the present invention, the multi-cluster resource processing apparatus 400 further includes a request encapsulation module (not shown in the figure), configured to: determine request encapsulation logic based on resource type and operation type before allocating a resource processing request to at least one target cluster for processing according to a request forwarding path; and encapsulate the resource processing request according to the request encapsulation logic to obtain a standard request. Furthermore, the request allocation processing module 403 can also be configured to: allocate the standard request to at least one target cluster for processing according to a request forwarding path.

[0059] According to one embodiment of the present invention, the request encapsulation module (not shown in the figure) can be specifically used to: convert resource processing requests into standard internal operation objects according to the request encapsulation logic, and convert the general operations in the internal operation objects into specific operations of the target cluster to obtain standard requests.

[0060] According to an embodiment of the present invention, the forwarding path determination module 402 can also be used to: determine the plugin configuration unit according to the resource type and the operation type, and determine the request forwarding path corresponding to the resource type through the plugin configuration unit.

[0061] According to one embodiment of the present invention, the multi-cluster resource processing apparatus 400 further includes a resource type addition module (not shown in the figure), which is used to: generate and register a new resource type plugin configuration unit in response to the addition of a new resource type, so as to load the new resource type plugin configuration unit when the system starts.

[0062] According to one embodiment of the present invention, the target cluster determination module 401 can be specifically used to: determine the request allocation method based on the health status, load status and priority of resource processing requests of each target cluster; and allocate the resource processing requests to at least one target cluster for processing according to the request allocation method.

[0063] According to one embodiment of the present invention, the multi-cluster resource processing apparatus 400 further includes a result detection module (not shown in the figure), used for: performing error detection on the processing results of each target cluster before aggregating the processing results of each target cluster, and determining whether a retry is needed; if a retry is determined, retrying the execution and updating the processing results, and determining the execution success rate, the list of successfully executed clusters, the list of failed clusters, and the reasons for failure based on the updated processing results, so as to generate partially successful information. The processing result aggregation module 404 can specifically be used for: aggregating the processing results of successfully executed target clusters to obtain an aggregation result, converting the aggregation result into a format conforming to the cluster interface specification, and adding aggregation-related metadata and partially successful information to obtain the resource processing result.

[0064] According to an embodiment of the present invention, the processing result aggregation module 404 can be specifically used to: deduplicate the processing results of each target cluster according to the resource identifier, and merge the deduplicated processing results according to the merging rules, the merging rules including: in the case of resource conflict, select the resource with the highest resource version number or the resource with the latest timestamp or the resource with the highest cluster priority.

[0065] According to the technical solution of this invention, in response to a resource processing request sent by a client, at least one target cluster is determined based on the resource processing request, which includes a resource type and an operation type; a request forwarding path is determined based on the resource type and operation type; the resource processing request is allocated to at least one target cluster for processing according to the request forwarding path, and the processing results of each target cluster are obtained; the processing results of each target cluster are aggregated to obtain the resource processing result. This technical solution, by determining the request forwarding path based on the resource type and operation type to allocate the request to multiple clusters for processing and aggregating the processing results of multiple clusters, can achieve resource aggregation processing across multiple clusters, whether in small-scale and simple multi-cluster scenarios or complex multi-cluster scenarios. It has strong scenario adaptability and is simple to implement; at the same time, different resource aggregation processing is performed on different resource types and operation types, which improves the flexibility of resource aggregation processing.

[0066] Figure 5 An exemplary system architecture 500 is shown, in which the method or apparatus for multi-cluster resource processing of embodiments of the present invention can be applied.

[0067] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, and 503, a network 504, and a server 505. Network 504 serves as the medium for providing communication links between terminal devices 501, 502, and 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0068] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0069] Terminal devices 501, 502, and 503 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0070] Server 505 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 501, 502, and 503 (for example only). The backend management server can process received multi-cluster resource processing requests and other data by forwarding requests, aggregating results, and feeding back the processing results (such as the aggregated resource processing results - for example only) to the terminal devices.

[0071] It should be noted that the multi-cluster resource processing method provided in this embodiment of the invention is generally executed by server 505, and correspondingly, the multi-cluster resource processing device is generally located in server 505.

[0072] It should be understood that Figure 5 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0073] The following is for reference. Figure 6 It shows a schematic diagram of the structure of a computer system 600 suitable for implementing terminal devices or servers of the present invention. Figure 6 The terminal device or server shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0074] like Figure 6As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the system 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0075] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0076] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this invention.

[0077] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0078] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0079] The units or modules described in the embodiments of the present invention can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor can be described as including a target cluster determination module, a forwarding path determination module, a request allocation processing module, and a processing result aggregation module. The names of these units or modules do not necessarily limit the specific unit or module itself; for example, the processing result aggregation module can also be described as "a module for aggregating the processing results of various target clusters to obtain resource processing results."

[0080] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: responding to a resource processing request from a client; determining at least one target cluster based on the resource processing request, the resource processing request including a resource type and an operation type; determining a request forwarding path based on the resource type and the operation type; allocating the resource processing request to the at least one target cluster for processing according to the request forwarding path, obtaining processing results for each target cluster; and aggregating the processing results of each target cluster to obtain a resource processing result.

[0081] According to the technical solution of this invention, in response to a resource processing request sent by a client, at least one target cluster is determined based on the resource processing request, which includes a resource type and an operation type; a request forwarding path is determined based on the resource type and operation type; the resource processing request is allocated to at least one target cluster for processing according to the request forwarding path, and the processing results of each target cluster are obtained; the processing results of each target cluster are aggregated to obtain the resource processing result. This technical solution, by determining the request forwarding path based on the resource type and operation type to allocate the request to multiple clusters for processing and aggregating the processing results of multiple clusters, can achieve resource aggregation processing across multiple clusters, whether in small-scale and simple multi-cluster scenarios or complex multi-cluster scenarios. It has strong scenario adaptability and is simple to implement; at the same time, different resource aggregation processing is performed on different resource types and operation types, which improves the flexibility of resource aggregation processing.

[0082] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for processing multi-cluster resources, characterized in that, include: In response to a resource processing request from a client, at least one target cluster is determined based on the resource processing request, wherein the resource processing request includes a resource type and an operation type; Determine the request forwarding path based on the resource type and the operation type; Based on the request forwarding path, the resource processing request is allocated to the at least one target cluster for processing, and the processing results of each target cluster are obtained. The processing results of each target cluster are aggregated to obtain the resource processing result.

2. The method according to claim 1, characterized in that, The resource processing request also includes the cluster and resource identifier specified by the client; Determining at least one target cluster based on the resource processing request includes: Based on the resource types included in the resource processing request and the resource types that each configured cluster can provide, the target cluster is determined from the configured clusters. Alternatively, the target cluster may be determined based on the cluster specified by the client included in the resource processing request; Alternatively, the target cluster can be determined from the configured clusters based on the resource identifier included in the resource processing request and the resource types that can be provided by each configured cluster.

3. The method according to claim 1, characterized in that, Before allocating the resource processing request to the at least one target cluster for processing according to the request forwarding path, the method further includes: Determine the request encapsulation logic based on the resource type and the operation type; The resource processing request is encapsulated according to the request encapsulation logic to obtain a standard request; Based on the request forwarding path, the resource processing request is allocated to the at least one target cluster for processing, including: Based on the request forwarding path, the standard request is assigned to the at least one target cluster for processing.

4. The method according to claim 3, characterized in that, The resource processing request is encapsulated according to the request encapsulation logic to obtain a standard request, including: According to the request encapsulation logic, the resource processing request is converted into a standard internal operation object, and the general operations in the internal operation object are converted into specific operations of the target cluster to obtain a standard request.

5. The method according to claim 1, characterized in that, Determining the request forwarding path based on the resource type and the operation type includes: The plugin configuration unit is determined based on the resource type and operation type, and the request forwarding path corresponding to the resource type is determined through the plugin configuration unit.

6. The method according to claim 5, characterized in that, The method further includes: In response to the addition of a new resource type, a plugin configuration unit for the new resource type is generated and registered so that the plugin configuration unit for the new resource type is loaded when the system starts.

7. The method according to claim 1, characterized in that, Assigning the resource processing request to the at least one target cluster for processing includes: The request allocation method is determined based on the health status, load of each target cluster and the priority of the resource processing requests. The resource processing request is allocated to the at least one target cluster for processing according to the request allocation method.

8. The method according to claim 1, characterized in that, Before aggregating the processing results of the various target clusters, the following steps are also included: Error detection is performed on the processing results of each target cluster, and it is determined whether a retry is needed. If it is determined that a retry is necessary, the execution is retried and the processing result is updated. Based on the updated processing result, the execution success rate, the list of successfully executed clusters, the list of failed clusters, and the reasons for failure are determined to generate partially successful information. The processing results of each target cluster are aggregated to obtain resource processing results, including: The processing results of the successfully executed target cluster are aggregated to obtain an aggregated result. The aggregated result is converted into a format that conforms to the cluster interface specification, and aggregate-related metadata and the partially successful information are added to obtain the resource processing result.

9. The method according to claim 1, characterized in that, Aggregating the processing results of each target cluster includes: The processing results of each target cluster are deduplicated according to the resource identifier, and the deduplicated processing results are merged according to the merging rules. The merging rules include: in the case of resource conflicts, selecting the resource with the highest resource version number, the resource with the latest timestamp, or the resource with the highest cluster priority.

10. A device for multi-cluster resource processing, characterized in that, include: The target cluster determination module is used to determine at least one target cluster in response to a resource processing request sent by a client, wherein the resource processing request includes a resource type and an operation type. A forwarding path determination module is used to determine a request forwarding path based on the resource type and the operation type; The request allocation processing module is used to allocate the resource processing request to the at least one target cluster for processing according to the request forwarding path, and obtain the processing results of each target cluster. The processing result aggregation module is used to aggregate the processing results of each target cluster to obtain the resource processing results.

11. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-9.

12. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

Images