A service request processing method, device, system, apparatus and storage medium
Patent Information
- Application Number
- CN202311773836.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-21
- Publication Date
- 2026-10-23
- Estimated Expiration
- 2043-12-21
AI Technical Summary
[0005]现有的仅是实现集群内部的服务访问,无法实现跨集群的服务访问,也就是一个容器编排集群中的部署单元无法访问另一个容器编排集群中部署单元的服务,增加了服务访问的局限性
[0026]通过利用联邦管理集群管理每个工作集群,并确定每个联邦服务对应的联邦网关地址和联邦端点集合,使得联邦域名集群可以基于每个联邦服务对应的联邦网关地址和联邦端点集合,对当前工作集群中的第一部署单元待访问的目标联邦服务对应的目标域名解析请求进行处理,确定并返回目标联邦服务对应的目标IP地址。当前工作集群基于联邦域名集群返回的目标IP地址、通过联邦管理集群确定的每个联邦服务对应的联邦网关地址和联邦端点集合,确定目标工作集群中用于实现目标联邦服务的第二部署单元的IP地址,并基于第二部署单元的IP地址,将目标联邦服务请求发送至所述第二部署单元中,使得第二部署单元对目标联邦服务请求进行响应处理。由于目标联邦集群可以不同于当前联邦集群,从而实现了跨集群的服务访问,进而降低了服务访问的局限性,满足了个性化业务需求。
Smart Images

Figure CN117880296B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to computer technology, and more particularly to a service request processing method, apparatus, system, device, and storage medium. Background Technology
[0002] With the rapid development of computer technology and cloud-native technology, container orchestration clusters can be used to efficiently orchestrate and manage containerized applications throughout their entire lifecycle on cloud platforms. For example, Kubernetes clusters can be used to manage deployment units on multiple deployment nodes in a cloud platform, making container deployment simpler and more efficient.
[0003] Currently, service access is typically implemented within each container orchestration cluster. In other words, deployment units within the same deployment orchestration cluster can only access other deployment units within that cluster, and other deployment units can only handle service requests generated within that cluster.
[0004] However, in the process of realizing this invention, the inventors discovered at least the following problems in the prior art:
[0005] The existing technology only enables service access within the cluster and cannot enable cross-cluster service access. In other words, a deployment unit in one container orchestration cluster cannot access the services of a deployment unit in another container orchestration cluster, which increases the limitation of service access. Summary of the Invention
[0006] This invention provides a service request processing method, apparatus, system, device, and storage medium to enable cross-cluster service access, thereby reducing the limitations of service access and meeting personalized business needs.
[0007] In a first aspect, embodiments of the present invention provide a service request processing method, applied to each working cluster managed by a federated management cluster, comprising:
[0008] Generate a target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current work cluster;
[0009] The target domain name resolution request is sent to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines and returns the target IP address corresponding to the target federated service, wherein the federated gateway address refers to the cluster IP address allocated to the federated service, and the federated endpoint set includes the deployment unit IP address in each working cluster used to implement the federated service.
[0010] Based on the target IP address, the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster, the IP address of the second deployment unit in the target working cluster used to implement the target federal service is determined;
[0011] Based on the IP address of the second deployment unit, the target federated service request is sent to the second deployment unit so that the second deployment unit can respond to the target federated service request.
[0012] Secondly, embodiments of the present invention also provide a service request processing apparatus, integrated into each work cluster managed by a federated management cluster, comprising:
[0013] The domain name resolution request generation module is used to generate the target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster.
[0014] The domain name resolution request sending module is used to send the target domain name resolution request to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines and returns the target IP address corresponding to the target federated service, wherein the federated gateway address refers to the cluster IP address allocated to the federated service, and the federated endpoint set includes the deployment unit IP address in each working cluster used to implement the federated service;
[0015] The deployment unit IP address determination module is used to determine the IP address of the second deployment unit in the target working cluster that implements the target federal service based on the target IP address, the federal gateway address and the set of federal endpoints corresponding to each federal service determined by the federal management cluster;
[0016] The federated service request sending module is used to send a target federated service request to the second deployment unit based on the IP address of the second deployment unit, so that the second deployment unit can respond to the target federated service request.
[0017] Thirdly, embodiments of the present invention also provide a service request processing system, the system comprising: a federated management cluster, multiple working clusters, and a federated domain name cluster; wherein,
[0018] The federated management cluster is used to manage each working cluster and to determine the federated gateway address and federated endpoint set corresponding to each federated service;
[0019] Each working cluster is used to implement the service request processing method as provided in any embodiment of the present invention.
[0020] Fourthly, embodiments of the present invention also provide an electronic device, the electronic device comprising:
[0021] One or more processors;
[0022] Memory, used to store one or more programs;
[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the service request processing method provided in any embodiment of the present invention.
[0024] Fifthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the service request processing method as provided in any embodiment of the present invention.
[0025] One embodiment of the above invention has the following advantages or beneficial effects:
[0026] By utilizing a federated management cluster to manage each working cluster and determining the federated gateway address and federated endpoint set corresponding to each federated service, the federated domain name cluster can process the target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster, based on the federated gateway address and federated endpoint set corresponding to each federated service, and determine and return the target IP address corresponding to the target federated service. The current working cluster, based on the target IP address returned by the federated domain name cluster and the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines the IP address of the second deployment unit in the target working cluster used to implement the target federated service, and sends the target federated service request to the second deployment unit based on the second deployment unit's IP address, enabling the second deployment unit to respond to the target federated service request. Since the target federated cluster can be different from the current federated cluster, cross-cluster service access is achieved, thereby reducing the limitations of service access and meeting personalized business needs. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a flowchart of a service request processing method provided in one embodiment of the present invention;
[0029] Figure 2This is an example diagram illustrating a service request processing procedure according to an embodiment of the present invention;
[0030] Figure 3 This is a flowchart of another service request processing method provided in one embodiment of the present invention;
[0031] Figure 4 This is an example diagram illustrating a process for determining a federated gateway address and a federated endpoint set according to an embodiment of the present invention;
[0032] Figure 5 This is a schematic diagram of the structure of a service request processing device provided in one embodiment of the present invention;
[0033] Figure 6 This is a schematic diagram of the structure of a service request processing system provided in one embodiment of the present invention;
[0034] Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation
[0035] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0036] Figure 1 This is a flowchart illustrating a service request processing method according to an embodiment of the present invention. This embodiment is applicable to cross-cluster access between multiple work clusters managed by a federated management cluster. The method can be executed by a service request processing device, which can be implemented in software and / or hardware and integrated into each work cluster managed by the federated management cluster.
[0037] The federated management cluster serves as the primary federated cluster, providing unified management of multiple clusters across clouds, unified application deployment, and cross-cluster elastic scaling capabilities. Worker clusters act as sub-clusters, executing the workloads of the deployed units. The federated management cluster abstracts another layer of cluster concept on top of multiple worker clusters. It can uniformly manage multiple worker clusters, enabling cross-cluster service access. These worker clusters can be geographically dispersed, cross-cloud vendor-specific, or internally built clusters. Both the federated management cluster and each worker cluster can refer to container orchestration clusters, such as Kubernetes clusters, thus leveraging native service resources and workloads to achieve cross-cluster access capabilities within the federated cluster, maximizing compatibility with single-cluster usage habits.
[0038] like Figure 1 As shown, the method specifically includes the following steps:
[0039] S110. Generate a target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster.
[0040] The current work cluster can be any work cluster that currently requires service access. Each work cluster includes multiple deployment nodes. Each deployment node is the device that actually deploys the containerized application. One or more deployment units (Pods) can be created on each deployment node. A deployment unit (Pod) is the smallest deployment unit created and managed within a deployment node. Each deployment unit can run one container, or multiple containers simultaneously. The first deployment unit refers to the deployment unit in the current work cluster that requires service access. A federated service refers to a business service that can be accessed across clusters. The target federated service refers to the federated service that the first deployment unit currently needs to access. The target domain name resolution request can be a request used to resolve the domain name of the target federated service.
[0041] Specifically, when the first deployment unit in the current work cluster needs to access the target federated service, it can first generate a target domain name resolution request corresponding to the target federated service for service domain name resolution.
[0042] For example, S110 may include: generating a current domain name resolution request corresponding to the service to be accessed by the first deployment unit in the current working cluster; if it is detected that the service identification information corresponding to the service to be accessed is the target federated service identification information, then it is determined that the current domain name resolution request is the target domain name resolution request corresponding to the target federated service.
[0043] The service to be accessed can be a federated service, allowing cross-cluster access. Alternatively, it can be a non-federated service, allowing only intra-cluster access. The target federated service identification information can be used to characterize it as a federated service.
[0044] Specifically, see Figure 2 In the preceding worker cluster (e.g., worker cluster 1), the first deployment unit can access both federated and non-federated services. Therefore, after generating a current domain name resolution request for the service to be accessed, the first deployment unit can send this request to the domain name resolution module for service domain name resolution. If the domain name resolution module detects that the service identifier information corresponding to the service to be accessed matches the identifier information of the target federated service, it determines that the service to be accessed is the target federated service, and the corresponding current domain name resolution request becomes the target domain name resolution request for the target federated service.
[0045] It should be noted that the domain name resolution module within the working cluster can only resolve service domain names accessed within the cluster, and cannot resolve domain names of federated services. If the domain name resolution module detects that the service identifier information corresponding to the service to be accessed is not federated service identifier information, it can directly perform domain name resolution for the current domain name resolution request and allow normal service access within the cluster.
[0046] S120. Send the target domain name resolution request to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, and determines and returns the target IP address corresponding to the target federated service.
[0047] The FederalGateway is the cluster IP address assigned to the federated service. The FederalEndpoint set includes the IP addresses of the deployment units used to implement the federated service in each worker cluster. Each federated service corresponds to one FederalGateway and one FederalEndpoint set. The FederalGateway is a predefined data structure that can contain the cluster IP address, port, and the name of the corresponding FederalEndpoint set. The FederalEndpoint set is also a predefined data structure that includes the IP addresses and port information of all deployment unit Pods used to implement the federated service in all worker clusters. The data in the FederalGateway and FederalEndpoint sets are global data corresponding to all worker clusters to support cross-cluster service access. The Federal Domain Name Cluster can be the cluster used to resolve the domain name of the federated service. The Target IP Address is the IP address obtained after the Federal Domain Name Cluster resolves the domain name of the target federated service. The Target IP Address is the destination address of the request. The Target IP Address can be a specific cluster IP address or a specific deployment unit IP address.
[0048] Specifically, the federated management cluster can determine the federated gateway address and set of federated endpoints for each federated service. The domain name controller in the federated domain name cluster can obtain the federated gateway address and set of federated endpoints for each federated service by subscribing to them from the federated management cluster, and generate corresponding federated domain name records based on these records. Figure 2As shown. After the domain name resolution module in the current working cluster determines that the service to be accessed is the target federated service, it can forward the target domain name resolution request corresponding to the target federated service to the federated domain name cluster for domain name resolution of the federated service. The federated domain name cluster can resolve the federated service domain name in the target domain name resolution request to the corresponding target IP address based on the federated gateway address and federated endpoint set corresponding to each federated service. For example, it can resolve the federated service domain name in the target domain name resolution request to the corresponding target IP address based on the federated domain name record, and send the target IP address to the current working cluster.
[0049] For example, the step S120, "processing the target domain name resolution request based on the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster, and determining and returning the target IP address corresponding to the target federal service," may include: resolving the target domain name resolution request to obtain the target service type corresponding to the target federal service; performing domain name resolution based on the target service type and the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster, and determining the target IP address corresponding to the target federal service after resolution.
[0050] The target service type can refer to the service type to which the target federated service belongs. For example, the target service type can be either ClusterIP or Headless. A ClusterIP service type federated service will be assigned a cluster IP address. A Headless service type federated service is a headless service that neither assigns a cluster IP address nor exposes any ports.
[0051] Specifically, the federal domain name cluster can generate domain name resolution records for each service type based on the federal gateway address and set of federal endpoints corresponding to each federal service determined by the federal management cluster, such as... Figure 2 As shown, after the federated domain name cluster resolves the received target domain name resolution request, it obtains the target service type corresponding to the target federated service. Based on the target domain name resolution record corresponding to the target service type, it resolves the domain name of the target federated service to obtain the target IP address corresponding to the target federated service after resolution.
[0052] For example, determining the target IP address corresponding to the target federal service after domain name resolution based on the target service type and the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster may include: if the target service type is a ClusterIP service type, then the target federal gateway address corresponding to the target federal service is determined based on the federal gateway address corresponding to each federal service determined by the federal management cluster, and the target cluster IP address corresponding to the target federal gateway address is determined as the target IP address; if the target service type is a Headless service type, then the target federal endpoint set corresponding to the target federal service is determined based on the federal endpoint set corresponding to each federal service determined by the federal management cluster, and all deployment unit IP addresses in the target federal endpoint set are determined as the target IP address.
[0053] Specifically, such as Figure 2 As shown, for ClusterIP service type federated services, the server domain name can be resolved to the cluster IP address corresponding to each federated service subscribed from the federated management cluster, based on the federated gateway address corresponding to each federated service. For Headless service type federated services, the server domain name can be resolved to the IP addresses of all deployment units in all worker clusters used to implement the federated service, based on the federated endpoint set corresponding to each federated service subscribed from the federated management cluster. When the target service type is ClusterIP service type, the target cluster IP address of the target federated service corresponding to the target federated gateway address can be determined as the target IP address based on the domain name resolution record corresponding to the ClusterIP service type. The target federated gateway address is the same as the target cluster IP address assigned to the target federated service. When the target service type is Headless service type, the IP addresses of all deployment units in the target federated endpoint set corresponding to the target federated service can be determined as the target IP address based on the domain name resolution record corresponding to the Headless service type.
[0054] S130. Based on the target IP address, the set of federal gateway addresses and federal endpoints corresponding to each federal service determined by the federal management cluster, determine the IP address of the second deployment unit in the target working cluster used to implement the target federal service.
[0055] The target work cluster can be the work cluster ultimately accessed by the current work cluster. The second deployment unit refers to one deployment unit selected from all deployment units within the target work cluster used to implement the target federated service. The second deployment unit is the deployment unit ultimately used to process the target service request. The target work cluster can be different from the current work cluster, thus enabling cross-cluster service access. The target work cluster can also be the same as the current work cluster. For example, if the current work cluster also contains deployment units used to implement the target federated service, then the current work cluster can be used as the target work cluster, and one deployment unit within the current work cluster used to implement the target federated service can be used as the second deployment unit, thus enabling intra-cluster service access.
[0056] Specifically, the current working cluster can pre-subscribe to the federal gateway address and federal endpoint set corresponding to each federal service from the federal management cluster, enabling the current working cluster to obtain the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster. Based on the target IP address returned by the federal domain name cluster, and the federal gateway address and federal endpoint set corresponding to each federal service obtained from the federal management cluster, load balancing is performed to determine the IP address of the second deployment unit in the final target working cluster used to implement the target federal service.
[0057] For example, S130 may include: if the target IP address is detected as the target cluster IP address based on the federal gateway address corresponding to each federal service determined by the federal management cluster, then load balancing is performed based on the target cluster IP address and the set of federal endpoints corresponding to each federal service determined by the federal management cluster to determine the IP address of the second deployment unit in the target working cluster used to implement the target federal service; if the target IP address is detected as multiple deployment unit IP addresses, then the IP address of the second deployment unit in the target working cluster used to implement the target federal service is determined from the multiple deployment unit IP addresses.
[0058] Specifically, see Figure 2The load balancing module in the current working cluster (e.g., working cluster 1) can pre-subscribe to the federated gateway address and federated endpoint set corresponding to each federated service from the federated management cluster, thus obtaining the federated gateway address and federated endpoint set corresponding to each federated service. Based on the pre-subscribed federated gateway address corresponding to each federated service, the load balancing module can detect whether the target IP address returned by the federated domain name cluster is the target cluster IP address. If so, based on the pre-subscribed federated endpoint set corresponding to each federated service, it determines the target federated endpoint set corresponding to the target cluster IP address. Then, based on the load balancing strategy, it selects a suitable deployment unit IP address from all deployment unit IP addresses corresponding to all working clusters included in the target federated endpoint set as the IP address of the second deployment unit. The working cluster where the second deployment unit is located is taken as the target working cluster, thereby ensuring global workload balancing and improving the processing speed of service requests. The load balancing module can detect whether the target IP address returned by the federated domain name cluster is one of multiple deployment unit IP addresses. If so, it can select a suitable deployment unit IP address from the multiple deployment unit IP addresses as the IP address of the second deployment unit based on the load balancing strategy. The working cluster where the second deployment unit is located is used as the target working cluster, thereby ensuring global workload balancing and improving the processing speed of service requests.
[0059] S140. Based on the IP address of the second deployment unit, the target federated service request is sent to the second deployment unit so that the second deployment unit can respond to the target federated service request.
[0060] Specifically, the first deployment unit in the current working cluster sends a target federated service request with the target IP address as the destination address. The load balancing module in the current working cluster can intercept this target federated service request and convert the target IP address in the target federated service request to the IP address of the second deployment unit. The module then sends the target federated service request with the IP address of the second deployment unit as the destination address, and forwards it to the second deployment unit in the target working cluster. The second deployment unit then processes the response to the target federated service request, thereby enabling service access between multiple working clusters and achieving global load balancing.
[0061] The technical solution of this embodiment utilizes a federated management cluster to manage each working cluster and determines the federated gateway address and federated endpoint set corresponding to each federated service. This allows the federated domain name cluster to process the target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster, based on the federated gateway address and federated endpoint set corresponding to each federated service, and to determine and return the target IP address corresponding to the target federated service. The current working cluster, based on the target IP address returned by the federated domain name cluster and the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines the IP address of the second deployment unit in the target working cluster used to implement the target federated service. Based on the IP address of the second deployment unit, the target federated service request is sent to the second deployment unit, enabling the second deployment unit to respond to the target federated service request. Since the target federated cluster can be different from the current federated cluster, cross-cluster service access is achieved, thereby reducing the limitations of service access and meeting personalized business needs.
[0062] Figure 3 This is a flowchart illustrating another service request processing method according to an embodiment of the present invention. Based on the above embodiments, this embodiment describes in detail the process of determining the federated gateway address and federated endpoint set corresponding to each federated service. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.
[0063] See Figure 3 Another service request processing method provided in this embodiment specifically includes the following steps:
[0064] S310, the Federation Management Cluster obtains the currently deployed object and the currently service object corresponding to the current federated service to be created.
[0065] Here, "current federated service" can refer to the federated service that needs to be created at present. "Current deployment object (Deployment)" can refer to the deployment declaration information corresponding to the deployment unit used to implement the current federated service. "Current service object (Service)" can refer to the service information of the current federated service to be created.
[0066] Specifically, see Figure 4 Users can create the current deployment object and the current service object corresponding to the current federated service in the federated management cluster through the user terminal (such as a client, web page, etc.), so that the federated management cluster can obtain the current deployment object and the current service object.
[0067] S320. Based on the deployment work cluster information in the current deployment object, send the current deployment object to the deployment work cluster to create a corresponding deployment unit so that the created deployment unit can implement the current federated service.
[0068] The current deployment object can include information about at least one deployment work cluster used to implement the current federated service, as well as information such as the number of deployment units created in each deployment work cluster, affinity, and system architecture (e.g., arm64 or amd64). A deployment work cluster can be the work cluster in which the deployment units used to implement the current federated service reside. A deployment work cluster can be all work clusters, ensuring that all work clusters support access to the current federated service. Alternatively, a deployment work cluster can be a subset of work clusters, ensuring that only a portion of the work clusters support access to the current federated service.
[0069] Specifically, the federated management cluster can send the current deployment object to each configured deployment work cluster based on the deployment work cluster information within the current deployment object. This allows each deployment work cluster to create deployment units to implement the current federated service based on the deployment unit information within the distributed current deployment object. Resource allocation is then performed within the deployment work cluster based on the current deployment object, and deployment units are run on demand. The deployment units of the current deployment object will be distributed and run across multiple deployment work clusters. For example, see... Figure 4 Work cluster 1 is a deployment work cluster. Work cluster 1 can create two deployment units based on the currently deployed objects.
[0070] S330. Assign an IP address to the current service object, determine the current cluster IP address corresponding to the current service object, and add the current cluster IP address, federated service identification information, and deployment unit selector to the current service object.
[0071] The federated service identifier information can be used to identify that the service object was issued by the federated management cluster, so that the working cluster can identify the set of endpoints corresponding to the federated service based on the federated service identifier information. The deployment unit selector PodSelector can be used to filter deployment units that can implement the current federated service.
[0072] Specifically, the federated management cluster can randomly select a cluster IP address from the reserved cluster IP address range as the current cluster IP address corresponding to the current service object. The assigned current cluster IP address, federated service identification information, and deployment unit selector are then added to the current service object.
[0073] S340. The added current service object is sent to each work cluster so that each work cluster can determine the current endpoint set corresponding to the current federated service in each work cluster based on the received current service object. The current endpoint set includes the IP address of the current deployment unit used to implement the current federated service.
[0074] Specifically, see Figure 4 The federated management cluster sends the added current service object to each worker cluster. Each worker cluster, based on the deployment unit selector in the received current service object, determines the IP addresses of all current deployment units in each worker cluster used to implement the current federated service, thereby generating a current endpoint set (EndpointSlice) for the current federated service in each worker cluster. Each worker cluster can then send the generated current endpoint set to the federated management cluster. For example, Figure 4 Work cluster 1 generates the current endpoint set based on the deployment unit selector in the current service object and sends the current endpoint set to the federated management cluster.
[0075] It should be noted that if a certain work cluster fails to select a deployment unit for implementing the current federated service based on the deployment unit selector in the current service object, it indicates that the work cluster is not the deployment work cluster corresponding to the current federated service. In this case, the endpoint set generated by the work cluster is an empty set, and therefore the work cluster does not need to send the endpoint set to the federated management cluster.
[0076] For example, "each working cluster determines the set of current endpoints corresponding to the current federated service in each working cluster based on the received current service object" in S340 may include: each working cluster determines the current deployment unit for implementing the current federated service from the created deployment units based on the deployment unit selector in the received current service object; and determines the set of current endpoints corresponding to the current federated service in each working cluster based on the IP address of the current deployment unit.
[0077] Specifically, each work cluster, based on the deployment unit selector in the received current service object, checks whether each created deployment unit meets the selection condition corresponding to the deployment unit selector. If the selection condition is met, it indicates that the deployment unit is the current deployment unit used to implement the current federated service. Thus, the deployment unit selector can be used to determine whether there is a current deployment unit in each work cluster used to implement the current federated service, and the IP address of the selected current deployment unit is stored in the current endpoint set, thereby generating the current endpoint set corresponding to the current federated service in each work cluster.
[0078] S350. Based on the current endpoint set sent by each working cluster, generate the current federated endpoint set corresponding to the current federated service, convert the current service object into a federated gateway, and obtain the current federated gateway address corresponding to the current federated service.
[0079] Specifically, see Figure 4The federated management cluster can obtain the current endpoint set (EndpointSlice) corresponding to the current federated service sent by each working cluster. The federated management cluster merges all obtained EndpointSlices to obtain the current federated endpoint set (FederalEndpoint) corresponding to the current federated service, thus obtaining global information about all working clusters. Since the cluster IP address in each working cluster changes, it is necessary to convert the current service object corresponding to the current federated service into a federated gateway (FederalGateway) to obtain the current federated gateway address. The federated gateway address is the current cluster IP address assigned to the current service object.
[0080] By using the method described in steps S310-S350 above for determining the current federated gateway address and the current federated endpoint set corresponding to the current federated service, the federated gateway address and federated endpoint set corresponding to each federated service can be determined.
[0081] It should be noted that each working cluster and the federated domain name cluster can subscribe to the federated gateway address and federated endpoint set corresponding to each federated service in the federated management cluster, so that each working cluster and the federated domain name cluster can obtain global information of all working clusters, thereby enabling cross-cluster service access.
[0082] S360: Generate a target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster.
[0083] S370. Send the target domain name resolution request to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service subscribed from the federated management cluster, determines and returns the target IP address corresponding to the target federated service.
[0084] S380. Based on the target IP address, the federal gateway address and the set of federal endpoints corresponding to each federal service subscribed from the federal management cluster, determine the IP address of the second deployment unit in the target working cluster used to implement the target federal service.
[0085] S390. Based on the IP address of the second deployment unit, the target federated service request is sent to the second deployment unit so that the second deployment unit can respond to the target federated service request.
[0086] In this embodiment, the federated management cluster obtains the current deployment object and current service object corresponding to the current federated service to be created. Based on the deployment work cluster information in the current deployment object, it sends the current deployment object to the deployment work cluster to create a corresponding deployment unit, so that the created deployment unit implements the current federated service. An IP address is assigned to the current service object, the current cluster IP address corresponding to the current service object is determined, and the current cluster IP address, federated service identifier information, and deployment unit selector are added to the current service object. The added current service object is sent to each work cluster, so that each work cluster determines the current endpoint set corresponding to the current federated service in each work cluster based on the received current service object. Based on the current endpoint set sent by each work cluster, the federated management cluster generates the current federated endpoint set corresponding to the current federated service, converts the current service object into a federated gateway, and obtains the current federated gateway address corresponding to the current federated service. Thus, the federated management cluster can determine the federated gateway address and federated endpoint set corresponding to each federated service, realizing cross-cluster service discovery. Furthermore, each work cluster and the federated domain name cluster can obtain global information of all work clusters in the federated management cluster, thereby realizing cross-cluster service access.
[0087] The following are embodiments of the service request processing apparatus provided in this invention. This apparatus and the service request processing methods in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the service request processing apparatus, please refer to the embodiments of the above service request processing methods.
[0088] Figure 5 This is a schematic diagram of a service request processing device according to an embodiment of the present invention. This embodiment is applicable to cross-cluster access between multiple work clusters managed by a federated management cluster. Figure 5 As shown, the device specifically includes: a domain name resolution request generation module 510, a domain name resolution request sending module 520, a deployment unit IP address determination module 530, and a federated service request sending module 540.
[0089] The system includes the following modules: a domain name resolution request generation module 510, used to generate a target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster; a domain name resolution request sending module 520, used to send the target domain name resolution request to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines and returns the target IP address corresponding to the target federated service, wherein the federated gateway address refers to the cluster IP address allocated to the federated service, and the federated endpoint set includes the IP address of the deployment unit in each working cluster used to implement the federated service; a deployment unit IP address determination module 530, used to determine the IP address of the second deployment unit in the target working cluster used to implement the target federated service based on the target IP address, the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster; and a federated service request sending module 540, used to send the target federated service request to the second deployment unit based on the IP address of the second deployment unit, so that the second deployment unit responds to the target federated service request.
[0090] The technical solution of this embodiment utilizes a federated management cluster to manage each working cluster and determines the federated gateway address and federated endpoint set corresponding to each federated service. This allows the federated domain name cluster to process the target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster, based on the federated gateway address and federated endpoint set corresponding to each federated service, and to determine and return the target IP address corresponding to the target federated service. The current working cluster, based on the target IP address returned by the federated domain name cluster and the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines the IP address of the second deployment unit in the target working cluster used to implement the target federated service. Based on the IP address of the second deployment unit, the target federated service request is sent to the second deployment unit, enabling the second deployment unit to respond to the target federated service request. Since the target federated cluster can be different from the current federated cluster, cross-cluster service access is achieved, thereby reducing the limitations of service access and meeting personalized business needs.
[0091] Optionally, the domain name resolution request generation module 510 is specifically used for:
[0092] Generate a current domain name resolution request corresponding to the service to be accessed in the first deployment unit of the current working cluster; if the service identifier information corresponding to the service to be accessed is detected to be the target federated service identifier information, then determine that the current domain name resolution request is the target domain name resolution request corresponding to the target federated service.
[0093] Optionally, a federated domain name cluster includes:
[0094] The request resolution module is used to parse the target domain name resolution request to obtain the target service type corresponding to the target federated service;
[0095] The federal domain name resolution module is used to perform domain name resolution based on the target service type and the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster, and to determine the target IP address corresponding to the target federal service after resolution.
[0096] Optionally, the federated domain name resolution module is specifically used for:
[0097] If the target service type is a ClusterIP service type, then based on the federal gateway address corresponding to each federal service determined by the federal management cluster, the target federal gateway address corresponding to the target federal service is determined, and the target cluster IP address corresponding to the target federal gateway address is determined as the target IP address;
[0098] If the target service type is a headless service type, then based on the set of federal endpoints corresponding to each federal service determined by the federal management cluster, the target federal endpoint set corresponding to the target federal service is determined, and all deployment unit IP addresses in the target federal endpoint set are determined as the target IP address.
[0099] Optionally, the deployment unit IP address determination module 530 is specifically used for:
[0100] If the target IP address is detected as the target cluster IP address based on the federal gateway address corresponding to each federal service determined by the federal management cluster, then load balancing is performed based on the target cluster IP address and the set of federal endpoints corresponding to each federal service determined by the federal management cluster to determine the IP address of the second deployment unit in the target working cluster used to implement the target federal service.
[0101] If the target IP address is detected to be multiple deployment unit IP addresses, then the IP address of the second deployment unit in the target work cluster used to implement the target federated service is determined from the multiple deployment unit IP addresses.
[0102] Optionally, the federated management cluster may include:
[0103] The object retrieval module is used to retrieve the currently deployed object and the current service object corresponding to the current federated service to be created.
[0104] The deployment object sending module is used to send the current deployment object to the deployment work cluster to create a corresponding deployment unit based on the deployment work cluster information in the current deployment object, so that the created deployment unit can implement the current federated service;
[0105] The information addition module is used to assign an IP address to the current service object, determine the current cluster IP address corresponding to the current service object, and add the current cluster IP address, federated service identification information and deployment unit selector to the current service object;
[0106] The service object sending module is used to send the added current service object to each work cluster, so that each work cluster can determine the current endpoint set corresponding to the current federated service in each work cluster based on the received current service object. The current endpoint set includes the IP address of the current deployment unit used to implement the current federated service.
[0107] The global information determination module is used to generate the current federation endpoint set corresponding to the current federation service based on the current endpoint set sent by each working cluster, and to convert the current service object into a federation gateway to obtain the current federation gateway address corresponding to the current federation service.
[0108] Optionally, each work cluster includes:
[0109] The endpoint set determination module is used to determine the current deployment unit for implementing the current federated service from the created deployment units based on the deployment unit selector in the received current service object; and to determine the current endpoint set corresponding to the current federated service in each work cluster based on the IP address of the current deployment unit.
[0110] The service request processing apparatus provided in the embodiments of the present invention can execute the service request processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the service request processing method.
[0111] It is worth noting that in the embodiments of the above-mentioned service request processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.
[0112] Figure 6 This is a schematic diagram of a service request processing system according to an embodiment of the present invention. This embodiment is applicable to cross-cluster access between multiple work clusters managed by a federated management cluster. Figure 6As shown, the system specifically includes: a federal management cluster 610, multiple working clusters 620, and a federal domain name cluster 630.
[0113] The federated management cluster 610 manages each working cluster 620 and determines the federated gateway address and federated endpoint set corresponding to each federated service. Each working cluster 620 implements the service request processing method provided in any embodiment of the present invention. The federated domain name cluster 630 processes the target domain name resolution request corresponding to the target federated service to be accessed by each working cluster based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, and determines and returns the target IP address corresponding to the target federated service.
[0114] The service request processing system provided in this embodiment manages each working cluster using a federated management cluster and determines the federated gateway address and federated endpoint set corresponding to each federated service. This allows the federated domain name cluster to process the target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster, based on the federated gateway address and federated endpoint set corresponding to each federated service, and to determine and return the target IP address corresponding to the target federated service. The current working cluster, based on the target IP address returned by the federated domain name cluster and the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines the IP address of the second deployment unit in the target working cluster used to implement the target federated service. Based on the IP address of the second deployment unit, the system sends the target federated service request to the second deployment unit, enabling the second deployment unit to respond to the target federated service request. Since the target federated cluster can be different from the current federated cluster, cross-cluster service access is achieved, thereby reducing the limitations of service access and meeting personalized business needs.
[0115] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 7 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 7 The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.
[0116] like Figure 7 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).
[0117] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0118] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.
[0119] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 7 Not shown; usually referred to as a "hard drive"). Although Figure 7 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.
[0120] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.
[0121] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0122] Processing unit 16 executes various functional applications and data processing by running programs stored in system memory 28, such as implementing the steps of a service request processing method provided in any embodiment of the present invention, the method including:
[0123] Generate a target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster;
[0124] The target domain name resolution request is sent to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines and returns the target IP address corresponding to the target federated service, wherein the federated gateway address refers to the cluster IP address allocated to the federated service, and the federated endpoint set includes the deployment unit IP address in each working cluster used to implement the federated service.
[0125] Based on the target IP address, the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster, the IP address of the second deployment unit in the target working cluster used to implement the target federal service is determined;
[0126] Based on the IP address of the second deployment unit, the target federated service request is sent to the second deployment unit so that the second deployment unit can respond to the target federated service request.
[0127] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the service request processing method provided in any embodiment of the present invention.
[0128] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the service request processing method steps provided in any embodiment of the present invention.
[0129] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0130] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.
[0131] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0132] Computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0133] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.
[0134] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.
Claims
1. A service request processing method, characterized in that, The method, applied to each work cluster managed by the federated management cluster, includes: Generate a target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current work cluster; The target domain name resolution request is sent to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines and returns the target IP address corresponding to the target federated service, wherein the federated gateway address refers to the cluster IP address allocated to the federated service, and the federated endpoint set includes the deployment unit IP address in each working cluster used to implement the federated service. Based on the target IP address, the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster, the IP address of the second deployment unit in the target working cluster used to implement the target federal service is determined; Based on the IP address of the second deployment unit, the target federated service request is sent to the second deployment unit so that the second deployment unit can respond to the target federated service request.
2. The method according to claim 1, characterized in that, Generate the target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster, including: Generate the current domain name resolution request for the service to be accessed by the first deployment unit in the current work cluster; If the service identifier information corresponding to the service to be accessed is detected to be the target federated service identifier information, then the current domain name resolution request is determined to be the target domain name resolution request corresponding to the target federated service.
3. The method according to claim 1, characterized in that, Based on the federated gateway address and federated endpoint set corresponding to each federated service determined through the federated management cluster, the target domain name resolution request is processed to determine and return the target IP address corresponding to the target federated service, including: The target domain name resolution request is parsed to obtain the target service type corresponding to the target federated service; Based on the target service type and the set of federal gateway addresses and federal endpoints corresponding to each federal service determined by the federal management cluster, domain name resolution is performed to determine the target IP address corresponding to the target federal service after resolution.
4. The method according to claim 3, characterized in that, The process of performing domain name resolution based on the target service type and the set of federal gateway addresses and federal endpoints corresponding to each federal service determined by the federal management cluster, to determine the target IP address corresponding to the target federal service after resolution, includes: If the target service type is a ClusterIP service type, then based on the federal gateway address corresponding to each federal service determined by the federal management cluster, the target federal gateway address corresponding to the target federal service is determined, and the target cluster IP address corresponding to the target federal gateway address is determined as the target IP address; If the target service type is a headless service type, then based on the set of federal endpoints corresponding to each federal service determined by the federal management cluster, the target federal endpoint set corresponding to the target federal service is determined, and all deployment unit IP addresses in the target federal endpoint set are determined as the target IP address.
5. The method according to claim 1, characterized in that, Based on the target IP address, the federal gateway address and federal endpoint set corresponding to each federal service determined by the federal management cluster, the IP address of the second deployment unit in the target working cluster used to implement the target federal service is determined, including: If the target IP address is detected as the target cluster IP address based on the federal gateway address corresponding to each federal service determined by the federal management cluster, then load balancing is performed based on the target cluster IP address and the set of federal endpoints corresponding to each federal service determined by the federal management cluster to determine the IP address of the second deployment unit in the target working cluster used to implement the target federal service. If the target IP address is detected to be multiple deployment unit IP addresses, then the IP address of the second deployment unit in the target work cluster used to implement the target federated service is determined from the multiple deployment unit IP addresses.
6. The method according to any one of claims 1-5, characterized in that, The process of determining the federal gateway address and set of federal endpoints for each federal service includes: The federated management cluster obtains the currently deployed object and the current service object corresponding to the current federated service to be created. Based on the deployment work cluster information in the current deployment object, the current deployment object is sent to the deployment work cluster to create a corresponding deployment unit, so that the created deployment unit can implement the current federated service; Assign an IP address to the current service object, determine the current cluster IP address corresponding to the current service object, and add the current cluster IP address, federated service identification information, and deployment unit selector to the current service object; The added current service object is sent to each work cluster so that each work cluster can determine the current endpoint set corresponding to the current federated service in each work cluster based on the received current service object. The current endpoint set includes the IP address of the current deployment unit used to implement the current federated service. Based on the current endpoint set sent by each working cluster, generate the current federated endpoint set corresponding to the current federated service, convert the current service object into a federated gateway, and obtain the current federated gateway address corresponding to the current federated service.
7. The method according to claim 6, characterized in that, Each work cluster, based on the received current service object, determines the current endpoint set corresponding to the current federated service in each work cluster, including: Each work cluster determines the current deployment unit for implementing the current federated service from the created deployment units based on the deployment unit selector in the received current service object; Based on the IP address of the currently deployed unit, determine the current endpoint set corresponding to the current federated service in each working cluster.
8. A service request processing apparatus, characterized in that, The device, integrated into each work cluster managed by the federated management cluster, includes: The domain name resolution request generation module is used to generate the target domain name resolution request for the target federated service to be accessed by the first deployment unit in the current working cluster. The domain name resolution request sending module is used to send the target domain name resolution request to the federated domain name cluster, so that the federated domain name cluster processes the target domain name resolution request based on the federated gateway address and federated endpoint set corresponding to each federated service determined by the federated management cluster, determines and returns the target IP address corresponding to the target federated service, wherein the federated gateway address refers to the cluster IP address allocated to the federated service, and the federated endpoint set includes the deployment unit IP address in each working cluster used to implement the federated service; The deployment unit IP address determination module is used to determine the IP address of the second deployment unit in the target working cluster that implements the target federal service based on the target IP address, the federal gateway address and the set of federal endpoints corresponding to each federal service determined by the federal management cluster; The federated service request sending module is used to send a target federated service request to the second deployment unit based on the IP address of the second deployment unit, so that the second deployment unit can respond to the target federated service request.
9. A service request processing system, characterized in that, The system includes: a federated management cluster, multiple working clusters, and a federated domain name cluster; wherein... The federated management cluster is used to manage each working cluster and to determine the federated gateway address and federated endpoint set corresponding to each federated service; Each working cluster is used to implement the service request processing method as described in any one of claims 1-7.
10. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the service request processing method as described in any one of claims 1-7.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the service request processing method as described in any one of claims 1-7.
Citation Information
Patent Citations
Product service iteration method and device, electronic equipment and storage medium
CN115756806A
Migration method of Internet of Things cluster users
CN116095095A