Backend service calling method, device, electronic device and program product

Abstract

The embodiments of the present disclosure disclose a back-end service calling method, device, electronic device and program product, the method comprising: receiving a call request for a target service from a client; registering the call request for the target service as an IO event and placing it into a synchronization thread queue; utilizing a synchronization thread started on the CPU core to poll the IO events in the synchronization thread queue, and determining whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event; after the currently polled IO event is determined to be a synchronously executed IO event, the synchronization thread processes the synchronously executed IO event, and after the currently polled IO event is determined to be an asynchronously executed IO event, the synchronization thread sends the asynchronously executed IO event to a corresponding asynchronous thread for processing, and places the asynchronously executed IO event that has been completed by the asynchronous thread back into the synchronization thread queue.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of communication technology, and in particular to a backend service calling method, device, electronic device, and program product. Background Art

[0002] An API (Application Program Interface) gateway takes over all ingress traffic to a system or server, forwarding all user requests to the corresponding backend services. For example, an e-commerce system involves many backend microservices, such as membership, product, and recommendation services. This raises the question of how users can access these backend services through the client. If the business is relatively simple, each backend service can be assigned a separate domain name. However, this approach requires each backend service to repeatedly implement some of the same logic, such as authentication, rate limiting, and permission verification, complicating the implementation of backend services. Deploying each backend service requires operations and maintenance involvement, including applying for a domain name and configuring Nginx. This process is complex and labor-intensive, and the cost of manual work can be high. To address this issue, an API gateway can be introduced. Clients only need to interact with the API gateway, rather than communicating with the interfaces of each backend service separately. However, introducing an additional component introduces a potential point of failure. Therefore, implementing a high-performance, stable API gateway is a technical issue that currently needs to be addressed. Summary of the Invention

[0003] Embodiments of the present disclosure provide a backend service calling method, device, electronic device, and program product.

[0004] In a first aspect, an embodiment of the present disclosure provides a backend service calling method, wherein the method is executed on the current API gateway and includes:

[0005] Receive the client's call request for the target service;

[0006] Registering the target service call request as an IO event and placing it in the synchronization thread queue corresponding to the CPU core; the CPU core is located at the node where the current API gateway is located;

[0007] Utilize the synchronization thread started on the CPU core to poll the IO events in the synchronization thread queue, and determine whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0008] When the IO event currently polled is a synchronously executed IO event, the synchronous thread processes the synchronously executed IO event. When the IO event currently polled is an asynchronously executed IO event, the synchronous thread sends the asynchronously executed IO event to the corresponding asynchronous thread for processing. The asynchronously executed IO event completed by the asynchronous thread is put back into the synchronous thread queue.

[0009] In a second aspect, an embodiment of the present disclosure provides a multi-node deployment method for an API gateway, the method being executed on a management and control server, comprising:

[0010] Receive registration requests from one or more API gateways based on synchronous locking;

[0011] Determine the API gateway that first requests registration as the current central server;

[0012] Determine the API gateway that requested the second registration as the next central server; after an exception occurs in the current central server, the next central server is switched to the current central server;

[0013] After the API gateway is successfully registered, it performs the following steps:

[0014] Receive the client's call request for the target service;

[0015] Register the target service call request as an IO event and put it into the synchronous thread queue corresponding to the CPU core;

[0016] Using a synchronous thread to poll the IO events in the synchronous thread queue, and determining whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0017] When the IO event currently polled is an IO event for synchronous execution, the IO event for synchronous execution is executed in the synchronous thread. When the IO event currently polled is an IO event for asynchronous execution, the IO event for asynchronous execution is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is put back into the synchronous thread queue.

[0018] In a third aspect, an embodiment of the present disclosure provides a backend service calling method, which is executed on an API gateway cluster system, wherein the API gateway cluster system includes multiple API gateways and a management and control server; including:

[0019] The control server receives registration requests from one or more API gateways based on synchronous locking;

[0020] The control server determines the API gateway that first requests registration as the current central server;

[0021] The control server determines the API gateway that requested the second registration as the next central server; after an exception occurs in the current central server, the next central server is switched to the current central server;

[0022] After successful registration, the API gateway receives the client's call request for the target service, registers the call request for the target service as an IO event, and puts it into the synchronous thread queue corresponding to the CPU core, and uses the synchronous thread to poll the IO events in the synchronous thread queue, and determines whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. After the currently polled IO event is a synchronously executed IO event, the synchronously executed IO event is executed in the synchronous thread, and after the currently polled IO event is an asynchronously executed IO event, the asynchronously executed IO event is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is placed back into the synchronous thread queue.

[0023] In a fourth aspect, an embodiment of the present disclosure provides a backend service calling device, wherein the device is executed on the current API gateway; comprising:

[0024] A first receiving module is configured to receive a call request from a client to a target service;

[0025] A first registration module is configured to register the call request of the target service as an IO event and place it in the synchronization thread queue corresponding to the CPU core; the CPU core is located at the node where the current API gateway is located;

[0026] A first polling module is configured to use the synchronization thread started on the CPU core to poll the IO events in the synchronization thread queue, and determine whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0027] The first execution module is configured to, when the IO event currently polled is a synchronously executed IO event, process the synchronously executed IO event by the synchronous thread; and when the IO event currently polled is an asynchronously executed IO event, send the asynchronously executed IO event to the corresponding asynchronous thread for processing by the synchronous thread, and put the asynchronously executed IO event executed by the asynchronous thread back into the synchronous thread queue.

[0028] In a fifth aspect, an embodiment of the present disclosure provides a multi-node deployment device for an API gateway, the device being executed on a management and control server, including:

[0029] A second receiving module is configured to receive registration requests from one or more API gateways based on a synchronous locking method;

[0030] A first determining module is configured to determine the API gateway that first requests registration as the current central server;

[0031] The second determining module is configured to determine the API gateway that requested the second registration as the next central server; the next central server is switched to the current central server after an exception occurs in the current central server;

[0032] After successful registration, the API gateway is implemented as the following modules:

[0033] A third receiving module is configured to receive a call request from a client to a target service;

[0034] The second registration module is configured to register the call request of the target service as an IO event and put it into the synchronization thread queue corresponding to the CPU core;

[0035] The second polling module is configured to use the synchronous thread to poll the IO events in the synchronous thread queue, and determine whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0036] The second execution module is configured to execute the synchronously executed IO event in the synchronous thread after the currently polled IO event is a synchronously executed IO event, and to send the asynchronously executed IO event to the corresponding asynchronous thread for processing after the currently polled IO event is an asynchronously executed IO event, and put the asynchronously executed IO event executed by the asynchronous thread back into the synchronous thread queue.

[0037] In a sixth aspect, an embodiment of the present disclosure provides an API gateway cluster system, including multiple API gateways and a management and control server;

[0038] The control server receives registration requests from one or more API gateways based on synchronous locking;

[0039] The control server determines the API gateway that first requests registration as the current central server;

[0040] The control server determines the API gateway that requested the second registration as the next central server; after an exception occurs in the current central server, the next central server is switched to the current central server;

[0041] After successful registration, the API gateway receives the client's call request for the target service, registers the call request for the target service as an IO event, and puts it into the synchronous thread queue corresponding to the CPU core, and uses the synchronous thread to poll the IO events in the synchronous thread queue, and determines whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. After the currently polled IO event is a synchronously executed IO event, the synchronously executed IO event is executed in the synchronous thread, and after the currently polled IO event is an asynchronously executed IO event, the asynchronously executed IO event is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is placed back into the synchronous thread queue.

[0042] The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions.

[0043] In one possible design, the apparatus includes a memory and a processor. The memory is configured to store one or more computer instructions that enable the apparatus to perform the corresponding method, and the processor is configured to execute the computer instructions stored in the memory. The apparatus may also include a communication interface for communicating with other devices or a communication network.

[0044] In a seventh aspect, embodiments of the present disclosure provide an electronic device, comprising a memory and a processor, wherein the memory is configured to store one or more computer instructions that support any of the aforementioned devices in executing the aforementioned corresponding methods, and the processor is configured to execute the computer instructions stored in the memory. Any of the aforementioned devices may also include a communication interface for communicating with other devices or a communication network.

[0045] In an eighth aspect, an embodiment of the present disclosure provides a computer-readable storage medium for storing computer instructions used by any of the above-mentioned devices, which includes computer instructions involved in executing any of the above-mentioned methods.

[0046] In a ninth aspect, an embodiment of the present disclosure provides a computer program product comprising computer instructions, which, when executed by a processor, are used to implement the steps of the method described in any of the above aspects.

[0047] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0048] In the embodiment of the present disclosure, when the API gateway is implemented, a synchronous thread is used to poll the IO events registered in the synchronous thread queue, and based on the pre-script configuration, it is determined whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. When it is a synchronously executed IO event, the IO event is processed by the synchronous thread, and when it is an asynchronously executed IO event, the synchronous thread hands it over to the asynchronous thread for processing. This can avoid the business that requires a long time to process being blocked at the synchronous thread, resulting in slow system response speed and other problems. At the same time, the synchronous thread and the asynchronous thread are coordinated by polling the queue, the asynchronous thread obeys the synchronous thread's allocation, and the asynchronous thread returns the processing result to the queue, thereby realizing an efficient API gateway.

[0049] It should be understood that the foregoing general description and the following detailed description are merely exemplary and explanatory and are not restrictive of the embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Other features, objectives and advantages of the embodiments of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0051] Figure 1 A flowchart showing a backend service calling method according to an embodiment of the present disclosure is shown;

[0052] Figure 2 A flowchart illustrating a multi-node deployment method for an API gateway according to an embodiment of the present disclosure is shown;

[0053] Figure 3 A flowchart showing a backend service calling method according to an embodiment of the present disclosure is shown;

[0054] Figure 4 A schematic diagram of the design architecture of an API gateway according to one embodiment of the present disclosure is shown;

[0055] Figure 5 A structural block diagram of a backend service calling device according to an embodiment of the present disclosure is shown;

[0056] Figure 6 A structural block diagram of a multi-node deployment device for an API gateway according to an embodiment of the present disclosure is shown;

[0057] Figure 7 A structural block diagram of an API gateway cluster system according to an embodiment of the present disclosure is shown;

[0058] Figure 8 It is a structural diagram of a computer system suitable for implementing a backend service calling method and / or multi-node deployment of an API gateway according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0059] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0060] In the embodiments of the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the existence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in this specification, and are not intended to exclude the possibility of one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof existing or being added.

[0061] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present disclosure can be combined with each other. The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0062] Figure 1 A flowchart of a backend service calling method according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the backend service calling method includes the following steps:

[0063] In step S101, a call request for a target service from a client is received;

[0064] In step S102, the call request of the target service is registered as an IO event and placed in the synchronization thread queue corresponding to the CPU core; the CPU core is located at the node where the current API gateway is located;

[0065] In step S103, the synchronization thread started on the CPU core is used to poll the IO events in the synchronization thread queue, and it is determined whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0066] In step S104, after the IO event currently polled is an IO event executed synchronously, the synchronous thread processes the IO event executed synchronously, and after the IO event currently polled is an IO event executed asynchronously, the synchronous thread sends the asynchronously executed IO event to the corresponding asynchronous thread for processing, and the asynchronously executed IO event executed by the asynchronous thread is put back into the synchronous thread queue.

[0067] As mentioned above, an API (Application Program Interface) gateway manages all ingress traffic to a system or server, forwarding all user requests to the corresponding backend services. For example, an e-commerce system involves many backend microservices, such as membership, product, and recommendation services. This raises the question of how users can access these backend services through client applications. If the business is relatively simple, each backend service can be assigned a separate domain name. However, this approach requires each backend service to duplicate some of the same logic, such as authentication, rate limiting, and permission verification, complicating the implementation of backend services. Deploying each backend service requires operations and maintenance involvement, including domain name registration and Nginx configuration. This process is complex and labor-intensive. To address this issue, an API gateway can be introduced. Clients only need to interact with the API gateway, rather than communicating with the interfaces of each backend service separately. However, introducing an additional component introduces a potential point of failure. Therefore, implementing a high-performance and stable API gateway is a current technical challenge that needs to be addressed.

[0068] In a microservices architecture, large services are typically broken down into independent microservices, each of which typically provides external services via a RESTful API. However, for human-computer interaction, data from different microservices needs to be displayed on a single page, requiring a unified entry point for calling the corresponding microservice APIs. In this scenario, an API gateway acts as a unified entry point for multiple services. It encapsulates the complex internal structure of the system and may also provide other common API management and call functions, such as authentication, rate limiting, and flow control.

[0069] From a deployment perspective, in a microservice deployment model that doesn't use an API gateway, clients interact directly with a load balancer to complete service calls. However, this model doesn't support dynamic scalability. Each time a service is launched, the load balancer must be deployed or modified, making it impossible to dynamically turn services on and off. To deactivate a service, operations personnel must remove the service address from the load balancer. Furthermore, each microservice must implement its own control over interface call flow control and security, increasing its complexity and violating the single responsibility principle of microservice design.

[0070] The API gateway, as the unified entry point to the system, integrates microservices while being client-friendly and shielding system complexity and diversity. Using the API gateway allows for seamless dynamic scaling of microservices. It can automatically disconnect inaccessible services without requiring human intervention. As the unified entry point to the system, the API gateway allows for the common functions of each microservice to be implemented within the API gateway, minimizing the responsibilities of each service.

[0071] Existing API gateways typically operate in synchronous blocking mode. In this mode, each incoming request is assigned a dedicated thread to handle it. This thread is not released and returned to the container thread pool until a response is returned to the client. If the backend service call is time-consuming, this thread will be blocked. During this period, thread resources are occupied, preventing other tasks from taking place, which can easily lead to slow or unresponsive service calls.

[0072] Existing technologies have also proposed asynchronous blocking mode, but the API gateway in asynchronous blocking mode is currently limited to theory and is relatively complex and difficult to implement.

[0073] The disclosed embodiment implements a simple and easy-to-implement asynchronous API gateway, and the backend service call method proposed in the disclosed embodiment can be executed on the API gateway. In the API gateway, a synchronization thread can be started for each CPU core. The synchronization thread polls the IO events in the synchronization thread queue and determines whether the polled IO event is a synchronously executed IO event or an asynchronously executed IO event. When the IO event is a synchronously executed IO event, the synchronization thread can process the IO event. When the IO event is an asynchronously executed IO event, the synchronization thread will hand over the asynchronously executed IO event to the corresponding asynchronous thread for processing.

[0074] In some embodiments, the current API gateway also starts a receiving thread, which is used to poll and receive call requests from the client to the target service, and delegate the call request polling to the working thread. The working thread registers IO events based on the call request and puts the IO event into the synchronization thread queue for the synchronization thread to perform call processing.

[0075] In some embodiments, whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event depends mainly on the business corresponding to the event, that is, the service requested by the client. In some embodiments, it can be determined based on the configuration in the business script corresponding to the service.

[0076] When a synchronous thread currently polls a synchronous IO event, it processes the synchronous IO event. When it currently polls an asynchronous IO event, it hands the event over to the corresponding asynchronous thread for processing. It should be noted that different services can correspond to different asynchronous threads. If the asynchronous thread corresponding to the service that the asynchronous IO event requires access to is not started, the synchronous thread can also start the asynchronous thread so that the asynchronous thread can process the asynchronous IO event.

[0077] After the asynchronous thread processes the asynchronously executed IO event, it puts the execution completion task of the asynchronously executed IO event into the synchronous thread queue, so that the synchronous thread polls the execution completion task from the synchronous thread queue and performs corresponding processing, such as returning the processing result to the client, or performing the next processing, etc.

[0078] In the embodiment of the present disclosure, when the API gateway is implemented, a synchronous thread is used to poll the IO events registered in the synchronous thread queue, and based on the pre-script configuration, it is determined whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. When it is a synchronously executed IO event, the IO event is processed by the synchronous thread, and when it is an asynchronously executed IO event, the synchronous thread hands it over to the asynchronous thread for processing. This can avoid the business that requires a long time to process being blocked at the synchronous thread, resulting in slow system response speed and other problems. At the same time, the synchronous thread and the asynchronous thread are coordinated by polling the queue, the asynchronous thread obeys the synchronous thread's allocation, and the asynchronous thread returns the processing result to the queue, thereby realizing an efficient API gateway.

[0079] In one embodiment of the present disclosure, the method further includes:

[0080] Receive registration requests for at least one or more target services; the registration requests include a business script corresponding to the target services; the business script is configured with a service identifier and a service address of the target service, a calling method of the target service, and an identifier of whether the target service corresponds to synchronously executed IO events or asynchronously executed IO events;

[0081] The business script of the target service is stored in a specified directory so that the current API gateway can support the client's call to the newly registered target service.

[0082] In this optional implementation, the API gateway can support calls to multiple target services. Different target services can first register with the API gateway. The registration request can carry the business script of the target service. The business script contains relevant configuration information of the target service, such as the service identifier, service address, the calling method of the target service, and the identifier of whether the target service corresponds to a synchronous or asynchronous IO event. The business script can be stored in a designated directory and can take effect immediately. That is, the client can request to call the target service through the API gateway, and the API gateway supports the client's call to the target service through the relevant configuration information in the business script. For example, the client can send a call request for the target service to the API gateway through the access layer http socket. After the call request enters the polling queue of the receiving thread, the receiving thread delegates the polled call request to the worker thread. The worker thread registers an IO event based on the call request and puts the IO event into the synchronization thread queue. It should be noted that the IO event placed in the synchronization thread queue includes the business script corresponding to the target service. That is, the worker thread will retrieve its corresponding business script from the designated directory, register it as an IO event, and then put it into the synchronization thread queue for the synchronization thread to poll and retrieve.

[0083] In one embodiment of the present disclosure, the synchronization thread determines whether the IO event is a synchronously executed IO event or an asynchronously executed IO event based on a service script of a target service corresponding to the IO event.

[0084] In one embodiment of the present disclosure, the method further includes:

[0085] After the current API gateway is started, a registration request is submitted to the management and control server; the registration request includes the address information of the current API gateway;

[0086] Receive the registration confirmation information returned by the control server, wherein the registration confirmation information includes information on whether the node where the current API gateway is located is registered as the current central server.

[0087] In this optional implementation, the API gateway can be implemented in the cluster node, that is, the API gateway can be started on multiple nodes. The client can send the call request of the target service to any API gateway node in the cluster through the cluster management node, and the API gateway on the node will process the call request.

[0088] A central server can be elected in the cluster where multiple API gateways are located, and the central server will centrally manage and allocate resources, such as limiting traffic.

[0089] In the embodiment of the present disclosure, the setting of the central server adopts a self-election strategy, and the first node that requests to register as an API gateway to the control server is determined as the current central server, and the second node that requests to register as an API gateway to the control server is determined as the next central server, while other API gateways are ordinary API gateways. The control server can determine the node where the API gateway is located as the current central server and the next central server based on the existing registration order of the API gateway, and then send information to the corresponding API gateway whether it is the current central server. If it is not the current central server, it can also return relevant information of the current central server to the API gateway, so that the API gateway that is not the central server can request the current central server to allocate lock resources based on the relevant information. It can be understood that lock resources are resources that need to be uniformly allocated to multiple API gateways, and need to be allocated in a locked manner.

[0090] The current central server and the next central server can be understood as the primary and backup servers, with the current central server being the primary server and the next central server being the backup server. A long link can be established between the current central server and the next central server, and the central server will synchronize the relevant information about resource allocation performed on it to the next central server, so that when the current central server is unable to provide services normally, the next central server will switch to the current central server. It should be noted that the current central server, the next central server, and other API gateways, as members of the API gateway, also provide normal API gateway services. When locked resource allocation is required, they all request locks from the central server, and the central server will make the corresponding lock resource allocation.

[0091] In one embodiment of the present disclosure, when the node where the current API gateway is located is the first API gateway registered with the control server, the current API gateway is determined as the current central server; when the node where the current API gateway is located is the second API gateway registered with the control server, the current API gateway is determined as the next central server; the next central server is switched to the current central server after an exception occurs in the current central server.

[0092] In one embodiment of the present disclosure, when the node where the current API gateway is located is the current central server, the method further includes:

[0093] Receive distributed lock resource requests from other nodes; other distributed API gateways are running on the other nodes;

[0094] The requested lock resource is allocated based on the distributed lock resource request, and the allocated lock resource information is synchronized to the next central server; the next central server is switched to the current central server after an exception occurs in the current central server.

[0095] In this optional implementation, as described above, the current central server receives requests for distributed lock resources from other API gateways and uniformly allocates distributed lock resources to each API gateway. At the same time, the current central server can also synchronize the currently allocated lock resource information to the next central server, so that the relevant information about lock resource allocation on the next central server remains synchronized with the current central server. Even if an exception occurs on the current central server, the next central server can be switched to the current central server and continue to perform its corresponding duties.

[0096] In one embodiment of the present disclosure, when the node where the current API gateway is located is not the current central server, the method further includes:

[0097] In response to a demand event for a distributed lock resource, a distributed lock resource request is sent to the current central server so that the current central server allocates the requested lock resource to the current API gateway.

[0098] In this optional implementation, as described above, after an API gateway that is not the current central server generates a demand for distributed lock resources, it can request the current central server for the allocation of distributed lock resources and receive the allocation of distributed lock resources from the current central server. The API gateway that is not the current central server includes the next central server and other nodes.

[0099] In one embodiment of the present disclosure, the number of synchronization threads started on the current API gateway is related to the number of CPU cores on the node where the current API gateway is located.

[0100] In this optional implementation, the number of synchronous threads can be related to the number of CPU cores on the node where the API gateway is located. If there is only one CPU core, only one synchronous thread is started; if there are multiple CPU cores, multiple synchronous threads are started. The number of asynchronous threads can be based on the number of backend services, for example, one asynchronous thread can be associated with each backend service.

[0101] Figure 2 A flowchart of a multi-node deployment method of an API gateway according to an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the multi-node deployment method of the API gateway includes the following steps:

[0102] In step S201, one or more API gateway registration requests are received based on a synchronous locking approach;

[0103] In step S202, the API gateway that first requests registration is determined as the current central server;

[0104] In step S203, the API gateway that requested the second registration is determined as the next central server; the next central server is switched to the current central server after an exception occurs in the current central server;

[0105] After the API gateway is successfully registered, it performs the following steps:

[0106] Receive the client's call request for the target service;

[0107] Register the target service call request as an IO event and put it into the synchronous thread queue corresponding to the CPU core;

[0108] Using a synchronous thread to poll the IO events in the synchronous thread queue, and determining whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0109] When the IO event currently polled is an IO event for synchronous execution, the IO event for synchronous execution is executed in the synchronous thread. When the IO event currently polled is an IO event for asynchronous execution, the IO event for asynchronous execution is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is put back into the synchronous thread queue.

[0110] As mentioned above, an API (Application Program Interface) gateway manages all ingress traffic to a system or server, forwarding all user requests to the corresponding backend services. For example, an e-commerce system involves many backend microservices, such as membership, product, and recommendation services. This raises the question of how users can access these backend services through client applications. If the business is relatively simple, each backend service can be assigned a separate domain name. However, this approach requires each backend service to duplicate some of the same logic, such as authentication, rate limiting, and permission verification, complicating the implementation of backend services. Deploying each backend service requires operations and maintenance involvement, including domain name registration and Nginx configuration. This process is complex and labor-intensive. To address this issue, an API gateway can be introduced. Clients only need to interact with the API gateway, rather than communicating with the interfaces of each backend service separately. However, introducing an additional component introduces a potential point of failure. Therefore, implementing a high-performance and stable API gateway is a current technical challenge that needs to be addressed.

[0111] In a microservices architecture, large services are typically broken down into independent microservices, each of which typically provides external services via a RESTful API. However, for human-computer interaction, data from different microservices needs to be displayed on a single page, requiring a unified entry point for calling the corresponding microservice APIs. In this scenario, an API gateway acts as a unified entry point for multiple services. It encapsulates the complex internal structure of the system and may also provide other common API management and call functions, such as authentication, rate limiting, and flow control.

[0112] From a deployment perspective, in a microservice deployment model that doesn't use an API gateway, clients interact directly with a load balancer to complete service calls. However, this model doesn't support dynamic scalability. Each time a service is launched, the load balancer must be deployed or modified, making it impossible to dynamically turn services on and off. To deactivate a service, operations personnel must remove the service address from the load balancer. Furthermore, each microservice must implement its own control over interface call flow control and security, increasing its complexity and violating the single responsibility principle of microservice design.

[0113] The API gateway, as the unified entry point to the system, integrates microservices while being client-friendly and shielding system complexity and diversity. Using the API gateway allows for seamless dynamic scaling of microservices. It can automatically disconnect inaccessible services without requiring human intervention. As the unified entry point to the system, the API gateway allows for the common functions of each microservice to be implemented within the API gateway, minimizing the responsibilities of each service.

[0114] Existing API gateways typically operate in synchronous blocking mode. In this mode, each incoming request is assigned a dedicated thread to handle it. This thread is not released and returned to the container thread pool until a response is returned to the client. If the backend service call is time-consuming, this thread will be blocked. During this period, thread resources are occupied, preventing other tasks from taking place, which can easily lead to slow or unresponsive service calls.

[0115] Existing technologies have also proposed asynchronous blocking mode, but the API gateway in asynchronous blocking mode is currently limited to theory and is relatively complex and difficult to implement.

[0116] In one embodiment of the present disclosure, the multi-node deployment method of the API grid is executed on a management and control server.

[0117] The disclosed embodiment implements a simple and easy-to-implement asynchronous API gateway, and the backend service call method proposed in the disclosed embodiment can be executed on the API gateway. In the API gateway, a synchronization thread can be started for each CPU core. The synchronization thread polls the IO events in the synchronization thread queue and determines whether the polled IO event is a synchronously executed IO event or an asynchronously executed IO event. When the IO event is a synchronously executed IO event, the synchronization thread can process the IO event. When the IO event is an asynchronously executed IO event, the synchronization thread will hand over the asynchronously executed IO event to the corresponding asynchronous thread for processing.

[0118] In some embodiments, the current API gateway also starts a receiving thread, which is used to poll the received client's call request for the target service, and delegate the polled call request to the working thread. The working thread registers the IO event based on the call request and puts the IO event into the synchronization thread queue for the synchronization thread to perform call processing.

[0119] In some embodiments, whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event depends mainly on the business corresponding to the event, that is, the service requested by the client. In some embodiments, it can be determined based on the configuration in the business script corresponding to the service.

[0120] When a synchronous thread currently polls a synchronous IO event, it processes the synchronous IO event. When it currently polls an asynchronous IO event, it hands the event over to the corresponding asynchronous thread for processing. It should be noted that different services can correspond to different asynchronous threads. If the asynchronous thread corresponding to the service that the asynchronous IO event requires access to is not started, the synchronous thread can also start the asynchronous thread so that the asynchronous thread can process the asynchronous IO event.

[0121] After the asynchronous thread processes the asynchronously executed IO event, it puts the execution completion task of the asynchronously executed IO event into the synchronous thread queue, so that the synchronous thread polls the execution completion task from the synchronous thread queue and performs corresponding processing, such as returning the processing result to the client, or performing the next processing, etc.

[0122] In the embodiment of the present disclosure, when the API gateway is implemented, a synchronous thread is used to poll the IO events registered in the synchronous thread queue, and based on the script configuration written in advance for the service call, it is determined whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. When it is a synchronously executed IO event, the IO event is processed by the synchronous thread, and when it is an asynchronously executed IO event, the synchronous thread hands it over to the asynchronous thread for processing. This can avoid the business that requires a long time to process being blocked at the synchronous thread, resulting in slow system response speed and other problems. At the same time, the synchronous thread and the asynchronous thread are realized to work together by polling the queue, the asynchronous thread obeys the synchronous thread's allocation, and the asynchronous thread returns the processing result to the queue, thereby realizing an efficient API gateway.

[0123] The API gateway can be implemented in the cluster nodes, that is, the API gateway can be started on multiple nodes. The client can send the call request of the target service to any API gateway node in the cluster through the cluster management node, and the API gateway on the node will process the call request.

[0124] A central server can be elected in the cluster where multiple API gateways are located, and the central server will centrally manage and allocate resources, such as limiting traffic.

[0125] The current central server and the next central server can be understood as the primary and backup servers, with the current central server being the primary server and the next central server being the backup server. A long link can be established between the current central server and the next central server, and the central server will synchronize the relevant information about resource allocation performed on it to the next central server, so that when the current central server is unable to provide services normally, the next central server will switch to the current central server. It should be noted that the current central server, the next central server, and other API gateways, as members of the API gateway, also provide normal API gateway services. When locked resource allocation is required, they all request locks from the central server, and the central server will make the corresponding lock resource allocation.

[0126] In the embodiment of the present disclosure, a self-election strategy is adopted for the central server, and the first node that requests to register as an API gateway to the control server is determined as the current central server, and the second node that requests to register as an API gateway to the control server is determined as the next central server, while other API gateways are ordinary API gateways. The control server can determine the node where the API gateway is located as the current central server and the next central server based on the registration order of the API gateway, and then send information to the corresponding API gateway on whether it is the current central server. If it is not the current central server, it can also return relevant information of the current central server to the API gateway, so that the API gateway that is not the central server can request the current central server to allocate lock resources based on the relevant information. It can be understood that lock resources are resources that need to be uniformly allocated to multiple API gateways, and need to be allocated by locking.

[0127] In addition, the embodiments of the present disclosure implement a cluster of API gateways and a simple and efficient centralized resource management model, thereby improving exception handling capabilities and improving the service efficiency of the clustered API gateways.

[0128] In one embodiment of the present disclosure, the method further comprises:

[0129] In response to the abnormal restart completion event, the existing local API gateway registration information is cleared; the API gateway registration information includes information of the current central server and the next central server;

[0130] Update the local API gateway registration information based on the actual current central server information and the next central server information.

[0131] In this optional implementation, as described above, the management and control server determines the first registered API gateway as the current central server and the second registered API as the next central server. When an exception occurs in the current central server, the next central server is switched to the current central server.

[0132] When an exception occurs in the control server, it does not affect the current central server. After the control server is restarted, all registrations carry the corresponding fields of the current central server and the next central server. Therefore, after the control server is restarted, the registration table of the API gateway can be cleared and the actual IP addresses of the current central server and the next central server after the restart can be automatically filled in, thereby ensuring that it is not related to the exception of the control server.

[0133] In one embodiment of the present disclosure, the method further includes:

[0134] Receive registration requests for at least one or more target services; the registration requests include a business script corresponding to the target services; the business script is configured with a service identifier and a service address of the target service, a calling method of the target service, and an identifier of whether the target service corresponds to synchronously executed IO events or asynchronously executed IO events;

[0135] The business script of the target service is stored in a specified directory so that the current API gateway can support the client's call to the newly registered target service.

[0136] In this optional implementation, the API gateway can support calls to multiple target services. Different target services can first register with the API gateway. The registration request can carry the business script of the target service. The business script contains relevant configuration information of the target service, such as the service identifier, service address, the calling method of the target service, and the identifier of whether the target service corresponds to a synchronous or asynchronous IO event. The business script can be stored in a designated directory and can take effect immediately. That is, the client can request to call the target service through the API gateway, and the API gateway supports the client's call to the target service through the relevant configuration information in the business script. For example, the client can send a call request for the target service to the API gateway through the access layer http socket. After the call request enters the polling queue of the receiving thread, the receiving thread delegates the polled call request to the worker thread. The worker thread registers an IO event based on the call request and puts the IO event into the synchronization thread queue. It should be noted that the IO event placed in the synchronization thread queue includes the business script corresponding to the target service. That is, the worker thread will retrieve its corresponding business script from the designated directory, register it as an IO event, and then put it into the synchronization thread queue for the synchronization thread to poll and retrieve.

[0137] In one embodiment of the present disclosure, the synchronization thread determines whether the IO event is a synchronously executed IO event or an asynchronously executed IO event based on a service script of a target service corresponding to the IO event.

[0138] In one embodiment of the present disclosure, when the node where the current API gateway is located is the current central server, the method further includes:

[0139] Receive distributed lock resource requests from other nodes; other distributed API gateways are running on the other nodes;

[0140] The requested lock resource is allocated based on the distributed lock resource request, and the allocated lock resource information is synchronized to the next central server; the next central server is switched to the current central server after an exception occurs in the current central server.

[0141] In this optional implementation, as described above, the current central server receives requests for distributed lock resources from other API gateways and uniformly allocates distributed lock resources to each API gateway. At the same time, the current central server can also synchronize the currently allocated lock resource information to the next central server, so that the relevant information about lock resource allocation on the next central server remains synchronized with the current central server. Even if an exception occurs on the current central server, the next central server can be switched to the current central server and continue to perform its corresponding duties.

[0142] In one embodiment of the present disclosure, when the node where the current API gateway is located is not the current central server, the method further includes:

[0143] In response to a demand event for a distributed lock resource, a distributed lock resource request is sent to the current central server so that the current central server allocates the requested lock resource to the current API gateway.

[0144] In this optional implementation, as described above, after an API gateway that is not the current central server generates a demand for distributed lock resources, it can request the current central server for the allocation of distributed lock resources and receive the allocation of distributed lock resources from the current central server. The API gateway that is not the current central server includes the next central server and other nodes.

[0145] In one embodiment of the present disclosure, the number of synchronization threads started on the current API gateway is related to the number of CPU cores on the node where the current API gateway is located.

[0146] In this optional implementation, the number of synchronous threads can be related to the number of CPU cores on the node where the API gateway is located. If there is only one CPU core, only one synchronous thread is started; if there are multiple CPU cores, multiple synchronous threads are started. The number of asynchronous threads can be based on the number of backend services, for example, one asynchronous thread can be associated with each backend service.

[0147] Figure 3 A flowchart of a backend service calling method according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the backend service calling method includes the following steps:

[0148] In step S301, the management and control server receives registration requests from one or more API gateways based on a synchronous locking method;

[0149] In step S302, the control server determines the API gateway that first requests registration as the current central server;

[0150] In step S303, the control server determines the API gateway that requested the second registration as the next central server; the next central server is switched to the current central server after an exception occurs in the current central server;

[0151] In step S304, after the API gateway is successfully registered, it receives the client's call request for the target service, registers the call request for the target service as an IO event, and puts it into the synchronous thread queue corresponding to the CPU core, and uses the synchronous thread to poll the IO events in the synchronous thread queue, and determines whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. After the currently polled IO event is a synchronously executed IO event, the synchronously executed IO event is executed in the synchronous thread, and after the currently polled IO event is an asynchronously executed IO event, the asynchronously executed IO event is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been completed by the asynchronous thread is placed back into the synchronous thread queue.

[0152] As mentioned above, an API (Application Program Interface) gateway manages all ingress traffic to a system or server, forwarding all user requests to the corresponding backend services. For example, an e-commerce system involves many backend microservices, such as membership, product, and recommendation services. This raises the question of how users can access these backend services through client applications. If the business is relatively simple, each backend service can be assigned a separate domain name. However, this approach requires each backend service to duplicate some of the same logic, such as authentication, rate limiting, and permission verification, complicating the implementation of backend services. Deploying each backend service requires operations and maintenance involvement, including domain name registration and Nginx configuration. This process is complex and labor-intensive. To address this issue, an API gateway can be introduced. Clients only need to interact with the API gateway, rather than communicating with the interfaces of each backend service separately. However, introducing an additional component introduces a potential point of failure. Therefore, implementing a high-performance and stable API gateway is a current technical challenge that needs to be addressed.

[0153] In a microservices architecture, large services are typically broken down into independent microservices, each of which typically provides external services via a RESTful API. However, for human-computer interaction, data from different microservices needs to be displayed on a single page, requiring a unified entry point for calling the corresponding microservice APIs. In this scenario, an API gateway acts as a unified entry point for multiple services. It encapsulates the complex internal structure of the system and may also provide other common API management and call functions, such as authentication, rate limiting, and flow control.

[0154] From a deployment perspective, in a microservice deployment model that doesn't use an API gateway, clients interact directly with a load balancer to complete service calls. However, this model doesn't support dynamic scalability. Each time a service is launched, the load balancer must be deployed or modified, making it impossible to dynamically turn services on and off. To deactivate a service, operations personnel must remove the service address from the load balancer. Furthermore, each microservice must implement its own control over interface call flow control and security, increasing its complexity and violating the single responsibility principle of microservice design.

[0155] The API gateway, as the unified entry point to the system, integrates microservices while being client-friendly and shielding system complexity and diversity. Using the API gateway allows for seamless dynamic scaling of microservices. It can automatically disconnect inaccessible services without requiring human intervention. As the unified entry point to the system, the API gateway allows for the common functions of each microservice to be implemented within the API gateway, minimizing the responsibilities of each service.

[0156] Existing API gateways typically operate in synchronous blocking mode. In this mode, each incoming request is assigned a dedicated thread to handle it. This thread is not released and returned to the container thread pool until a response is returned to the client. If the backend service call is time-consuming, this thread will be blocked. During this period, thread resources are occupied, preventing other tasks from taking place, which can easily lead to slow or unresponsive service calls.

[0157] Existing technologies have also proposed asynchronous blocking mode, but the API gateway in asynchronous blocking mode is currently limited to theory and is relatively complex and difficult to implement.

[0158] In one embodiment of the present disclosure, the multi-node deployment method of the API grid is executed on a management and control server.

[0159] The disclosed embodiment implements a simple and easy-to-implement asynchronous API gateway, and the backend service call method proposed in the disclosed embodiment can be executed on the API gateway. In the API gateway, a synchronization thread can be started for each CPU core. The synchronization thread polls the IO events in the synchronization thread queue and determines whether the polled IO event is a synchronously executed IO event or an asynchronously executed IO event. When the IO event is a synchronously executed IO event, the synchronization thread can process the IO event. When the IO event is an asynchronously executed IO event, the synchronization thread will hand over the asynchronously executed IO event to the corresponding asynchronous thread for processing.

[0160] In some embodiments, the current API gateway also starts a receiving thread, which is used to poll the received client's call request for the target service, and delegate the polled call request to the working thread. The working thread registers the IO event based on the call request and puts the IO event into the synchronization thread queue for the synchronization thread to perform call processing.

[0161] In some embodiments, whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event depends mainly on the business corresponding to the event, that is, the service requested by the client. In some embodiments, it can be determined based on the configuration in the business script corresponding to the service.

[0162] When a synchronous thread currently polls a synchronous IO event, it processes the synchronous IO event. When it currently polls an asynchronous IO event, it hands the event over to the corresponding asynchronous thread for processing. It should be noted that different services can correspond to different asynchronous threads. If the asynchronous thread corresponding to the service that the asynchronous IO event requires access to is not started, the synchronous thread can also start the asynchronous thread so that the asynchronous thread can process the asynchronous IO event.

[0163] After the asynchronous thread processes the asynchronously executed IO event, it puts the execution completion task of the asynchronously executed IO event into the synchronous thread queue, so that the synchronous thread polls the execution completion task from the synchronous thread queue and performs corresponding processing, such as returning the processing result to the client, or performing the next processing, etc.

[0164] In the embodiment of the present disclosure, when the API gateway is implemented, a synchronous thread is used to poll the IO events registered in the synchronous thread queue, and based on the script configuration written in advance for the service call, it is determined whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. When it is a synchronously executed IO event, the IO event is processed by the synchronous thread, and when it is an asynchronously executed IO event, the synchronous thread hands it over to the asynchronous thread for processing. This can avoid the business that requires a long time to process being blocked at the synchronous thread, resulting in slow system response speed and other problems. At the same time, the synchronous thread and the asynchronous thread are realized to work together by polling the queue, the asynchronous thread obeys the synchronous thread's allocation, and the asynchronous thread returns the processing result to the queue, thereby realizing an efficient API gateway.

[0165] The API gateway can be implemented in the cluster nodes, that is, the API gateway can be started on multiple nodes. The client can send the call request of the target service to any API gateway node in the cluster through the cluster management node, and the API gateway on the node will process the call request.

[0166] A central server can be elected in the cluster where multiple API gateways are located, and the central server will centrally manage and allocate resources, such as limiting traffic.

[0167] The current central server and the next central server can be understood as the primary and backup servers, with the current central server being the primary server and the next central server being the backup server. A long link can be established between the current central server and the next central server, and the central server will synchronize the relevant information about resource allocation performed on it to the next central server, so that when the current central server is unable to provide services normally, the next central server will switch to the current central server. It should be noted that the current central server, the next central server, and other API gateways, as members of the API gateway, also provide normal API gateway services. When locked resource allocation is required, they all request locks from the central server, and the central server will make the corresponding lock resource allocation.

[0168] In the embodiment of the present disclosure, a self-election strategy is adopted for the central server, and the first node that requests to register as an API gateway to the control server is determined as the current central server, and the second node that requests to register as an API gateway to the control server is determined as the next central server, while other API gateways are ordinary API gateways. The control server can determine the node where the API gateway is located as the current central server and the next central server based on the registration order of the API gateway, and then send information to the corresponding API gateway on whether it is the current central server. If it is not the current central server, it can also return relevant information of the current central server to the API gateway, so that the API gateway that is not the central server can request the current central server to allocate lock resources based on the relevant information. It can be understood that lock resources are resources that need to be uniformly allocated to multiple API gateways, and need to be allocated by locking.

[0169] In addition, the disclosed embodiments implement a cluster of API gateways and a simple and efficient centralized resource management model, thereby improving exception handling capabilities and the service efficiency of the clustered API gateways.

[0170] Figure 3 The technical terms and technical features involved in the embodiments shown and related Figure 1-Figure 2 The technical terms and technical features mentioned in the embodiments shown and related are the same or similar. Figure 2 The explanation and description of the technical terms and technical features involved in the embodiments shown and related can refer to the above Figure 1-Figure 2 The explanations of the illustrated and related embodiments will not be repeated here.

[0171] Figure 4 The following is a schematic diagram showing the design architecture of the API gateway according to one embodiment of the present disclosure. Figure 4As shown, the core of the API gateway consists of a CPU core that launches a synchronous IO thread. Received external requests are polled by an accept thread, which then delegates them to a worker thread for processing. The worker thread registers IO events based on external requests and places them in the synchronous thread queue. The IO thread then polls the synchronous thread queue and processes the polled IO events accordingly. When the IO thread is first started, the first script placed in the synchronous thread queue is the entry point script. This script parses the data in the external request and retrieves the IP address of the requester. Authentication verifies that the external request is authorized. Full rate limiting implements rate limiting. Business scripts are scripts that specifically call service interfaces and can be handled by asynchronous threads. Conversion and routing determine which cluster to send external requests to, including IDCs, Alibaba Cloud, and containers. Return processing returns the request results to the client. These are all functions implemented by the API gateway. Each API gateway must register with the management server. Relevant personnel can also configure the API gateway through the configuration center, and configuration information can be stored in a database for persistence.

[0172] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein.

[0173] Figure 5 The structure block diagram of the backend service calling device according to one embodiment of the present disclosure is shown. The device can be implemented as part or all of the electronic device through software, hardware or a combination of both. Figure 5 As shown, the backend service calling device includes:

[0174] The first receiving module 501 is configured to receive a call request from a client to a target service;

[0175] The registration module 502 is configured to register the call request of the target service as an IO event and place it in the synchronization thread queue corresponding to the CPU core; the CPU core is located at the node where the current API gateway is located;

[0176] The polling module 503 is configured to use the synchronization thread started on the CPU core to poll the IO events in the synchronization thread queue, and determine whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0177] The execution module 504 is configured to, when the IO event currently polled is a synchronously executed IO event, have the synchronous thread process the synchronously executed IO event, and when the IO event currently polled is an asynchronously executed IO event, have the synchronous thread send the asynchronously executed IO event to the corresponding asynchronous thread for processing, and put the asynchronously executed IO event executed by the asynchronous thread back into the synchronous thread queue.

[0178] Figure 6 The following is a structural block diagram of a multi-node deployment device of an API gateway according to an embodiment of the present disclosure. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 6 As shown, the multi-node deployment device of the API gateway includes:

[0179] The second receiving module 601 is configured to receive one or more API gateway registration requests based on a synchronous locking method;

[0180] A first determining module 602 is configured to determine the API gateway that first requests registration as the current central server;

[0181] The second determining module 603 is configured to determine the API gateway that requested the second registration as the next central server; the next central server is switched to the current central server after an exception occurs in the current central server;

[0182] After successful registration, the API gateway is implemented as follows:

[0183] Receive the client's call request for the target service;

[0184] Register the target service call request as an IO event and put it into the synchronous thread queue corresponding to the CPU core;

[0185] Using a synchronous thread to poll the IO events in the synchronous thread queue, and determining whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event;

[0186] When the IO event currently polled is an IO event for synchronous execution, the IO event for synchronous execution is executed in the synchronous thread. When the IO event currently polled is an IO event for asynchronous execution, the IO event for asynchronous execution is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is put back into the synchronous thread queue.

[0187] The technical features involved in the above-mentioned device embodiments and their corresponding explanations and descriptions are the same, corresponding or similar to the technical features involved in the method embodiments described above and their corresponding explanations and descriptions. For the technical features involved in the above-mentioned device embodiments and their corresponding explanations and descriptions, reference may be made to the technical features involved in the above-mentioned method embodiments and their corresponding explanations and descriptions, and this disclosure will not repeat them here.

[0188] Figure 7 The following is a block diagram of the structure of an API gateway cluster system according to an embodiment of the present disclosure. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 7 As shown, the API gateway cluster system includes: multiple API gateways 701 and a management and control server 702;

[0189] The control server 702 receives registration requests from one or more API gateways 701 based on synchronous locking;

[0190] The control server 702 determines the API gateway 701 that first requests registration as the current central server;

[0191] The control server 702 determines the API gateway 701 that requested the second registration as the next central server; after an exception occurs in the current central server, the next central server is switched to the current central server;

[0192] After successful registration, the API gateway 701 receives the client's call request for the target service, registers the call request for the target service as an IO event, and puts it into the synchronous thread queue corresponding to the CPU core, and uses the synchronous thread to poll the IO events in the synchronous thread queue, and determines whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. After the currently polled IO event is a synchronously executed IO event, the synchronously executed IO event is executed in the synchronous thread, and after the currently polled IO event is an asynchronously executed IO event, the asynchronously executed IO event is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event completed by the asynchronous thread is placed back into the synchronous thread queue.

[0193] The present disclosure also discloses an electronic device, which includes a memory and a processor; wherein:

[0194] The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement any of the above method steps.

[0195] Figure 8It is a structural diagram of a computer system suitable for implementing a backend service calling method and / or multi-node deployment of an API gateway according to an embodiment of the present disclosure.

[0196] like Figure 8 As shown, the computer system 800 includes a processing unit 801, which can execute various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the computer system 800 are also stored in the RAM 803. The processing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0197] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom can be installed into the storage section 808 as needed. Among them, the processing unit 801 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0198] In particular, according to embodiments of the present disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for executing the data transmission method. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion 809 and / or installed from a removable medium 811.

[0199] The embodiments of the present disclosure further disclose a computer program product, which includes a computer program / instruction, and implements any of the above method steps when the computer program / instruction is executed by a processor.

[0200] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0201] The units or modules described in the embodiments of the present disclosure may be implemented in software or hardware. The units or modules described may also be provided in a processor, and the names of these units or modules do not, in certain circumstances, limit the units or modules themselves.

[0202] As another aspect, embodiments of the present disclosure further provide a computer-readable storage medium. This computer-readable storage medium may be included in the apparatus described in the above embodiments, or may be a standalone computer-readable storage medium not incorporated into the apparatus. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the methods described in the embodiments of the present disclosure.

[0203] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A backend service calling method, wherein: The method is executed on the current API gateway and includes: Receive the client's call request for the target service; Registering the target service call request as an IO event and placing it in the synchronization thread queue corresponding to the CPU core; the CPU core is located at the node where the current API gateway is located; Utilize the synchronization thread started on the CPU core to poll the IO events in the synchronization thread queue, and determine whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event; When the IO event currently polled is a synchronously executed IO event, the synchronous thread processes the synchronously executed IO event. When the IO event currently polled is an asynchronously executed IO event, the synchronous thread sends the asynchronously executed IO event to the corresponding asynchronous thread for processing. The asynchronously executed IO event completed by the asynchronous thread is put back into the synchronous thread queue.

2. The method according to claim 1, wherein The method further comprises: Receive registration requests for at least one or more target services; the registration requests include a business script corresponding to the target services; the business script is configured with a service identifier and a service address of the target service, a calling method of the target service, and an identifier of whether the target service corresponds to synchronously executed IO events or asynchronously executed IO events; The business script of the target service is stored in a specified directory so that the current API gateway can support the client's call to the newly registered target service.

3. The method according to claim 2, wherein: The synchronization thread determines whether the IO event is a synchronously executed IO event or an asynchronously executed IO event based on a business script of a target service corresponding to the IO event.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: After the current API gateway is started, a registration request is submitted to the management and control server; the registration request includes the address information of the current API gateway; Receive the registration confirmation information returned by the control server, wherein the registration confirmation information includes information on whether the node where the current API gateway is located is registered as the current central server.

5. The method according to claim 4, wherein When the node where the current API gateway is located is the first API gateway registered with the control server, the current API gateway is determined as the current central server. When the node where the current API gateway is located is the second API gateway registered with the control server, the current API gateway is determined as the next central server. After an exception occurs in the current central server, the next central server is switched to the current central server.

6. The method according to claim 4, wherein: When the node where the current API gateway is located is the current central server, the method further includes: Receive distributed lock resource requests from other nodes; other distributed API gateways are running on the other nodes; The requested lock resource is allocated based on the distributed lock resource request, and the allocated lock resource information is synchronized to the next central server; the next central server is switched to the current central server after an exception occurs in the current central server.

7. The method according to claim 4, wherein: When the node where the current API gateway is located is not the current central server, the method further includes: In response to a demand event for a distributed lock resource, a distributed lock resource request is sent to the current central server so that the current central server allocates the requested lock resource to the current API gateway.

8. The method according to any one of claims 1-3, 5-7, wherein The number of synchronization threads started on the current API gateway is related to the number of CPU cores on the node where the current API gateway is located.

9. A multi-node deployment method for an API gateway, the method being executed on a management and control server, comprising: Receive registration requests from one or more API gateways based on synchronous locking; Determine the API gateway that first requests registration as the current central server; Determine the API gateway that requested the second registration as the next central server; The next central server is switched to the current central server after an exception occurs in the current central server; After the API gateway is successfully registered, it performs the following steps: Receive the client's call request for the target service; Register the target service call request as an IO event and put it into the synchronous thread queue corresponding to the CPU core; Using a synchronous thread to poll the IO events in the synchronous thread queue, and determining whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event; When the IO event currently polled is an IO event for synchronous execution, the IO event for synchronous execution is executed in the synchronous thread. When the IO event currently polled is an IO event for asynchronous execution, the IO event for asynchronous execution is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is put back into the synchronous thread queue.

10. The method according to claim 9, wherein: The method further comprises: In response to the abnormal restart completion event, the existing local API gateway registration information is cleared; the API gateway registration information includes information of the current central server and the next central server; Update the local API gateway registration information based on the actual current central server information and the next central server information.

11. The method according to claim 9 or 10, wherein: The method further comprises: Receive at least one or more registration requests for target services; the registration requests include a service script corresponding to the target services; The business script is configured with the service identifier of the target service, the service address, the calling method of the target service, and an identifier of whether the target service corresponds to a synchronously executed IO event or an asynchronously executed IO event; The business script of the target service is stored in a specified directory so that the current API gateway can support the client's call to the newly registered target service.

12. The method according to claim 11, wherein The synchronization thread determines whether the IO event is a synchronously executed IO event or an asynchronously executed IO event based on a business script of a target service corresponding to the IO event.

13. The method according to any one of claims 9 to 10 or 12, wherein: When the node where the current API gateway is located is the current central server, the method further includes: Receive distributed lock resource requests from other nodes; other distributed API gateways are running on the other nodes; The requested lock resource is allocated based on the distributed lock resource request, and the allocated lock resource information is synchronized to the next central server; the next central server is switched to the current central server after an exception occurs in the current central server.

14. The method according to any one of claims 9 to 10 or 12, wherein: When the node where the current API gateway is located is not the current central server, the method further includes: In response to a demand event for a distributed lock resource, a distributed lock resource request is sent to the current central server so that the current central server allocates the requested lock resource to the current API gateway.

15. The method according to any one of claims 9 to 10 or 12, wherein: The number of synchronization threads started on the current API gateway is related to the number of CPU cores on the node where the current API gateway is located.

16. A backend service invocation method, the method being executed on an API gateway cluster system, the API gateway cluster system comprising multiple API gateways and a management and control server; comprising: The control server receives registration requests from one or more API gateways based on synchronous locking; The control server determines the API gateway that first requests registration as the current central server; The control server determines the API gateway that requested the second registration as the next central server; The next central server is switched to the current central server after an exception occurs in the current central server; After successful registration, the API gateway receives the client's call request for the target service, registers the call request for the target service as an IO event, and puts it into the synchronous thread queue corresponding to the CPU core, and uses the synchronous thread to poll the IO events in the synchronous thread queue, and determines whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. After the currently polled IO event is a synchronously executed IO event, the synchronously executed IO event is executed in the synchronous thread, and after the currently polled IO event is an asynchronously executed IO event, the asynchronously executed IO event is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is placed back into the synchronous thread queue.

17. A backend service calling device, wherein: The device is executed on the current API gateway and includes: A first receiving module is configured to receive a call request from a client to a target service; A first registration module is configured to register the call request of the target service as an IO event and place it in the synchronization thread queue corresponding to the CPU core; the CPU core is located at the node where the current API gateway is located; A first polling module is configured to use the synchronization thread started on the CPU core to poll the IO events in the synchronization thread queue, and determine whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event; The first execution module is configured to, when the IO event currently polled is a synchronously executed IO event, process the synchronously executed IO event by the synchronous thread; and when the IO event currently polled is an asynchronously executed IO event, send the asynchronously executed IO event to the corresponding asynchronous thread for processing by the synchronous thread, and put the asynchronously executed IO event executed by the asynchronous thread back into the synchronous thread queue.

18. A multi-node deployment device for an API gateway, the device being executed on a management and control server, comprising: A second receiving module is configured to receive registration requests from one or more API gateways based on a synchronous locking method; A first determining module is configured to determine the API gateway that first requests registration as the current central server; A second determining module is configured to determine the API gateway that makes the second registration request as the next central server; The next central server is switched to the current central server after an exception occurs in the current central server; After successful registration, the API gateway is implemented as the following modules: A third receiving module is configured to receive a call request from a client to a target service; The second registration module is configured to register the call request of the target service as an IO event and put it into the synchronization thread queue corresponding to the CPU core; The second polling module is configured to use the synchronous thread to poll the IO events in the synchronous thread queue, and determine whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event; The second execution module is configured to execute the synchronously executed IO event in the synchronous thread after the currently polled IO event is a synchronously executed IO event, and to send the asynchronously executed IO event to the corresponding asynchronous thread for processing after the currently polled IO event is an asynchronously executed IO event, and put the asynchronously executed IO event executed by the asynchronous thread back into the synchronous thread queue.

19. An API gateway cluster system, including multiple API gateways and a management and control server; The control server receives registration requests from one or more API gateways based on synchronous locking; The control server determines the API gateway that first requests registration as the current central server; The control server determines the API gateway that requested the second registration as the next central server; The next central server is switched to the current central server after an exception occurs in the current central server; After successful registration, the API gateway receives the client's call request for the target service, registers the call request for the target service as an IO event, and puts it into the synchronous thread queue corresponding to the CPU core, and uses the synchronous thread to poll the IO events in the synchronous thread queue, and determines whether the currently polled IO event is a synchronously executed IO event or an asynchronously executed IO event. After the currently polled IO event is a synchronously executed IO event, the synchronously executed IO event is executed in the synchronous thread, and after the currently polled IO event is an asynchronously executed IO event, the asynchronously executed IO event is sent to the corresponding asynchronous thread for processing, and the asynchronously executed IO event that has been executed by the asynchronous thread is placed back into the synchronous thread queue.

20. An electronic device comprising a memory and a processor; wherein: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 16.

21. A computer-readable storage medium having computer instructions stored thereon, wherein: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

22. A computer program product comprising a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 16 when executed by a processor.

Images