Request scheduling method, device and equipment based on API gateway

Abstract

The invention provides a request scheduling method, device and equipment based on an API gateway, and relates to the field of gateway technology and Internet of Things in computer technology, in particular to the field of the request scheduling method, device and equipment based on the API gateway. According to the specific implementation scheme, when it is determined that a response to a received user request fails, request processing information of a back-end service corresponding to a current API gateway is determined; the request processing information represents the processing condition of the back-end service for processing the received user request within a preset time period; determining a service state value of the back-end service according to the request processing information of the back-end service; the service state value represents the user request processing capability of the back-end service; determining a target back-end service from back-end services corresponding to the current API gateway according to the service state value; and sending the user request to the target back-end service for response processing. And thus, the user request can be effectively ensured to be responded by the back-end service.

Description

Technical Field

[0001] The present disclosure relates to gateway technology and the Internet of Things in computer technology, and in particular to a request scheduling method, device and equipment based on an API gateway. Background Art

[0002] The Application Programming Interface Gateway (API Gateway) can be used as a request entry point to receive user requests sent by the client; the API network then sends the user request to the backend service for response. There may be situations where the user request cannot be properly responded to and processed by the backend service.

[0003] Furthermore, there is an urgent need for a solution that can effectively ensure that user requests can be responded to by backend services. Summary of the invention

[0004] The present disclosure provides a request scheduling method, apparatus and device based on an API gateway for effectively ensuring that user requests can be responded to by backend services.

[0005] According to the first aspect of the present disclosure, a request scheduling method based on an API gateway is provided, including: when it is determined that the response to a received user request fails, determining the request processing information of the backend service corresponding to the current API gateway; wherein the user request is used to indicate the call of the backend service; the request processing information represents the processing of the received user request by the backend service within a preset time period; according to the request processing information of the backend service, determining the service status value of the backend service; wherein the service status value represents the ability of the backend service to process the user request; according to the service status value, determining the target backend service from the backend services corresponding to the current API gateway; and sending the user request to the target backend service for response processing.

[0006] According to a second aspect of the present disclosure, a request scheduling device based on an API gateway is provided, including: a first determination unit, for determining the request processing information of a backend service corresponding to the current API gateway when it is determined that the response to a received user request has failed; wherein the user request is used to indicate the call to the backend service; the request processing information represents the processing status of the backend service for the received user request within a preset time period; a second determination unit, for determining the service status value of the backend service according to the request processing information of the backend service; wherein the service status value represents the ability of the backend service to process the user request; a third determination unit, for determining a target backend service from the backend services corresponding to the current API gateway according to the service status value; and sending the user request to the target backend service for response processing.

[0007] According to a third aspect of the present disclosure, an API gateway is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 4.

[0008] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an API gateway can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the API gateway executes the method described in the first aspect.

[0009] According to the technology disclosed in the present invention, when it is determined that the response to the received user request has failed, the information of the response failure is not directly passed to the user, but the backend service corresponding to the current API gateway is scored based on the ability to process the user request, and each backend service status value is obtained. According to the service status value, the target backend service is determined from the backend services corresponding to the current API gateway. By selecting the backend based on the service status value, the API gateway can process requests more efficiently and reduce request failures caused by backend service problems, thereby improving the user experience. The retry mechanism increases the possibility of request success. Even in the case of an initial request failure, there is a chance to restore the service through retries, which improves the fault tolerance of the system. This ensures that user requests can be responded to by the backend service.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a scene graph according to an embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a first embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a second embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0017] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0018] Figure 7 A schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] Figure 1 is a scene graph according to an embodiment of the present disclosure, Figure 1 As shown in the figure, API Gateway is an API managed service that provides developers with management of the entire life cycle of API creation, maintenance, publishing, and monitoring. Through the API Gateway, various backend services can be encapsulated and provided to all parties in the form of APIs. The API Gateway can serve as an entry point for request traffic, receiving client requests and routing them to the corresponding services. It acts as a "portal" between the front-end and back-end microservices, coordinating the request traffic and service access of the entire microservice system.

[0021] In the related field, the application programming interface gateway 102, in the process of forwarding the request of the client 101, generally selects the backend service 103 by round-robin or by weight, and does not have a retry function after forwarding (if it has a retry capability, it will only retry once for anomalies at the network connection level, and will not retry specific business processes). Therefore, if the load of the backend instance forwarded to has reached the upper limit, it will respond to the user with errors related to the backend service being temporarily unavailable or the request exceeding the limit.

[0022] The present invention provides a request scheduling method, device and equipment based on an API gateway, which are applied to gateway technology and the Internet of Things in computer technology to effectively ensure that user requests can be responded to by backend services.

[0023] In order to enable readers to more deeply understand the implementation principle of the present disclosure, the following Figure 2-Figure 7 right Figure 2 The illustrated embodiment is further refined.

[0024] Figure 2 is a schematic diagram according to the first embodiment of the present disclosure, such as Figure 2 As shown, the present disclosure provides a request scheduling method based on an API gateway, the method comprising:

[0025] 201. When it is determined that the response to the received user request fails, determine the request processing information of the backend service corresponding to the current API gateway; wherein the user request is used to indicate the call of the backend service; the request processing information represents the processing status of the backend service for the received user request within a preset time period.

[0026] Exemplarily, the execution entity of this embodiment may be an API gateway.

[0027] In a modern microservices architecture, the API gateway acts as an intermediary between the client and the backend service. When the API gateway receives a user request and finds that the response has failed, it needs to decide how to reprocess the request.

[0028] If a response is received from the backend service that is processed based on the user request, the response is determined to be successful. If a response is received from the backend service that is unprocessable based on the user request, the response to the user request is determined to be unsuccessful.

[0029] 202. Determine a service status value of the backend service according to the request processing information of the backend service; wherein the service status value represents the ability of the backend service to process the user request.

[0030] Exemplarily, the service status value of this embodiment may be a score calculated based on the request processing information.

[0031] When determining that a user request response fails, the API gateway needs to evaluate the processing capacity and status of the currently connected backend services. To this end, the API gateway collects and analyzes the request processing information of these backend services within a preset period of time.

[0032] 203. According to the service status value, determine the target backend service from the backend services corresponding to the current API gateway; and send the user request to the target backend service for response processing.

[0033] The API Gateway uses the service status value to evaluate and select the best backend service. This selection mechanism ensures that requests can be processed quickly and reliably.

[0034] The most appropriate backend service is selected for retry by calculating the "score" (i.e., service status value) of the backend service.

[0035] In this embodiment, when it is determined that the response to the received user request has failed, the information of the response failure is not directly passed to the user, but the backend service corresponding to the current API gateway is scored based on the ability to process the user request, and the status value of each backend service is obtained. According to the service status value, the target backend service is determined from the backend services corresponding to the current API gateway. By selecting the backend according to the service status value, the API gateway can process requests more efficiently and reduce request failures caused by backend service problems, thereby improving the user experience. The retry mechanism increases the possibility of request success. Even in the case of an initial request failure, there is a chance to restore the service by retrying, which improves the fault tolerance of the system. This ensures that the user request can be responded to by the backend service.

[0036] Figure 3 is a schematic diagram according to the second embodiment of the present disclosure, such as Figure 3 As shown, the present disclosure provides a request scheduling method based on an API gateway, the method comprising:

[0037] 301. When it is determined that the response to the received user request fails, determine the request error rate of the backend service corresponding to the current API gateway; wherein the user request is used to instruct to call the backend service.

[0038] In one example, when it is determined that the response to the received user request fails, the request processing information of the backend service corresponding to the current API gateway is determined to represent the processing of the received user request by the backend service within a preset period of time. The request processing information includes one or more of the following: request error rate, time ranking ratio, and request number ratio.

[0039] Among them, the request error rate represents the proportion of failed responses to user requests by the backend service within a preset time period; the time ranking ratio represents the ranking ratio of the average time taken for the backend service to successfully respond to user requests within a preset time period; the request number ratio represents the ratio of the number of user requests received by the backend service within a preset time period.

[0040] These three indicators reflect the performance and health of the backend service from different dimensions. The request error rate focuses on the stability of the service, the time ranking ratio focuses on the response speed of the service, and the request number ratio focuses on the load of the service. By comprehensively considering these indicators, the status of the backend service can be more comprehensively evaluated.

[0041] In one example, step 301 includes:

[0042] The first step of step 301 is to determine the number of failed responses to user requests by the backend service within a preset time period, which is the first number; and to determine the total number of failed responses to user requests by each backend service corresponding to the current API gateway within the preset time period, which is the second number.

[0043] The second step of step 301 is to determine the ratio of the first number to the second number, which is the request error rate of the backend service.

[0044] In one example, the first step of step 301 includes: obtaining the number of user requests received by the backend service within a preset time period, which is the first total number; and obtaining the number of user requests successfully responded to by the backend service within the preset time period, which is the first total response number; and obtaining the number of user requests that were not transmitted to the backend service within the preset time period, which is the first total transmission number; determining the difference between the first total number, the first total response number, and the first total transmission number, which is the first number.

[0045] In one example, the second step of step 301 includes: obtaining the total number of user requests received by each backend service within a preset time period, which is the second total number; and obtaining the total number of user requests successfully responded to by each backend service within the preset time period, which is the second total response number; and obtaining the total number of user requests that were not transmitted to each backend service within the preset time period, which is the second total transmission number; subtracting the second total response number from the second total number to obtain a second difference, and subtracting the second total transmission number from the second difference to obtain the second number.

[0046] Exemplarily, the execution entity of this embodiment may be an API gateway.

[0047] The API gateway records the total number (first total number) of all user requests received by each backend service within a preset time period (eg, the past 5 minutes).

[0048] You can use logging or real-time monitoring tools to monitor or record the total number of user requests received by the backend service within a preset period of time. For example, use an agent to collect logs into a time series database.

[0049] The API Gateway counts the number of user requests that each backend service successfully responded to in the same period (the total number of first responses). This data can be obtained by analyzing the HTTP status code of the request. For example, a status code of 2xx indicates a successful status code, indicating that the request was processed normally.

[0050] In some cases, the request may fail to reach the backend service due to network problems or other reasons. For example, the status code 4xx indicates a client error status code, indicating that the server cannot process the request. The API gateway needs to record the number of these unsuccessful transmission requests (the total number of first transmissions).

[0051] By calculating the first total number minus the first total number of responses and the first total number of transmissions, the API gateway can obtain the number of user requests to which the backend service failed to respond within a preset time period, that is, the first number.

[0052] That is, for each backend service, the API Gateway counts the number of all non-2xx and non-4xx status codes returned during the preset period. This number reflects the number of errors that occurred in the backend service during this period.

[0053] Then, the API gateway summarizes the total number of user requests received by all backend services within a preset time period (a second total number).

[0054] The API Gateway counts the total number of user requests that all backend services successfully responded to in the same period (the total number of second responses)

[0055] The API Gateway records the total number of unsuccessful requests for all backend services during the same period (the second total number of transmissions).

[0056] By calculating the second total number minus the second total number of responses and the second total number of transmissions, the API gateway can obtain the total number of user requests to which all backend services failed to respond within a preset time period, that is, the second number.

[0057] API Gateway aggregates the total number of all non-2xx and non-4xx status codes returned by all backend services during the same period. This total represents the error load across the entire backend service.

[0058] Finally, the API Gateway calculates the request error rate of the backend service by dividing the first number (the number of failures of a single backend service) by the second number (the total number of failures of all backend services). The API Gateway can obtain the request error rate of the backend service by dividing the number of non-2xx, 4xx status codes of a single backend address by the total number of non-2xx, 4xx status codes of all backend addresses. The higher this ratio is, the higher the error rate of the backend service is.

[0059] By monitoring the request error rate of backend services in real time, problems in the service can be quickly discovered and handled to reduce the impact on user experience. Based on the calculation results of the error rate, the system can dynamically adjust resource allocation to optimize overall performance.

[0060] 302. Determine the time-consuming ranking ratio of the backend service corresponding to the current API gateway.

[0061] In one example, step 302 includes:

[0062] The first step of step 302 is to determine the ranking value of the backend service in the preset set according to the average time consumption value of the backend service; wherein the average time consumption value represents the average time consumption of the backend service in responding to the user request within the preset time period; the preset set includes each backend service within the geographical scope to which the backend service belongs.

[0063] The second step of step 302 is to determine the ratio between the ranking value of the backend service and the backend quantity value, which is the time-consuming ranking ratio of the backend service; wherein the backend quantity value is the total number of backend services in the preset set.

[0064] In one example, the first step of step 302 includes: determining the average time consumption value of the backend services in the preset set; sorting the backend services in the preset set in order from low to high according to the average time consumption value to obtain a sorted preset set; determining the ranking order of the backend services in the sorted preset set, which is the ranking value of the backend services.

[0065] For example, the average time consumption of the backend services in the preset set can be determined by recording the response time of each request and then calculating the average value. All backend services are sorted from low to high according to their average time consumption values. In this way, the service with the lowest average time consumption will be ranked first, and the service with the highest average time consumption will be ranked last.

[0066] For each backend service, the API Gateway determines its position in the sorted preset set, which is the ranking value of the service. The smaller the ranking value, the lower the average time consumption of the service and the better the performance.

[0067] The API Gateway counts the total number of backend services in the preset collection (backend quantity value). By dividing the ranking value of a single backend service by the backend quantity value, the API Gateway can obtain the time-consuming ranking ratio of the backend service. The lower this ratio is, the higher the performance ranking of the service in the collection.

[0068] By sorting and ranking, the ranking value can be used as the basis for resource allocation. The best and worst performing backend services can be intuitively identified. This helps prioritize the optimization of services with poor performance, thereby improving the efficiency of the entire system.

[0069] For example, within one hour, there are five backend services in Beijing, among which the average time consumption of backend service A is 100 milliseconds. Their average times are: 65 milliseconds, 82 milliseconds, 100 milliseconds, 113 milliseconds, and 135 milliseconds respectively.

[0070] The ranking value of backend service A is 3. The time ranking ratio of backend service A is:

[0071]

[0072] A higher ratio means that the backend address takes more time.

[0073] 303. Determine the ratio of the number of requests to the backend service corresponding to the current API gateway.

[0074] In one example, step 302 includes: determining the number of user requests received by the backend service within a preset time period, which is a third number; and determining the total number of user requests received by each backend service corresponding to the current API gateway within the preset time period, which is a fourth number; determining the ratio of the third number to the fourth number, which is the request number ratio of the backend service.

[0075] Exemplarily, by dividing the number of user requests received by a single backend service in a preset period (the third number) by the total number of user requests received by all backend services in the same period (the fourth number), the API gateway can obtain the request number ratio of the backend service. The higher the ratio, the more requests the service has processed.

[0076] The request ratio can provide an intuitive understanding of the efficiency and load of backend services in processing user requests, which helps optimize resource allocation and ensure efficient system operation.

[0077] For example, backend service A received 400 user requests, and all backend services corresponding to the current API gateway received a total of 2,000 user requests. Then the request ratio of backend service A is:

[0078]

[0079] This means that backend service A handles 20% of the requests in the system.

[0080] 304. Determine a service status value of the backend service according to the request processing information of the backend service; wherein the service status value represents the ability of the backend service to process the user request.

[0081] In one example, step 304 includes: determining the sum of the values ​​of each information included in the request processing information as the service status value of the backend service; or, performing weighted summation on the values ​​of each information included in the request processing information to obtain the service status value of the backend service.

[0082] Exemplarily, the API gateway maintains a list of all backend services and their service status values. When receiving a user request, the API gateway selects the target backend service based on the preset service status value. Once the target backend service is determined, the API gateway forwards the user request to the backend service for processing. After the backend service processes the request, it returns the response to the API gateway, which then passes the response to the user.

[0083] For example, there are three backend services A, B, and C, and their service status values ​​are 90, 80, and 70 respectively.

[0084] The API gateway receives a user request. According to the preset routing policy, the API gateway selects the backend service with the lowest service status value as the target backend service. Since the service status value of backend service C is the highest, it is selected as the target backend service.

[0085] On the one hand, through dynamic routing decisions based on service status values, the API gateway can ensure that requests are routed to the most appropriate backend service at the moment, thereby improving the overall performance and stability of the system.

[0086] On the other hand, routing based on service status values ​​can ensure that requests are more often routed to backend services with stronger processing capabilities, avoiding situations where some backend services are overloaded while other backend services are idle, and effectively improving the utilization of backend services.

[0087] 305. According to the service status value, determine the target backend service from the backend services corresponding to the current API gateway; and send the user request to the target backend service for response processing.

[0088] 306. If it is determined that the number of times that the backend service fails to respond to the user request within the preset time period is greater than the first preset number, the backend service is subjected to fuse processing.

[0089] For example, the API gateway continuously monitors the number of failed requests for each backend service. If a backend service fails to respond to user requests within a preset time period (such as within 1 hour) for a number of times that exceeds a preset threshold (such as 5 consecutive failures), it is considered that there may be a problem with the backend service. Once it is determined that a circuit breaker is required, the API gateway will temporarily stop routing new user requests to the problematic backend service.

[0090] After a period of time (for example, 2 hours), the API Gateway automatically tries to resume requests to the backend service to detect whether the problem has been resolved.

[0091] If the backend service returns to normal, the API Gateway will continue to route requests to the service; if it still fails, the circuit breaker will continue to remain in the broken state. When a service is detected to be unavailable, the circuit breaker will immediately block subsequent requests to avoid unnecessary resource waste and waiting time.

[0092] The fuse mechanism can prevent the failure of a single service from causing a chain reaction in the entire system, thereby ensuring the stability of the system.

[0093] For example, there are three backend services A, B, and C. When the API network receives a user request, it forwards the user request to the backend service A according to the polling routing strategy.

[0094] This situation is detected, and it is determined that this exceeds the preset first preset number of times (for example, 5 times). The API gateway immediately performs a fuse process on the backend service A and stops sending new user requests to it. At the same time, the API gateway returns an abnormal response prompt to the user. After 5 minutes, the API gateway automatically attempts to restore the request to the backend service A. If the backend service A resumes normal response, the API gateway will continue to route the request to the service; if it still fails, it will continue to maintain the fuse state. Among them, the fuse process is used to deal with service call failure problems in distributed systems. When the downstream service becomes unavailable or responds too slowly for some reason, the upstream service will activate the fuse mechanism to temporarily cut off direct interaction with the downstream service, thereby avoiding resource waste and system instability caused by continuous requests.

[0095] 307. If it is determined that the number of times that the user request has not been responded to is greater than a second preset number, the user request is discarded.

[0096] Exemplarily, the API Gateway continuously monitors the status of each user request, including whether it is successfully processed by the backend service. If a user request is not processed by any backend service within a preset time (such as 30 seconds), the number of failures of the request is recorded. If the number of unresponsive processing times of a user request exceeds a preset threshold (such as 3 consecutive unprocessed times), it is considered that there may be a problem with the request or the backend service load is too high. The API Gateway will no longer attempt to route the request to any backend service and return a predefined error response to the user.

[0097] When a user request is not processed by the backend service for multiple consecutive times, the API gateway will quickly identify potential problems or high load conditions and take immediate measures to stop further request forwarding. This mechanism can respond to failures quickly, avoiding waste of system resources and degradation of user experience.

[0098] In this embodiment, by real-time monitoring of the request error rate of the backend service, problems in the service can be quickly discovered and handled to reduce the impact on the user experience. Based on the calculation results of the error rate, the system can dynamically adjust resource allocation to optimize overall performance. Through dynamic routing decisions based on service status values, the API gateway can ensure that requests are routed to the most appropriate backend service at the moment, thereby improving the overall performance and stability of the system.

[0099] Figure 4 is a schematic diagram according to the third embodiment of the present disclosure, Figure 4 As shown, the present disclosure provides a request scheduling device 400 based on an API gateway, including:

[0100] The first determination unit 401 is used to determine the request processing information of the backend service corresponding to the current API gateway when it is determined that the response to the received user request has failed; wherein the user request is used to indicate the call of the backend service; the request processing information represents the processing status of the backend service for the received user request within a preset time period.

[0101] The second determining unit 402 is used to determine a service status value of the backend service according to the request processing information of the backend service; wherein the service status value represents the ability of the backend service to process user requests.

[0102] The third determining unit 403 is used to determine the target backend service from the backend services corresponding to the current API gateway according to the service status value; and send the user request to the target backend service for response processing.

[0103] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0104] Figure 5is a schematic diagram according to a fourth embodiment of the present disclosure, Figure 5 As shown, the present disclosure provides a request scheduling device 500 based on an API gateway, including:

[0105] The first determination unit 501 is used to determine the request processing information of the backend service corresponding to the current API gateway when it is determined that the response to the received user request has failed; wherein the user request is used to indicate the call of the backend service; the request processing information represents the processing status of the backend service for the received user request within a preset time period.

[0106] The second determining unit 502 is used to determine a service status value of the backend service according to the request processing information of the backend service; wherein the service status value represents the ability of the backend service to process user requests.

[0107] The third determining unit 503 is used to determine the target backend service from the backend services corresponding to the current API gateway according to the service status value; and send the user request to the target backend service for response processing.

[0108] In one example, the request processing information includes one or more of the following: request error rate, time ranking ratio, and request number ratio.

[0109] Among them, the request error rate represents the proportion of failed responses to user requests by the backend service within a preset time period; the time ranking ratio represents the ranking ratio of the average time taken for the backend service to successfully respond to user requests within a preset time period; the request number ratio represents the ratio of the number of user requests received by the backend service within a preset time period.

[0110] In one example, when the request processing information includes a request error rate, the request error rate represents the proportion of failures of the backend service to respond to user requests within a preset period of time; the first determination unit 501 includes:

[0111] The first determination module 5011 is used to determine the number of failed responses to user requests by the backend service within a preset time period, which is a first number.

[0112] The second determination module 5012 is used to determine the total number of failed responses to user requests by each backend service corresponding to the current API gateway within a preset time period, which is the second number.

[0113] The third determination module 5013 is used to determine the ratio of the first number to the second number, which is the request error rate of the backend service.

[0114] In one example, the first determining module 5011 includes:

[0115] The first acquisition submodule 50111 is used to obtain the number of user requests received by the backend service within a preset time period, which is the first total number; and to obtain the number of user requests successfully responded to by the backend service within the preset time period, which is the first total number of responses; and to obtain the number of user requests that were not transmitted to the backend service within the preset time period, which is the first total number of transmissions.

[0116] The first determining submodule 50112 is configured to subtract the first total number of responses from the first total number to obtain a first difference, and to subtract the first total number of transmissions from the first difference to obtain the first number.

[0117] In one example, the second determining module 5012 includes:

[0118] The second acquisition submodule 50121 is used to obtain the total number of user requests received by each back-end service within a preset time period, which is the second total number; and to obtain the total number of user requests successfully responded to by each back-end service within the preset time period, which is the second total response number; and to obtain the total number of user requests that were not transmitted to each back-end service within the preset time period, which is the second total transmission number.

[0119] The second determining submodule 50122 is configured to subtract the second total number of responses from the second total number to obtain a second difference, and to subtract the second total number of transmissions from the second difference to obtain the second number.

[0120] In one example, when the request processing information includes a time ranking ratio, the time ranking ratio represents a ranking ratio of an average time taken by the backend service to successfully respond to a user request within a preset period of time; the first determination unit 501 includes:

[0121] The fourth determination module 5014 is used to determine the ranking value of the backend service in the preset set according to the average time consumption value of the backend service; wherein the average time consumption value represents the average time consumption of the backend service in responding to the user request within the preset time period; the preset set includes each backend service within the geographical scope to which the backend service belongs.

[0122] The fifth determination module 5015 is used to determine the ratio between the ranking value of the backend service and the backend quantity value, which is the time-consuming ranking ratio of the backend service; wherein the backend quantity value is the total number of backend services in the preset set.

[0123] In one example, the fourth determining module 5014 includes:

[0124] The third determining submodule 50141 is used to determine the average time consumption value of the backend services in the preset set.

[0125] The fourth determination submodule 50142 is used to sort the backend services in the preset set according to the average time consumption value from low to high to obtain a sorted preset set; determine the ranking order of the backend services in the sorted preset set as the ranking value of the backend services.

[0126] In one example, when the request processing information includes a request number ratio, the request number ratio represents the ratio of the number of user requests received by the backend service within a preset period of time; the first determining unit 501 includes:

[0127] The sixth determination module 5016 is used to determine the number of user requests received by the backend service within a preset time period, which is the third number; and determine the total number of user requests received by each backend service corresponding to the current API gateway within the preset time period, which is the fourth number; determine the ratio of the third number to the fourth number, which is the request number ratio of the backend service.

[0128] In one example, the second determining unit 502 is specifically configured to:

[0129] The sum of the values ​​of each information included in the request processing information is determined as the service status value of the backend service.

[0130] Alternatively, a weighted sum is performed on the values ​​of each information included in the request processing information to obtain the service status value of the backend service.

[0131] In one example, the present disclosure provides a request scheduling device 500 based on an API gateway, further comprising:

[0132] The first processing unit 504 is used to perform a fuse process on the backend service if it is determined that the number of times the backend service fails to respond to the user request within a preset time period is greater than a first preset number.

[0133] Alternatively, the second processing unit 505 is configured to discard the user request if it is determined that the number of times the user request has not been responded to is greater than a second preset number of times.

[0134] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0135] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure, Figure 6 As shown, the API gateway 600 in this embodiment may include: a processor 601 and a memory 602 .

[0136] The memory 602 is used to store programs. The memory 602 may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc. The memory may also include non-volatile memory, such as flash memory. The memory 602 is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc. The above computer programs, computer instructions, etc. may be partitioned and stored in one or more memories 602. And the above computer programs, computer instructions, data, etc. may be called by the processor 601.

[0137] The above-mentioned computer programs, computer instructions, etc. may be stored in partitions in one or more memories 602 . And the above-mentioned computer programs, computer instructions, etc. may be called by the processor 601 .

[0138] The processor 601 is used to execute the computer program stored in the memory 602 to implement each step of the method involved in the above embodiment.

[0139] For details, please refer to the relevant description in the previous method embodiment.

[0140] The processor 601 and the memory 602 may be independent structures or integrated structures. When the processor 601 and the memory 602 are independent structures, the memory 602 and the processor 601 may be coupled and connected via a bus 603 .

[0141] The electronic device of this embodiment can execute the technical solution in the above method, and its specific implementation process and technical principle are the same, which will not be repeated here.

[0142] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0143] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0144] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0145] Among them, the electronic device 700 can be an API gateway.

[0146] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0147] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0148] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a request scheduling method based on an API gateway. For example, in some embodiments, the request scheduling method based on an API gateway may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the request scheduling method based on the API gateway described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the API gateway-based request scheduling method in any other appropriate manner (eg, by means of firmware).

[0149] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0151] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0153] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0154] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0155] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0156] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A request scheduling method based on API gateway, comprising: When it is determined that the response to the received user request fails, determine the request processing information of the backend service corresponding to the current API gateway; wherein the user request is used to indicate the call of the backend service; the request processing information represents the processing of the received user request by the backend service within a preset time period; Determine the service status value of the backend service according to the request processing information of the backend service; wherein the service status value represents the ability of the backend service to process user requests; According to the service status value, a target backend service is determined from the backend services corresponding to the current API gateway; and the user request is sent to the target backend service for response processing.

2. The method according to claim 1, wherein: The request processing information includes one or more of the following: request error rate, time-consuming ranking ratio, and request number ratio; Among them, the request error rate represents the proportion of failed responses to user requests by the backend service within a preset time period; the time ranking ratio represents the ranking ratio of the average time taken for the backend service to successfully respond to user requests within a preset time period; and the request number ratio represents the ratio of the number of user requests received by the backend service within a preset time period.

3. The method according to claim 1 or 2, wherein: When the request processing information includes a request error rate, the request error rate represents the proportion of failures of the backend service to respond to user requests within a preset period of time; Determine the request error rate of the backend service corresponding to the current API gateway, including: Determine the number of failed responses to user requests by the backend service within a preset period of time, which is the first number; and determine the total number of failed responses to user requests by each backend service corresponding to the current API gateway within the preset period of time, which is the second number; A ratio of the first number to the second number is determined as a request error rate of the backend service.

4. The method according to claim 3, wherein: Determining the number of failed responses of the backend service to the user request within a preset time period as a first number includes: The number of user requests received by the backend service within a preset period of time is obtained as a first total number; the number of user requests successfully responded to by the backend service within the preset period of time is obtained as a first total number of responses; and the number of user requests not transmitted to the backend service within the preset period of time is obtained as a first total number of transmissions; The first total number is subtracted from the first total number to obtain a first difference, and the first total number of transmissions is subtracted from the first difference to obtain the first number.

5. The method according to claim 3, wherein: Determine the total number of failures of each backend service corresponding to the current API gateway to respond to user requests within a preset period of time, which is the second number, including: The total number of user requests received by each of the backend services within a preset period of time is obtained, which is the second total number; the total number of user requests successfully responded to by each of the backend services within the preset period of time is obtained, which is the second total number of responses; and the total number of user requests not transmitted to each of the backend services within the preset period of time is obtained, which is the second total number of transmissions; The second total number is subtracted from the second total number to obtain a second difference, and the second total number of transmissions is subtracted from the second difference to obtain the second number.

6. The method according to any one of claims 1 to 5, wherein: When the request processing information includes a time-consuming ranking ratio, the time-consuming ranking ratio represents a ranking ratio of an average time-consuming successful response of a backend service to a user request within a preset time period; Determine the time-consuming ranking ratio of the backend services corresponding to the current API gateway, including: Determine the ranking value of the backend service in a preset set according to the average time consumption value of the backend service; wherein the average time consumption value represents the average time consumption of the backend service in responding to the user request within a preset period; the preset set includes each backend service within the geographical range to which the backend service belongs; Determine a ratio between the ranking value of the backend service and the backend quantity value, which is the time-consuming ranking ratio of the backend service; wherein the backend quantity value is the total number of backend services in the preset set.

7. The method according to claim 6, wherein: Determining a ranking value of the backend service in a preset set according to the average time consumption value of the backend service includes: Determine the average time taken by backend services in a preset set; Sorting the backend services in the preset set according to the order of the average time consumption values ​​from low to high to obtain a sorted preset set; Determine the ranking order of the backend service in the sorted preset set as the ranking value of the backend service.

8. The method according to any one of claims 1 to 7, wherein: When the request processing information includes a request number ratio, the request number ratio represents the ratio of the number of user requests received by the backend service within a preset period of time; determining the request number ratio of the backend service corresponding to the current API gateway includes: Determine the number of user requests received by the backend service within the preset period of time, which is the third number; and determine the total number of user requests received by each backend service corresponding to the current API gateway within the preset period of time, which is the fourth number; A ratio between the third number and the fourth number is determined as a ratio of the number of requests served by the backend.

9. The method according to any one of claims 1 to 8, wherein: Determining a service status value of the backend service according to the request processing information of the backend service includes: Determine the sum of the values ​​of each information included in the request processing information as the service status value of the backend service; Alternatively, a weighted sum is performed on the values ​​of each information included in the request processing information to obtain the service status value of the backend service.

10. The method according to any one of claims 1 to 9, further comprising: If it is determined that the number of times that the backend service fails to respond to the user request within the preset time period is greater than the first preset number, the backend service is subjected to fuse processing; Alternatively, if it is determined that the number of times that the user request has not been responded to is greater than a second preset number, the user request is discarded.

11. A request scheduling device based on an API gateway, comprising: A first determination unit is used to determine the request processing information of the backend service corresponding to the current API gateway when it is determined that the response to the received user request fails; wherein the user request is used to indicate the call of the backend service; and the request processing information represents the processing status of the backend service for the received user request within a preset time period; A second determining unit is used to determine a service status value of the backend service according to the request processing information of the backend service; wherein the service status value represents the ability of the backend service to process user requests; The third determination unit is used to determine a target backend service from the backend services corresponding to the current API gateway according to the service status value; and send the user request to the target backend service for response processing.

12. The device according to claim 11, wherein The request processing information includes one or more of the following: request error rate, time-consuming ranking ratio, and request number ratio; Among them, the request error rate represents the proportion of failed responses to user requests by the backend service within a preset time period; the time ranking ratio represents the ranking ratio of the average time taken for the backend service to successfully respond to user requests within a preset time period; and the request number ratio represents the ratio of the number of user requests received by the backend service within a preset time period.

13. The device according to claim 11 or 12, wherein: When the request processing information includes a request error rate, the request error rate represents the proportion of failures of the backend service to respond to user requests within a preset period of time; The first determining unit includes: A first determination module is used to determine the number of failed responses of the backend service to user requests within a preset time period, which is a first number; The second determination module is used to determine the total number of failed responses to user requests by each backend service corresponding to the current API gateway within a preset time period, which is the second number; The third determination module is used to determine a ratio between the first number and the second number, which is a request error rate of the backend service.

14. The device according to claim 13, wherein: The first determination module includes: A first acquisition submodule is used to acquire the number of user requests received by the backend service within a preset period of time, which is a first total number; and to acquire the number of user requests successfully responded to by the backend service within the preset period of time, which is a first total number of responses; and to acquire the number of user requests that were not transmitted to the backend service within the preset period of time, which is a first total number of transmissions; The first determination submodule is used to determine a difference between the first total number, the first total number of responses, and the first total number of transmissions as the first number.

15. The device according to claim 13, wherein: The second determining module includes: The second acquisition submodule is used to acquire the total number of user requests received by each of the backend services within a preset period of time, which is the second total number; and to acquire the total number of user requests successfully responded to by each of the backend services within the preset period of time, which is the second total number of responses; and to acquire the total number of user requests that were not transmitted to each of the backend services within the preset period of time, which is the second total number of transmissions; The second determining submodule is configured to subtract the second total number of responses from the second total number to obtain a second difference, and to subtract the second total number of transmissions from the second difference to obtain the second number.

16. The device according to any one of claims 11 to 15, wherein: When the request processing information includes a time-consuming ranking ratio, the time-consuming ranking ratio represents a ranking ratio of an average time-consuming successful response of a backend service to a user request within a preset time period; The first determining unit includes: A fourth determination module is used to determine the ranking value of the backend service in a preset set according to the average time consumption value of the backend service; wherein the average time consumption value represents the average time consumption of the backend service in responding to the user request within a preset period of time; and the preset set includes each backend service within the geographical range to which the backend service belongs; The fifth determination module is used to determine the ratio between the ranking value of the backend service and the backend quantity value, which is the time-consuming ranking ratio of the backend service; wherein the backend quantity value is the total number of backend services in the preset set.

17. The device according to claim 16, wherein: The fourth determining module includes: The third determination submodule is used to determine the average time consumption value of the backend services in the preset set; The fourth determination submodule is used to sort the backend services in the preset set according to the order of the average time consumption values ​​from low to high to obtain a sorted preset set; determine the ranking order of the backend services in the sorted preset set as the ranking value of the backend services.

18. The device according to any one of claims 11 to 17, wherein: When the request processing information includes a request number ratio, the request number ratio represents a ratio of the number of user requests received by the backend service within a preset period of time; The first determining unit includes: a sixth determining module, configured to determine the number of user requests received by the backend service within a preset period of time, which is a third number; And determine the total number of user requests received by each backend service corresponding to the current API gateway within a preset time period, which is the fourth number; determine the ratio of the third number to the fourth number, which is the request number ratio of the backend service.

19. The device according to any one of claims 11 to 18, wherein: The second determining unit is specifically configured to: Determine the sum of the values ​​of each information included in the request processing information as the service status value of the backend service; Alternatively, a weighted sum is performed on the values ​​of each information included in the request processing information to obtain the service status value of the backend service.

20. The apparatus according to any one of claims 11 to 19, further comprising: A first processing unit is configured to perform a fuse process on the backend service if it is determined that the number of times that the backend service fails to respond to the user request within a preset time period is greater than a first preset number; Alternatively, the second processing unit is configured to discard the user request if it is determined that the number of times the user request has not been responded to is greater than a second preset number of times.

21. An API gateway, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images