Remote procedure call control method, device and equipment and computer storage medium

Abstract

This application discloses a remote procedure call control method, apparatus, device, and computer storage medium for gradually reducing timeout durations during microservice calls to avoid end-to-end timeouts and reduce resource waste. The method includes: when the processing of a received first call request involves a call to a second microservice, determining the reduction duration corresponding to the second microservice; determining a second timeout duration corresponding to the second microservice based on a first timeout duration corresponding to the first call request; initiating a second call request to the second microservice based on the first timeout duration and the reduction duration; and processing the first call request based on a lossy response returned by the second microservice in response to the second call request; the lossy response is the actual timeout duration determined by the second microservice based on the second timeout duration and the reduction duration, returned based on the processed portion of the second call request when the second call request processing times out.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to the field of remote procedure call technology, providing a remote procedure call control method, apparatus, device, and computer storage medium. Background Technology

[0002] As system complexity increases and the demand for system scalability rises, microservice architecture has emerged to meet these seemingly contradictory requirements. In microservice architecture, different functions and modules are cut into services, becoming cross-process or even cross-server functional components, and collaborating and interconnecting with each other through remote procedure calls (RPC).

[0003] If microservice A calls another microservice B and microservice B encounters an exception, microservice B will experience a service timeout. Under heavy network pressure, microservice B may accumulate a large number of timed-out calls. When more requests arrive, the timeout situation may worsen, potentially causing microservice B to crash. To avoid this, a timeout (deadline time) parameter is introduced in microservice architectures. This parameter is passed throughout the call chain. For example, when microservice B detects that its business processing time has exceeded the timeout, it will stop processing the timed-out business logic and release related memory and network resources, thus preventing microservice B from crashing.

[0004] However, since the timeout duration is currently shared across all microservices in the entire call chain, once a microservice times out, other microservices that called that microservice will also time out, rendering the business logic that these microservices had already completed invalid, thus resulting in resource waste. Summary of the Invention

[0005] This application provides a remote procedure call control method, apparatus, device, and computer storage medium, which are used to gradually reduce timeouts during microservice calls, avoid end-to-end timeouts, and reduce resource waste.

[0006] On the one hand, a remote procedure call control method is provided, applied to a first microservice in a microservice cluster, wherein the first microservice is any microservice in the microservice cluster, and the method includes:

[0007] When the processing of the first received call request involves a call to the second microservice, the reduction time corresponding to the second microservice is determined;

[0008] Based on the first timeout duration corresponding to the first call request, the second timeout duration corresponding to the second microservice is determined, and the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration.

[0009] Based on the first timeout duration and the reduced duration, a second call request is initiated to the second microservice;

[0010] The first call request is processed based on the lossy response returned by the second microservice in response to the second call request; the lossy response is the response returned by the second microservice based on the processed portion of the second call request when the second call request processing times out, determined by the actual timeout duration based on the second timeout duration and the reduced duration.

[0011] On the one hand, a remote procedure call control device is provided, applied to a first microservice in a microservice cluster, wherein the first microservice is any microservice in the microservice cluster, and the device includes:

[0012] The determining unit is configured to determine the reduced duration corresponding to the second microservice when the processing of the received first call request involves a call to the second microservice; and to determine the second timeout duration corresponding to the second microservice based on the first timeout duration corresponding to the first call request, wherein the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration.

[0013] The calling unit initiates a second call request to the second microservice based on the first timeout duration and the reduced duration;

[0014] The timeout execution unit is used to process the first call request based on the lossy response returned by the second microservice in response to the second call request; the lossy response is the response returned by the second microservice based on the processed portion of the second call request when the second call request processing times out, determined by the second microservice according to the actual timeout duration and the reduced duration.

[0015] Optionally, the determining unit is used for:

[0016] When the processing of the first call request involves a call to the second microservice, the predicted call latency for calling the second microservice is determined, wherein the predicted call latency is the predicted duration of data transmission path required during the call to the second microservice;

[0017] The reduction duration is determined based on the obtained predicted call latency.

[0018] Optionally, the determining unit is configured to determine the predicted call delay using at least one of the following methods:

[0019] The predicted call latency is determined based on the actual call latency consumed by the data transmission path when calling the second microservice during the same period in the historical cycle.

[0020] The predicted call latency is determined based on the network topology information between the first microservice and the second microservice;

[0021] Based on the first feature of the request packet that calls the second microservice, the second feature of the corresponding response packet is determined, and based on the first feature and the second feature, the predicted call latency is determined.

[0022] Optionally, the device further includes a delay statistics unit, used for:

[0023] When initiating a third call request to the second microservice, record the time when the third call request is sent;

[0024] When receiving the response message returned by the second microservice in response to the third call request, record the time of receiving the response message;

[0025] The total call time of the third call request is determined based on the sending time and the receiving time;

[0026] Based on the response message, the total processing time of the second microservice in processing the third call request is obtained, and the actual call latency of the third call request is determined based on the total call time and the total processing time.

[0027] Optionally, the delay statistics unit is specifically used for:

[0028] When the third call request is sent, the current time data of its corresponding monotonic clock is read and determined as the sending time; wherein, the duration data of the monotonic clock is monotonically increasing;

[0029] When the response message is received, the current time data of its corresponding monotonic clock is read and determined as the receiving time.

[0030] Optionally, if the second microservice includes multiple service nodes located in different geographical locations, then the determining unit is specifically used for:

[0031] When calling the multiple service nodes, obtain the actual call latency of each service node;

[0032] Based on the obtained actual call latency, the multiple service nodes are divided into network regions to obtain at least one network region category. Each network region category corresponds to a latency interval, and the latency intervals corresponding to different network region categories are different.

[0033] The predicted call latency is determined based on the latency interval corresponding to the network region category where the service node corresponding to the second call request is located.

[0034] Optionally, the determining unit is specifically used for:

[0035] The second timeout duration is determined based on the difference between the first timeout duration and the processing time of the first call request; or,

[0036] Based on the reduced duration, at least one candidate value for the second timeout duration is determined, and among the at least one candidate value, the candidate value whose timeout time is earlier than the timeout time corresponding to the first timeout duration is determined as the second timeout duration; or...

[0037] The second timeout duration is determined based on the difference between the first timeout duration and the reduced duration.

[0038] Optionally, the device further includes a timeout control unit, for:

[0039] Based on the first call request, obtain the first timeout duration;

[0040] In response to the first call request, obtain the reduction time relative to the first timeout duration when the first microservice processes the first call request;

[0041] Based on the first timeout duration and the reduced duration, the actual timeout duration for the first microservice to process the first call request is determined;

[0042] During the processing of the first call request, if it is determined that the processing of the first call request has timed out based on the actual timeout duration, a response is returned to the sender of the first call request based on the processed portion of the first call request.

[0043] Optionally, the timeout control unit is specifically used for:

[0044] The first call request is parsed to obtain the business data included in the first call request, the business data including a timeout duration field indicating the timeout duration;

[0045] The business parsing method corresponding to the business data is invoked to parse the parameter value corresponding to the timeout duration field in order to obtain the first timeout duration.

[0046] Optionally, the timeout control unit is specifically used for:

[0047] Upon receiving the first call request, the call control attribute information of the first call request is obtained from the sender of the first call request. The call control attribute information is carried in the header field of the Hypertext Transfer Protocol or in the metadata of the Remote Procedure Call Protocol.

[0048] The parameter value corresponding to the timeout duration field included in the call control attribute information is parsed to obtain the first timeout duration.

[0049] Optionally, the timeout execution unit is specifically used for:

[0050] When initiating a second call request to the second microservice, record the time when the second call request is sent;

[0051] During the invocation of the second invocation request, if it is determined that the difference between the current time and the time when the second invocation request was sent is greater than the second timeout duration, and no response from the second microservice to the second invocation request has been received, then it is determined that the response to the second invocation request has timed out.

[0052] Based on the processed portion of the first call request, a response is returned to the sender corresponding to the first call request.

[0053] On one hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0054] On the one hand, a computer storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of any of the above methods.

[0055] On one hand, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the methods described above.

[0056] In this embodiment, when the first microservice is invoked to process the first call request, if the processing involves a call to the second microservice, the first microservice needs to determine the reduced timeout duration when calling the second microservice, and combine it with its own first timeout duration to determine the second timeout duration corresponding to the second microservice, so that the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration. The reduced timeout duration and the second timeout duration are then sent to the second microservice, allowing the second microservice to further reduce the timeout duration based on the second timeout duration, thus triggering the timeout earlier. In this way, once the second microservice times out, the second microservice can perform service degradation, that is, return a lossy response based on the processed portion. Since the first microservice has not yet timed out, the first microservice can continue to process the first call request based on the lossy response, thereby avoiding a single microservice timeout causing a full-link timeout, and the processed portions of the first and second call requests will not be wasted, thus avoiding the entire system being filled with a large amount of useless and discarded network traffic and reducing resource waste. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0058] Figure 1 This is a diagram illustrating a microservice architecture.

[0059] Figure 2 This is a timing diagram for a normal call in the relevant technology without a timeout.

[0060] Figure 3 This is a timing diagram of when a timeout event occurs in the relevant technology;

[0061] Figure 4 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0062] Figure 5 A flowchart illustrating the remote procedure call control method provided in this application embodiment;

[0063] Figure 6 A schematic diagram illustrating the timing principle of the timeout control mechanism provided in the embodiments of this application;

[0064] Figure 7 A schematic diagram of the timeout detection process provided in an embodiment of this application;

[0065] Figure 8A data flow diagram of an RPC call process provided in an embodiment of this application;

[0066] Figure 9 A timing diagram of a single RPC call provided in an embodiment of this application;

[0067] Figure 10 This is a schematic diagram of network region division provided in an embodiment of this application;

[0068] Figure 11 A sequence diagram of a first microservice calling a second microservice provided in an embodiment of this application;

[0069] Figure 12 A schematic diagram illustrating the process of obtaining the recommended article list provided in this application embodiment;

[0070] Figure 13 A schematic diagram of a remote procedure call control device provided in an embodiment of this application;

[0071] Figure 14 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0073] To facilitate understanding of the technical solutions provided in the embodiments of this application, some key terms used in the embodiments of this application will be explained below:

[0074] Remote Procedure Call (RPC): Within an RPC framework, remote services can be called as if they were local services. For example, consider two servers, A and B. An application deployed on server A wants to call a function or method provided by an application on server B. Since they are not in the same memory space, direct calls are not possible. The semantics of the call and the data transmitted need to be expressed over the network, hence the need for an RPC call. Especially in microservice architectures, different functions and modules are divided into different services, becoming cross-process or even cross-server functional components that collaborate and interconnect via RPC. For instance, if a server integrates a facial recognition algorithm, when a task requires calling this algorithm, an RPC call can be used to obtain the facial recognition capability. This approach reduces system coupling and simplifies development costs. The principle of RPC calls is quite simple, similar to a three-tier client / server (C / S) system. The caller is equivalent to the client, and the called party is equivalent to the server. After the caller initiates an RPC request, the called party processes the request and returns the result to the caller.

[0075] Metadata: Common RPC architectures include gRPC (Google RPC) and bRPC (Baidu RPC). Under these RPC architectures, data transmission can take two forms. The first is explicit transmission, where data is passed as request parameters within the business data. This requires a dedicated business data module to parse the parameters. The second is to use a data transmission mechanism based on the entire RPC framework, which is independent of the RPC input and output parameters. This could be the Hypertext Transfer Protocol (HTTP) header or gRPC's metadata mechanism. The advantage of this approach is that it facilitates the integration of unified plugin logic into the framework without intruding on business code.

[0076] Wall clocks: These are the real-world time recorded in a computer. Theoretically, wall clocks should be exactly the same as real-world time. However, due to differences in the precision of timers in computer systems, wall clocks in computers often need to be synchronized. This causes the wall clocks to be adjusted, so the recorded differences in the wall clocks within the same time interval may not be the same, and there may even be a case of rewinding. Wall clocks between different computers may also differ.

[0077] Monotonic clocks: Compared to wall clocks, monotonic clocks refer to clocks recorded in the same computer that are guaranteed to increase monotonically. Monotonic clocks rely solely on the computer system's own timer and are not strongly correlated with real time. They also do not exhibit the backspin phenomenon seen in wall clocks. Therefore, computer programs can use the time difference between monotonic clocks to accurately measure the elapsed time.

[0078] To facilitate understanding of the technical solution of this application, the relevant content of microservice architecture will be introduced below.

[0079] See Figure 1 The diagram shows a microservice architecture, which can be divided into an external client and a microservice cluster. Within the microservice cluster, functions are switched in a more detailed manner, and the calling process between services becomes more complex as the functions become more complex. In addition, in some cases, a microservice may need to concurrently call other services.

[0080] When a network request arrives at the microservice cluster from outside the cluster, the RPC process within the cluster is called a call chain. A call chain is a directed acyclic graph (DAG). See [link to DAG]. Figure 1 As shown, network requests from external clients first reach the Application Programming Interface (API) gateway. The API gateway first uses an RPC call to the authentication module to authenticate the network request. After successful authentication, the RPC calls the backend logic service to process the network request. During the processing of the backend logic service, RPC calls to other microservices may also be involved, such as calls to the recommendation module, caching service, and content module, etc., which will not be elaborated further.

[0081] Since a single logical call may require the participation of multiple microservices in a microservice cluster, and the same microservice may handle multiple requests simultaneously, methods are often added to distinguish different call chains. For example, a common metadata information is passed throughout the call chain, and data that uniquely identifies this call chain is stored in the metadata. A typical example is a trace ID generated using a Universally Unique Identifier (UUID). See also Figure 1As shown, for example, the following links use metadata to pass different traceIDs, distinguishing three simultaneous but completely different call chains, which belong to two different clients and have different logic (link topology). Link A and Link B originate from the same external client 1. During logical processing, Link A calls the content module to obtain relevant content, while Link B needs to call the user module to obtain user-related information. Link C originates from another external client 2, and during logical processing, it needs to call both the content module and the user module.

[0082] With metadata, microservice vendors can add more information or variables to the metadata to achieve greater control over the call chain, such as the timeout control involved in this application's embodiments. Below, the call chain is converted into a sequence diagram to more clearly understand the timeout control process. See [link to relevant documentation]. Figure 2 The diagram shown illustrates the timing of a normal call without timeout in the relevant technology. As can be seen, in the background logic service, a call is first initiated to backend module A. Then, based on the result from module A, a call is initiated to module B. Finally, the data from modules A and B are aggregated and returned to the client. When all microservices are functioning normally and no microservice experiences a timeout event, the call chain is completely normal, allowing the call result to be returned correctly.

[0083] However, if a microservice encounters an exception during this process, resulting in data loss during the call, or if there is a module logic error or network traffic loss, then the RPC process will time out. Under heavy network pressure, a failed microservice may accumulate a large amount of timeout-causing logic, potentially leading to microservice crashes. To avoid this, a timeout duration is introduced as a parameter in metadata and transmitted throughout the entire chain. When a microservice in the chain experiences a timeout, it knows that even if the current data is returned, it will inevitably be discarded by the caller. Therefore, the microservice immediately stops its current logic, releasing memory and network resources.

[0084] However, because the current end-to-end uses a unified timeout control, when a timeout event occurs, the entire link will uniformly stop processing. In other words, the timeout duration transmitted in this way is fixed across the entire link. Therefore, once a timeout occurs, all nodes in the entire link time out, making it impossible to effectively trigger fallback logic according to different interfaces. This results in the current call being invalidated, leading to resource waste. See also... Figure 3The diagram shows a sequence of events when a timeout occurs in related technologies. As can be seen, when module B experiences a timeout, it detects its own timeout and knows that even if the data is returned, it will inevitably be discarded by the background logic service. Therefore, module B immediately stops its current logic, releasing memory and network resources. However, for the various microservices preceding module B—namely, the background logic service and the API gateway—normally, after initiating a call request, if the caller does not receive a data response from the called party within a specified time, it will give up waiting and switch to timeout handling logic. Since these microservices share the same timeout duration as module B, they will also trigger timeouts when module B times out. Consequently, all microservices will stop processing, making the call invalid. The client receives no response and only displays an error message.

[0085] So, combining the above Figure 3 As shown, it's conceivable that before the timeout event occurs in module B, the backend logic service has already obtained the call result returned by module A. This means that a partial processing result already exists for the client's network request. Therefore, if service degradation can be applied to this network request—that is, returning a result based on the call result of module A, providing limited and degraded service to the client—then the complete invalidation of this network request can be avoided. Therefore, it's conceivable that, firstly, timeout control needs to be implemented in each microservice; that is, each microservice needs to determine whether it and the microservices it calls have timed out. Furthermore, as each microservice makes subsequent calls, the timeout duration should be gradually shortened, allowing the called party to trigger its timeout earlier, thus achieving service degradation.

[0086] In view of this, this application provides a remote procedure call control method. In this method, when a first microservice is called to process a first call request, if the processing involves a call to a second microservice, the first microservice needs to determine the reduced timeout duration when calling the second microservice, and combine it with its own first timeout duration to determine the second timeout duration corresponding to the second microservice, so that the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration. The reduced timeout duration and the second timeout duration are then sent to the second microservice, allowing the second microservice to further reduce the timeout duration based on the second timeout duration, so that a timeout can be triggered in advance. In this way, once the second microservice times out, the second microservice can perform service degradation, that is, return a lossy response based on the processed part. Since the first microservice has not yet timed out, the first microservice can continue to process the first call request based on the lossy response, thereby avoiding a single microservice timeout causing a full-link timeout, and the processed parts of the first and second call requests will not be wasted, thereby avoiding the entire system from being filled with a large amount of useless and discarded network traffic and reducing resource waste.

[0087] Furthermore, considering that a certain amount of time is consumed during the transmission path in the RPC call process, the time reserved for the business processing of the called party is squeezed after the transmission path consumption. Therefore, in this embodiment, the reduction time is determined based on the predicted call latency. The predicted call latency refers to the predicted time consumed by the data transmission path during the call to the second microservice. Moreover, considering that the actual call latency can only be accurately obtained after the call is completed, this embodiment provides multiple methods for predicting the call latency, and multiple methods can be combined to improve the accuracy of the predicted call latency, thereby enabling precise timeout control.

[0088] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0089] The solution provided in this application can be applied to scenarios involving most remote procedure calls, such as the microservice architecture scenario mentioned above. Figure 4 The diagram shown is an application scenario provided by an embodiment of this application. In this scenario, a terminal device 10 and a server cluster 20 may be included.

[0090] Terminal device 10 can be, for example, a mobile phone, tablet computer (PAD), laptop computer, desktop computer, smart TV, smart in-vehicle device, and smart wearable device. Terminal device 10 can have network applications installed that can use backend network services. The applications involved in this application embodiment can be software clients, web pages, mini-programs, etc., and the server is the backend server corresponding to the software, web pages, mini-programs, etc., without limiting the specific type of client.

[0091] Server cluster 20 can provide corresponding background services for network applications installed on terminal device 10. Server cluster 20 contains multiple servers, each server providing at least one functional service. See [link to documentation]. Figure 4 As shown, server cluster 20 includes access server 201, logic control server 202, and other functional servers 203-205, used to implement at least one functional service, such as user data acquisition, recommendation content acquisition, and facial recognition. Simultaneously, the servers included can also interact with a database to obtain the required data. See [link to documentation]. Figure 4As shown, both server 203 and server 205 can access the database to obtain data. It should be noted that this application embodiment does not limit the specific deployment of the server cluster; the RPC control method provided in this application embodiment can be applied to any server cluster involving RPC calls.

[0092] Each server in the server cluster 20 can execute the steps of the remote procedure call control method provided in this application embodiment to implement timeout control during the RPC call process. Each server can be an independent physical server, or a combination of nodes containing multiple physical nodes, such as a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, i.e., content delivery network (CDN), and big data and artificial intelligence platforms, but is not limited to these.

[0093] Each server in the server cluster 20 may include one or more processors, memory, and interactive I / O interfaces. The memory may also store program instructions for the remote procedure call control method provided in this embodiment. When these program instructions are executed by the processor 1021, they can be used to implement the steps of the remote procedure call control method provided in this embodiment, thereby achieving timeout control during the remote procedure call process.

[0094] Terminal device 10 and server cluster 20, as well as the servers within server cluster 20, can communicate directly or indirectly through one or more networks. This network can be a wired network or a wireless network; for example, a wireless network could be a mobile cellular network or a Wireless-Fidelity (WIFI) network, or any other possible network. This embodiment of the invention does not limit the types of networks used.

[0095] In one possible application scenario, in order to reduce the communication latency of retrieval, each server in the server cluster 20 can deploy corresponding server nodes in various regions, or for load balancing, different server nodes can serve different regions respectively.

[0096] For example, taking access server 201 as an example, if terminal device 10 is located at location a, it establishes a communication connection with access server 201 at service location a; if terminal device 10 is located at location b, it establishes a communication connection with access server 201 at service location b. Multiple access servers 201 form a data sharing system, achieving data sharing through blockchain. Each access server 201 in the data sharing system has a corresponding node identifier. Each access server 201 in the data sharing system can store the node identifiers of other access servers 201 in the data sharing system, so that the generated block can be broadcast to other access servers 201 in the data sharing system based on their node identifiers. Each access server 201 can maintain a node identifier list, storing the access server 201 name and node identifier in this list. The node identifier can be an Internet Protocol (IP) address or any other information that can be used to identify the node.

[0097] Of course, the methods provided in the embodiments of this application are not limited to... Figure 1 The application scenarios shown can also be used in other possible scenarios, and this application embodiment does not impose any limitations. Figure 1 The functions that each device in the application scenario shown can achieve will be described in the subsequent method embodiments, and will not be elaborated on here.

[0098] The method flows provided in the various embodiments of this application can be used... Figure 1 The execution can be performed by any server in the server cluster 20, or by the terminal device 10, or by the terminal device 10 and all the servers in the server cluster 20. Here, we mainly take the server cluster 20 as a microservice cluster, and the execution by the first microservice in the microservice cluster as an example. The first microservice can be any microservice in the microservice cluster.

[0099] See Figure 5 The diagram shown is a flowchart illustrating the remote procedure call control method provided in an embodiment of this application.

[0100] Step 501: The first microservice receives the first call request.

[0101] In this embodiment of the application, the first call request is used to request the first microservice to provide a certain functional service it possesses. The first call request may be initiated by other microservices in the microservice cluster or by the client.

[0102] When the first call request is initiated by the client, it is essentially a network request sent by the client. Correspondingly, the first microservice can be an access server providing access services. That is, when a user triggers a network request from the client to the backend microservice cluster to obtain a certain service, the client initiates the first call request to the first microservice. For example, when a client needs to perform face verification for a certain business function, the client encapsulates the verification data required for face verification and initiates the first call request to the first microservice to request the microservice cluster to utilize its face recognition capabilities to assist in the face verification process. Similarly, when a client enters a page that needs to display recommended data to the user, the client can also initiate the first call request to the first microservice to request the microservice cluster to distribute the recommended content.

[0103] When the first call request is initiated by another microservice in the microservice cluster, it belongs to an RPC call between service cluster contents. Therefore, the first microservice can be any callable microservice in the microservice cluster. For example, if another microservice is an access server and the first microservice is a logic server providing backend logic services, then when the access server successfully verifies the network request using the authentication service, it can initiate the first call request to the logic server to process the network request logically. Alternatively, if another server is a logic server and the first microservice is a content module, then when the logic server processes the network request and determines that it requests to play video A, it needs to obtain the resource information of video A. In this case, the logic server can call the content module, i.e., initiate the first call request to the content module, thereby obtaining the resource information of video A from the content module.

[0104] Step 502: The first microservice obtains the corresponding first timeout duration based on the received first call request.

[0105] In this embodiment of the application, to prevent the entire call chain from timing out when a microservice experiences a timeout event, thus avoiding resource waste, a new timeout control mechanism is provided. See also Figure 6The diagram illustrates the timing principle of this timeout control mechanism. Taking the first microservice as an example, the timeout mechanism is controlled within the background logic service. When module B is called, if the logic time of module B is too long, a timeout event occurs. Since the background logic service can control the timeout duration of module B, it can detect that module B has timed out during the call process. To avoid rendering the entire call process invalid, the background logic service can perform a lossy return, providing limited and degraded functionality to the client based on the call result of module A. This method is called service degradation. Although it cannot provide a complete service, it avoids rendering the entire call process invalid and only causing errors on the client side. This method not only saves resources but also prevents the inability to provide page content to the user.

[0106] Therefore, in this embodiment of the application, when making a call to the backend, a timeout duration can be passed to indicate the maximum duration allowed for the microservice business processing. For example, when the first microservice receives the first call request, it can obtain its own corresponding first timeout duration based on the first call request. The first timeout duration is the maximum duration allowed for the first microservice to process the first call request.

[0107] In this embodiment of the application, the microservice call chain requires a chain timeout duration during startup. Figure 1 Taking the microservice architecture shown as an example, the link timeout duration can be generated by the following nodes:

[0108] (1) External client: When a client initiates a network request, it can attach the desired link timeout duration to the network request. The link timeout duration is the maximum duration that the entire call link can consume.

[0109] (2) When the API gateway and various modules in the microservices, such as the backend logic service, user module, or content module, receive a call request, if they find that there is no link timeout configuration, they can generate a link timeout themselves and pass it in the call chain. For example, when the API gateway receives a network request from a client, if the network request does not carry a link timeout, the API gateway can generate a link timeout itself and determine the backend's timeout based on this link timeout when initiating a call to the backend.

[0110] In this application embodiment, the transmission methods for timeout duration include, but are not limited to, the following:

[0111] (1) The timeout duration can be passed as one of the parameters in the call request, thus enabling the timeout duration to be passed explicitly. Specifically, this method involves carrying the timeout duration in the business data of the request packet, thus requiring the corresponding business code to parse the parameter.

[0112] Correspondingly, when the first timeout duration is transmitted in this way, the first microservice, after receiving the first call request, can parse the first call request to obtain the business data included in the first call request. This parsing process is equivalent to performing protocol parsing on the request data packet and extracting the business data from the request data packet. The business data includes a timeout duration field that indicates the timeout duration. Then, the first microservice calls the business parsing method corresponding to the business data to parse the parameter value corresponding to the timeout duration field to obtain the first timeout duration.

[0113] In practical applications, various business processes may exist, and the data formats and representations corresponding to these processes may differ. Therefore, specific business code is required to parse the data for each process. For example, field A might represent the timeout duration in business 1, while field B might be used in business 2. Furthermore, the data formats in business 1 and business 2 might not be identical. Therefore, after the first microservice parses and obtains the business data from the first call request, it still needs to call the corresponding business code to parse it and obtain the first timeout duration.

[0114] (2) A data transmission mechanism based on the entire RPC framework, independent of the RPC input and output parameters, is used. This mechanism is the underlying data transmission protocol of the RPC framework itself. For example, when the RPC framework uses HTTP, the timeout duration can be transmitted based on the predefined data transmission structure of HTTP, such as using the HTTP header field set mechanism. Alternatively, when the RPC framework is gRPC, which encapsulates Transmission Control Protocol (TCP) packets, the timeout duration can be transmitted using the gRPC framework's metadata mechanism. Furthermore, some RPC frameworks are based on custom transport protocols, such as encapsulating User Datagram Protocol (UDP) packets, allowing the use of custom protocol transport mechanisms to transmit timeout durations. Compared to using the timeout duration as one of the parameters in the call request, this method facilitates the integration of unified plugin logic into the framework, does not require intrusion into business code, and is more versatile.

[0115] Correspondingly, when the first timeout duration is transmitted in this manner, the first microservice, upon receiving the first call request, can obtain the call control attribute information of the first call request from the sender corresponding to the first call request, and parse the parameter value corresponding to the timeout duration field contained in the call control attribute information to obtain the first timeout duration. The call control attribute information can be carried through the HTTP header or through gRPC metadata. Of course, when using other RPC frameworks, corresponding data transmission mechanisms can also be used for transmission; this application embodiment does not impose any limitations on this.

[0116] In one possible implementation, the first timeout duration can be sent along with the first call request. For example, after the business data corresponding to the first call request is encapsulated, the first timeout duration can be added to the corresponding protocol field during the protocol encapsulation process. After the first call request is sent to the first microservice, the first microservice can parse the first call request to obtain the first timeout duration. For instance, taking metadata as an example, when the first call request needs to be initiated, metadata is created and encapsulated into the first call request before being sent to the first microservice.

[0117] In another possible implementation, when initiating the first call request, a transmission data packet can also be generated based on the first timeout duration and sent to the first microservice through the aforementioned transmission mechanism. Again, taking metadata as an example, when initiating the first call request, metadata is created and sent to the first microservice.

[0118] Step 503: When the processing of the first call request involves a call to the second microservice, the first microservice determines the reduction duration of the second microservice relative to the first timeout duration.

[0119] In this embodiment, during the first microservice's processing of the first call request, it may need to utilize the functions of other microservices. In this case, RPC calls need to be initiated to those other microservices. For example, when displaying a product recommendation page on the client, it may involve a product recommendation module and a product sorting module. The product recommendation module can provide recommended products for the product recommendation page, while the product sorting module can sort these products so that they are displayed in the product recommendation page according to their sorting order. Therefore, when processing a display request for this product recommendation page, RPC calls need to be made to these two modules. Alternatively, when processing a network request for face verification, a face recognition algorithm is involved. If the face recognition algorithm is deployed on other microservices, then RPC calls need to be made to those other microservices to implement the face recognition process.

[0120] As mentioned above, when calling other microservices, if the timeout duration of those microservices is too long, the processing time of those microservices may be too long, leading to timeout events. If the first microservice has already timed out, the entire chain will time out, failing to return any results to the client and rendering the network request completely invalid. Therefore, to avoid this situation, the first microservice needs to limit the timeout duration of the called second microservice. Based on this, this application embodiment introduces a timeout reduction method, which requires reducing the timeout duration based on the first microservice's own initial timeout duration.

[0121] In one possible implementation, the reduction duration can be a preset fixed value, that is, a fixed reduction of a value based on the first timeout duration, which serves as the second timeout duration for the second microservice being called. Of course, considering that the processing time required for business processes of microservices at different levels varies, microservices can be classified into levels, and different reduction durations can be set for microservices at different levels. For example, levels can include interface level and method level, and the reduction durations for interface level and method level can be different.

[0122] In one possible implementation, the reduction duration can be predicted based on the current actual situation, thereby dynamically reducing the timeout duration to conform to the current actual situation and improving the accuracy of timeout control.

[0123] Step 504: The first microservice determines the second timeout duration corresponding to the second microservice based on its own first timeout duration. The timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration.

[0124] In this embodiment of the application, based on the first timeout duration and combined with other relevant information, the second timeout duration corresponding to the second microservice is determined, such that the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration. In this way, the first microservice can be scheduled to time out after the second microservice times out, thereby reserving the time required for service degradation and avoiding timeout of the entire call chain.

[0125] In one possible implementation, since the first microservice has already consumed a certain amount of time for the first call request before calling the second microservice, the second timeout duration can be determined based on the difference between the first timeout duration and the processing time of the first call request. For example, the second timeout duration can be obtained by subtracting the processing time of the first call request from the first timeout duration.

[0126] In one possible implementation, the first microservice can determine the second timeout duration based on the difference between the first timeout duration and the reduced duration. For example, the difference between the first timeout duration and the reduced duration can be used to determine the second timeout duration.

[0127] Specifically, when determining the second timeout duration, it is also possible to consider whether the first call request involves calls to other microservices besides the second microservice. If other microservices need to be called besides the second microservice, then the above difference can be directly determined as the second timeout duration. In this case, if the second microservice can return the result quickly and there is still enough time for other microservices, then the first call request can proceed normally. However, if the business processing time of the second microservice is too long, it will squeeze the business processing time of another microservice. Therefore, when the first call request involves calls to other microservices, it is also necessary to consider the processing time of other microservices. After obtaining the above difference, a certain amount of time can be reserved for other microservices, and the second timeout duration can be obtained based on the above difference and the reserved time.

[0128] In another possible implementation, the first microservice can also determine at least one candidate value for the second timeout duration based on the reduced duration. In practical applications, a mapping relationship between the reduced duration and the second timeout duration can be given. After the reduced duration is determined, multiple candidate values ​​can be determined based on this mapping relationship. Then, among the at least one candidate value, the candidate value whose timeout time is earlier than the timeout time corresponding to the first timeout duration is determined as the second timeout duration.

[0129] In practical use cases, the second timeout duration of the second microservice can also be determined based on the current actual business. The logic can be based on the complexity of the business and the business's latency requirements to determine the second timeout duration. When the business has high latency requirements, the second timeout duration can be relatively shorter. When the business is more complex and requires more time, the second timeout duration can be set to be longer.

[0130] As can be seen, since the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration, the first microservice has not yet timed out after the second microservice times out, which can reserve the time required for service degradation to perform service reduction processing, avoid end-to-end timeout, and reduce resource waste.

[0131] Step 505: The first microservice initiates a second call request to the second microservice based on the second timeout duration and the reduction duration.

[0132] Step 506: The second microservice determines the actual timeout duration based on the second timeout duration and the reduced duration.

[0133] Step 507: During the processing of the second call request, the second microservice determines that a timeout has occurred based on the actual timeout duration.

[0134] Step 508: The second microservice returns a lossy response based on the processed portion of the second call request.

[0135] Step 509: The first microservice processes the first call request based on the lossy response.

[0136] In this embodiment, the first microservice initiates a second call request to the second microservice to obtain the functions possessed by the second microservice. Specifically, when the first microservice initiates the second call request, it also sends a second timeout duration and a reduced duration to the second microservice. The second timeout duration and the reduced duration can be sent using any of the parameter transmission methods mentioned above.

[0137] In practical applications, it is necessary to determine whether the second microservice has timed out. This can be done in the following ways.

[0138] See Figure 7The diagram illustrates the timeout detection process. In one approach, the first microservice, when initiating a second call request, can send the reduced timeout duration and the second timeout duration to the second microservice. Correspondingly, after receiving the reduced and second timeout durations, the second microservice needs to determine its own actual timeout duration based on these two durations. During the processing of the second call request, it needs to check if the processing time exceeds the actual timeout duration. If it does, the current logic immediately terminates, and a lossy response is returned to the first microservice based on the currently processed portion. The first microservice can then continue processing the first call request based on this lossy response.

[0139] Considering that the second microservice's return of a timeout indication also depends on the network, and both the network and services are subject to fluctuations, it is possible that a timeout may occur but the second microservice may not be able to return a timeout indication in a timely manner. Therefore, see [link to relevant documentation]. Figure 7 As shown, in another approach, when the first microservice initiates a second call request to the second microservice, it records the sending time of the second call request. During the call process of the second microservice, the first microservice needs to determine if the difference between the current time and the sending time of the second call request is greater than a second timeout period, and if it still hasn't received a response from the second microservice for the second call request. If the difference is greater than the second timeout period and no response is received, then the response to the second call request has timed out. The first microservice can then return a response based on the processed portion of the first call request. For example, if the processed portion has already completed part of the first call request, a response can be returned based on that portion. If the processed portion indicates that the first call request has no processing result whatsoever, a fallback processing step can be performed before returning a response.

[0140] Of course, in practical applications, one of the two methods mentioned above can be implemented, or both can be implemented together to increase the reliability of timeout determination.

[0141] In one possible implementation, the first microservice can also perform timeout control for the first call request. Upon receiving the first call request, the first microservice obtains the corresponding first timeout duration, and in response to the first call request, obtains the reduction time relative to the first timeout duration for the first microservice to process the first call request.

[0142] Specifically, the first microservice can parse the first call request to obtain the reduced duration and the first timeout duration carried in the first call request; or, when the first microservice receives the first call request, it can also obtain the reduced duration and the first timeout duration from the sender of the first call request based on the above data transmission mechanism.

[0143] Furthermore, the first microservice can determine its actual timeout duration for processing the first call request based on the first timeout duration and the reduction duration. During the processing of the first call request, it can detect whether a timeout has occurred based on the actual timeout duration. If it is determined that the processing of the first call request has timed out based on the actual timeout duration, it can return a response to the sender of the first call request based on the processed part of the first call request, so as to avoid end-to-end timeout and reduce resource waste.

[0144] In one possible implementation, step 503 can be carried out using the following process.

[0145] Considering that during an RPC call, the caller and the called party may be located on different physical devices, and data needs to be transmitted through a data transmission path during the call, such as according to the seven-layer protocol framework of the Open System Interconnection Reference Model (OSI), the data transmission path will consume a certain amount of time delay. Therefore, in the process of determining the reduction time, the data transmission delay can be regarded as an influencing factor in determining the reduction time. Based on this, the embodiments of this application introduce the concept of call delay, which refers to the sum of the time consumed by a series of services on the data transmission path.

[0146] First, to facilitate understanding of the subsequent technical solutions, it is necessary to explain the definition of call latency. See [link to relevant documentation]. Figure 8 The diagram illustrates the data flow of an RPC call process. The business logic runs within a microservice framework, which in turn runs an application. This application runs on an operating system, which in turn runs on server hardware or containerized virtual server hardware. The servers are interconnected via a network, forming a server cluster. See also... Figure 8 As shown in the solid data flow diagram, when business code executes an RPC framework software development kit (SDK) or software interface to initiate an RPC call, network data is transmitted to the actual network device via electrical and optical signals through the RPC framework, which invokes operating system system calls and the operating system's network driver to operate the hardware. When the network signal reaches the server of the called party, the data flow processing is the reverse of the above process. After the called party completes its business, it needs to return data; see [link to relevant documentation]. Figure 8 As shown by the dashed data stream in the diagram, the response data returns from the called party to the called party through a similar process.

[0147] If represented by a sequence diagram, please refer to... Figure 9 The figure shows a timing diagram of an RPC call, where time p in the diagram is...busi (busi stands for business) represents the business processing time of the called party, and the transmission time of the call request to the business code of the called party is denoted as p. la1 (la represents latency), the sum of the transmission times for the call result to the calling party's business code is p. la2 Then call delay p la as follows:

[0148] p la =p la1 +p la2

[0149] Therefore, when the processing of the first call request involves a call to the second microservice, the first microservice can determine the predicted call latency of the second microservice, that is, the predicted time required for the data transmission path during the call to the second microservice, and thus determine the reduction time based on the obtained predicted call latency.

[0150] Specifically, the predicted call latency can be used as the reduction duration, that is, the second timeout duration of the second microservice can be determined based on the predicted call latency.

[0151] In one possible implementation, the predicted call latency and the second timeout duration should be positively correlated. A longer predicted call latency allows the microservice more time to process its business logic, resulting in a longer second timeout duration. The second timeout duration should always be greater than the predicted call latency to ensure a certain margin. For example, the predicted call latency and the second timeout duration can satisfy the following relationship:

[0152] P tmn =f(P′) lamn )=aP′ lamn +b

[0153] Among them, P tmn P represents the timeout duration when microservice n is invoked by microservice m. lamn The predicted call latency from microservice m to microservice n, where m and n represent the m-th or n-th microservice, m, n = 0, 1, 2, ..., and a and b are constants.

[0154] In one possible implementation, the first microservice can subtract a preset call delay from its own first timeout duration to obtain the second timeout duration of the second microservice.

[0155] When initiating an RPC call, the call latency is unknown. Therefore, in actual use, the call latency can only be predicted. In order to improve the accuracy of timeout control, it is necessary to accurately predict the call latency.

[0156] In this application embodiment, the following methods can be used to obtain the predicted call latency. In the specific implementation process, the following methods can be used individually or in combination.

[0157] (1) Based on the actual call latency consumed by the data transmission path when calling the second microservice during the same period in the historical cycle, determine the predicted call latency of this RPC call.

[0158] The same time period can include the following situations:

[0159] First, if a fixed duration is used as the cycle, then the same time period corresponds to the same time period in the historical cycle. For example, if the cycle is one week, then the same time period could refer to the same time period seven days ago, the same time period could refer to a time period, or it could refer to a point in time.

[0160] Second, if holidays are taken as the cycle, the same period of time can refer to the same period of time or similar periods of time in the days before and after the start of the holiday.

[0161] Third, if the cycle is based on operational activities, then the same time period can correspond to the pre-event period, the same time period after the start of the operational activity, or a similar time period. For example, for the May Day holiday, the call delay for the same time period during last year's May Day holiday can be referenced.

[0162] In this embodiment of the application, the actual call latency in the same period of the previous historical cycle can be determined as the predicted call latency of the current RPC call, or the average or median of the actual call latency in the same period of multiple historical cycles can be obtained as the predicted call latency of the current RPC call.

[0163] Therefore, in order to utilize the actual call latency data in subsequent use, the actual call latency of each call can be calculated during each call.

[0164] Theoretically, the latency of an RPC call can be calculated as follows:

[0165] See Figure 9 As shown, the caller knows the moment t when the RPC call request is issued from the business code. la10 Meanwhile, the called party knows the time t when the RPC call request arrives at the business code. la11 And the called party knows that after the business processing is completed, the RPC response begins to leave the business code, and the time t begins to return. la20 Furthermore, it has the capability to return these moments to the caller, ensuring that the caller knows the moment t when the RPC response returns from the called party to the caller's business code. la21 Therefore, the actual call latency can be calculated using the above-mentioned time points, i.e.:

[0166] p la =(t la11 -t la10 )+(t la21 -t la20 )

[0167] However, in reality, different business logic is highly likely to be deployed on different physical servers. No matter how well time synchronization is implemented on these servers, observable time differences are inevitable. Differences in wall clock times reaching 100ms are normal. Furthermore, wall clock times may be updated during synchronization, making the obtained time inaccurate. Under such clock conditions, the above method may not be able to accurately calculate p. la .

[0168] However, servers typically also use monotonic clocks. The monotonic time on the same server is calculable, depending on the server's monotonic clock rather than a wall clock, although p cannot be calculated. la1 and p la2 However, p can be calculated. la The specific method is as follows:

[0169] See Figure 9 As shown, when the first microservice initiates a third call request to the second microservice, it records the sending time of the third call request. For example, the sending time can be recorded using a monotonic clock, that is, by reading the current time data of its own corresponding monotonic clock, it determines the start time t′ of this RPC call. rpc0 Furthermore, when the first microservice receives the response message from the second microservice in response to the third call request, it can also record the reception time of the response message using a monotonic clock. Specifically, it reads the current time data of its own corresponding monotonic clock to determine the end time t′ of this RPC call. rpc1 Therefore, the total call time of the third call request can be determined based on the sending and receiving times, as follows:

[0170] p rpc =t′ rpc0 -t′ rpc1

[0171] The second microservice, acting as the called party, can also record the time t′ when the third call request is received. busi0 and processing completion time t′ busi1 Thus, the total processing time for handling the third call request is obtained as follows:

[0172] p busi =t′ busi0 -t′ busi1

[0173] Therefore, the first microservice can determine the actual call latency corresponding to the third call request based on the total call time and the total processing time, as follows:

[0174] p la =p rpc -p busi

[0175] Therefore, during each RPC call, the called party can accurately observe p using a monotonic clock. busi And it is returned to the primary regulator, who can then accurately observe p using a monotonic clock. rpc Thus, p can be accurately calculated. la This provides an accurate data foundation for determining the predicted call latency during subsequent calls.

[0176] (2) Determine the predicted call latency based on the network topology information between the first microservice and the second microservice.

[0177] In this embodiment, considering that each microservice may be deployed on different service nodes, and different service stages may be on different physical servers, distributed across different networks and data centers, or even in different cities and countries, similarly, the caller and the called party may also exist in different networks, different data centers, or even different cities and countries. Therefore, when the first microservice initiates an RPC and performs address lookup, if it can obtain the network topology location information of the called party, it can estimate the predicted call latency based on the network topology information.

[0178] In one possible implementation, if the network topology information between the various microservices can be pre-configured, then when the first microservice initiates an RPC call to the second microservice, it can know the network topology information between them and thus estimate the predicted call latency. When the second microservice has multiple service nodes located in different geographical locations, the predicted call latency for each of the multiple service nodes can also be estimated separately, and the optimal service node can be selected to initiate the call.

[0179] In another possible implementation, considering that in practical applications the called party is usually a service defined by name, and the name may contain a combination of multiple Internet Protocol (IP) ports, then the records of actual call latency in the first method can be combined to classify and summarize multiple nodes.

[0180] Specifically, the first microservice can obtain the actual call latency of each service node when calling multiple service nodes in the past, and divide the multiple service nodes into network regions based on the obtained actual call latency to obtain at least one network region category. Each network region category corresponds to a latency interval, and the latency intervals corresponding to different network region categories are different.

[0181] For example, as shown in Table 1 below, the second microservice has multiple service nodes located in different geographical locations:

[0182] 1 172.16.23.5:7562 2 172.16.15.138:5977 3 172.16.57.33:9572 4 172.16.174.9:6275 5 172.16.12.20:5689 6 172.16.47.218:13672

[0183] Table 1

[0184] As can be seen, these six nodes show no geographical relationship based on their IP addresses. Therefore, we can try using a round-robin approach to initiate RPC calls to these six nodes and calculate the average actual call latency of these six nodes after a period of time, as shown in Table 2 below:

[0185] 1 172.16.23.5:7562 12.1ms 2 172.16.15.138:5977 11.9ms 3 172.16.57.33:9572 49.5ms 4 172.16.174.9:6275 48.9ms 5 172.16.12.20:5689 12.3ms 6 172.16.47.218:13672 103.2ms

[0186] Table 2

[0187] It is evident that the average actual call latency of the aforementioned nodes is distributed across three intervals, as shown in Table 3 below. These nodes can be categorized into three network regions. (See Table 3 for details.) Figure 10 As shown, nodes 1, 2, and 5 are located in the same network area, meaning their relative distances to the first microservice are within a certain range. Nodes 3 and 4 are also located in one network area, while node 6 is located in another. It is evident that nodes 1, 2, and 5 have the lowest average actual call latency. Therefore, in subsequent load balancing traffic allocation, nodes 1, 2, and 5 can be given priority.

[0188]

[0189] Table 3

[0190] Furthermore, once the service node to be invoked by the second invocation request is determined, the predicted invocation latency can be determined based on the latency interval corresponding to the network region category where the service node is located. For example, after classifying the network region categories, the latency interval for each network region category can be determined, and the value within the latency interval can be selected as the predicted invocation latency for this invocation; alternatively, the average latency can be determined based on the latency interval as the actual invocation latency for that network region category, and then the actual invocation latency for that network region category can be determined as the predicted invocation latency for this invocation.

[0191] Of course, in practical use, P′ can also be calculated by referring to historically saved data on different network topologies. la The prediction means that this method can also be combined with the first method. That is, after determining the service node that the second call request needs to call, the actual call latency of the network area category corresponding to the service node in the same period of the historical period can be obtained.

[0192] In this embodiment of the application, the detection of the actual call latency can be used not only for subsequent call latency prediction, but also to estimate the network distance classification of multiple service nodes based on the call latency, thereby assisting in inferring whether multiple service nodes are located in the same data center.

[0193] Furthermore, in general, cloud service providers typically have at least two or more backup links between geographically separated network nodes, or when a link fails, network traffic is transmitted through other network topologies. In this case, the actual call latency detection described in this application embodiment can be used for network connectivity detection. For example, when fluctuations or anomalies in network latency are detected in obviously nearby nodes, an anomaly alarm can be issued, and operations and maintenance personnel can be notified to handle the situation accordingly.

[0194] (3) Determine the second feature of the corresponding response data packet based on the first feature of the request data packet for calling the second microservice, and determine the predicted call latency based on the first feature and the second feature.

[0195] In this embodiment of the application, considering that the size and complexity of data packets may also be factors that constrain call latency, the data packet factor can also be considered when determining the predicted call latency.

[0196] Specifically, when initiating an RPC call, the size of the request data packet is known to the first microservice, but the size of the returned data packet is unknown to the first microservice. Therefore, it is necessary to use artificial intelligence (AI) algorithms to predict the size based on the characteristics of the business. Then, based on the predicted size of the returned data packet, the network latency model is used to calculate the predicted call latency.

[0197] Generally speaking, when other conditions are the same, such as the same invocation time, the same caller, and the same callee, the predicted invocation latency is positively correlated with the size of the request data packet.

[0198] P′ la =f(D req ,D′ rsp )=aD req +bD′ rsp +c

[0199] Among them, D req This indicates the amount of data in the request packet, which is generally the length of the packet. Of course, in some cases, the complexity of the packet structure can also be considered. rsp It is the amount of data in the response data packet obtained based on the request data packet and the current business forecast. a, b, and c are constants, which can be obtained through fitting and parameter tuning through a machine learning process.

[0200] In practical applications, network topology information can also be used for prediction. For example, the network nodes where the main player is located are: N1, N2, ..., N m The network nodes where the target is located are: N1, N2, ..., N n Then the network topologies of the two locations can form a matrix:

[0201]

[0202] Furthermore, the prediction call latency corresponding to each element in the matrix can be calculated:

[0203] P′ la =f(D req ,D′ rsp ) = a mn D req +b mn D′ rsp +c mn

[0204] In this application embodiment, the above-mentioned methods can be used in combination, or combined with other possible prediction methods for the prediction process. This application embodiment does not limit this.

[0205] The following section describes the solution provided in the embodiments of this application, taking into account the predicted call latency. (See also...) Figure 11 The diagram shows the sequence of the first microservice calling the second microservice. It should be noted that, for the sake of easy display of the sequence diagram, the predicted call latency is simulated as a module.

[0206] First, when the first microservice receives the first call request, it can obtain its own first timeout duration, i.e. Figure 11 P shown t1 The corresponding timeout time is t′. t1 When the first microservice needs to call the second microservice, the first microservice needs to determine the second timeout duration corresponding to the second microservice.

[0207] In one possible implementation, a timeout duration can be preset for each microservice, for example, a timeout duration can be set for each microservice, or a timeout duration can be set for both the caller and the called party. Then, the corresponding second timeout duration can be determined based on the timeout duration set for the second microservice.

[0208] In one possible implementation, the first microservice can determine the predicted call latency when it calls the second microservice, i.e. Figure 11 The predicted call delay 1 in the middle consists of two parts: the call delay on the sending path and the call delay on the receiving path. Therefore, the second timeout duration can be calculated based on the calculation relationship between the predicted call delay and the second timeout duration, or the second timeout duration can be calculated based on the predicted call delay and the first timeout duration.

[0209] Then, in addition to sending the second timeout duration to the second microservice, the first microservice also sends the predicted call latency to the second microservice. Upon receiving the second timeout duration, the second microservice will reduce the predicted call latency based on the second timeout duration, using this as its actual timeout duration, and will then perform logical processing based on the actual timeout duration. Figure 11 P shown t2 The corresponding timeout time is t′. t2 As can be seen, P t2 <P t12 And t′ t2 Earlier than t′ t1 This is the result of the called party reducing the current service timeout time accordingly based on the timeout parameters of the calling party.

[0210] like Figure 11 As shown by the solid line, when no timeout event occurs, each microservice can return on time, and the entire call chain can proceed normally. In addition, after processing the data, the second microservice returns its total execution time to the first microservice. After receiving this data, the first microservice subtracts the processing time of the second microservice from its actual measured total call execution time, and the difference is the actual call latency.

[0211] When a timeout event occurs, such as Figure 11 As shown by the dashed line, at this time t′ t2 =t′ 21 The second microservice detects that the backend microservice has timed out, so it immediately abandons waiting for the backend service call and returns immediately, or performs fallback processing and returns immediately. We can see that at this point, t′ 13 Earlier than t′ t1 Therefore, for the first microservice, no timeout has occurred yet, and the entire business is running at P. t1The return was received earlier, avoiding errors caused by end-to-end timeouts.

[0212] Taking the client requesting the backend to display a list of recommended articles as an example, see... Figure 12 The diagram illustrates the process of retrieving the recommended article list. When a client sends a request to the API gateway to retrieve the recommended article list, the API gateway, after successful authentication via the authentication service, calls the backend logic processing service for processing. The backend logic service then calls the recommendation module to retrieve the recommendation list. During this process, when the API gateway initiates the call, it needs to determine the predicted call latency corresponding to the backend logic processing service and send both the link timeout duration and the predicted call latency to the backend logic processing service. The backend logic processing service will reduce the predicted call latency based on the link timeout duration to obtain its own actual timeout duration. Furthermore, it will further reduce the predicted call latency based on its own actual timeout duration to obtain the link timeout duration for the backend calling module, or specify the link timeout duration for the backend based on the actual business requirements. Similarly, the backend logic processing service will also determine the predicted call latency corresponding to the recommendation module and send both the link timeout duration and the predicted call latency to the recommendation module.

[0213] Once all microservices receive a response before the timeout, such as the recommendation module returning the recommendation list on time, the response is sequentially sent to the client via the backend logic processing service and the API gateway for display, so that a page containing this list of recommended articles is presented on the client.

[0214] When a microservice experiences a timeout event, for example Figure 12 If the recommendation module times out when returning the recommendation list, according to relevant technical solutions, a full-link timeout will occur, resulting in an error on the client side. However, in this embodiment, because the timeout duration has been shortened, the timeout time of the application itself is later than that of the backend, allowing for timely fallback processing. When the backend logic processing service detects a timeout, it performs fallback processing. Fallback processing refers to logical processing performed to prevent poor user experience or serious abnormalities in the call chain after an error occurs. For example, if the recommendation list cannot be obtained, the locally cached recent popular articles can be returned. The backend logic processing service then returns the list of popular articles to the client through the API gateway, so that the client displays a page containing this list of popular articles. It can be seen that by adopting the timeout control scheme of this embodiment, the timeout duration has been shortened, ensuring that the application's own timeout time is later than that of the backend, timely fallback processing can be performed, avoiding full-link timeouts and improving user experience.

[0215] In summary, in this embodiment of the application, the correct timeout fallback logic is implemented for business timeout situations to avoid RPC full-link timeouts that prevent the system from correctly performing fallback logic processing. At the same time, it avoids the system being filled with a large amount of useless and discarded network traffic, which would otherwise waste system resources.

[0216] Please see Figure 13 Based on the same inventive concept, this application also provides a remote procedure call control device 130, applied to a first microservice in a microservice cluster, wherein the first microservice is any microservice in the microservice cluster, and the device includes:

[0217] The determining unit 1301 is configured to determine the reduced duration corresponding to the second microservice when the processing of the received first call request involves a call to the second microservice; and to determine the second timeout duration corresponding to the second microservice based on the first timeout duration corresponding to the first call request, wherein the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration.

[0218] Calling unit 1302 initiates a second call request to the second microservice based on the first timeout duration and the reduced duration;

[0219] The timeout execution unit 1303 is used to process the first call request based on the lossy response returned by the second microservice for the second call request; the lossy response is returned by the second microservice based on the processed part of the second call request when the second call request processing times out, based on the actual timeout duration determined by the second microservice according to the second timeout duration and the reduced duration.

[0220] Optionally, the determining unit 1301 is used for:

[0221] When the processing of the first call request involves a call to the second microservice, the predicted call latency for the second microservice is determined. The predicted call latency is the predicted duration of the data transmission path required during the call to the second microservice.

[0222] Based on the obtained predicted call latency, the duration of the reduction is determined.

[0223] Optionally, the determining unit 1301 is used to determine the predicted call delay using at least one of the following methods:

[0224] Based on the actual call latency consumed by the data transmission path when calling the second microservice during the same period in the historical cycle, the predicted call latency is determined.

[0225] Based on the network topology information between the first microservice and the second microservice, the predicted call latency is determined;

[0226] The second feature of the corresponding response packet is determined based on the first feature of the request packet that calls the second microservice, and the predicted call latency is determined based on the first feature and the second feature.

[0227] Optionally, the device also includes a delay statistics unit 1304, used for:

[0228] When initiating a third call request to the second microservice, record the time when the third call request is sent;

[0229] When receiving the response message returned by the second microservice in response to the third call request, record the moment the response message is received;

[0230] The total call time for the third call request is determined based on the sending and receiving times.

[0231] Based on the response message, obtain the total processing time of the second microservice in processing the third call request, and determine the actual call latency of the third call request based on the total call time and the total processing time.

[0232] Optional, the delay statistics unit 1304 is specifically used for:

[0233] When the third call request is sent, the current time data of its corresponding monotonic clock is read and determined as the sending time; the duration data of the monotonic clock is monotonically increasing.

[0234] When responding to a message, the system reads the current time data of its own monotonic clock and determines it as the receiving time.

[0235] Optionally, if the second microservice includes multiple service nodes located in different geographical locations, then the determining unit 1301 is specifically used for:

[0236] When calling multiple service nodes, obtain the actual call latency for each service node;

[0237] Based on the obtained actual call latency, network regions are divided for multiple service nodes to obtain at least one network region category. Each network region category corresponds to a latency interval, and the latency intervals corresponding to different network region categories are different.

[0238] The predicted call latency is determined based on the latency range corresponding to the network region category of the service node corresponding to the second call request.

[0239] Optionally, the determining unit 1301 is specifically used for:

[0240] The second timeout duration is determined based on the difference between the first timeout duration and the processing time of the first call request; or,

[0241] Based on the reduction duration, at least one candidate value for the second timeout duration is determined, and among the at least one candidate value, the candidate value whose timeout time is earlier than the timeout time corresponding to the first timeout duration is determined as the second timeout duration; or...

[0242] The second timeout duration is determined based on the difference between the first timeout duration and the reduced duration.

[0243] Optionally, the device also includes a timeout control unit 1305, for:

[0244] Based on the first call request, obtain the first timeout duration;

[0245] In response to the first call request, obtain the reduced timeout period relative to the first timeout period when the first microservice processes the first call request;

[0246] Based on the first timeout duration and the reduced duration, determine the actual timeout duration for the first microservice to process the first call request;

[0247] During the processing of the first call request, if it is determined that the processing of the first call request has timed out based on the actual timeout duration, a response is returned to the sender of the first call request based on the processed portion of the first call request.

[0248] Optional, the timeout control unit 1305 is specifically used for:

[0249] The first call request is parsed to obtain the business data included in the first call request, which includes a timeout duration field indicating the timeout duration.

[0250] Call the business parsing method corresponding to the business data to parse the parameter value corresponding to the timeout duration field to obtain the first timeout duration.

[0251] Optional, the timeout control unit 1305 is specifically used for:

[0252] Upon receiving the first call request, the call control attribute information of the first call request is obtained from the sender of the first call request. The call control attribute information is carried in the header field of the Hypertext Transfer Protocol or in the metadata of the Remote Procedure Call Protocol.

[0253] The parameter value corresponding to the timeout duration field contained in the call control attribute information is parsed to obtain the first timeout duration.

[0254] Optionally, the timeout execution unit 1303 is specifically used for:

[0255] When initiating a second call request to the second microservice, record the time when the second call request is sent;

[0256] During the invocation of the second call request, if it is determined that the difference between the current time and the time when the second call request was sent is greater than the second timeout duration, and no response from the second microservice for the second call request has been received, then it is determined that the response to the second call request has timed out.

[0257] Based on the processed portion of the first call request, a response is returned to the sender corresponding to the first call request.

[0258] With the above-described device, when the first microservice is invoked to process the first call request, if the processing involves a call to the second microservice, the first microservice needs to determine the reduced timeout duration when calling the second microservice. Based on its own first timeout duration, it combines the reduced timeout duration to obtain the second timeout duration corresponding to the second microservice, ensuring that the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration. Thus, if the second microservice times out, since the first microservice has not yet timed out, the first microservice can perform service degradation processing based on the processed portion of the first call request. This avoids a single microservice timeout causing a full-link timeout, and the processed portion of the first call request is not wasted, thereby preventing the entire system from being filled with a large amount of useless, discarded network traffic and reducing resource waste.

[0259] This device can be used to execute the methods shown in the various embodiments of this application. Therefore, the functions that each functional module of this device can achieve can be referred to the description of the foregoing embodiments, and will not be repeated here.

[0260] Please see Figure 14 Based on the same technical concept, this application also provides a computer device 140, which can be used for... Figure 4 Each of the servers shown may include a memory 1401 and a processor 1402.

[0261] The memory 1401 is used to store computer programs executed by the processor 1402. The memory 1401 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function, etc.; the data storage area may store data created based on the use of the computer device, etc. The processor 1402 may be a central processing unit (CPU), or a digital processing unit, etc. This application embodiment does not limit the specific connection medium between the memory 1401 and the processor 1402. This application embodiment... Figure 14 The memory 1401 and the processor 1402 are connected via a bus 1403, and the bus 1403 is in Figure 14The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus 1403 can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 14 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0262] Memory 1401 may be volatile memory, such as random-access memory (RAM); memory 1401 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1401 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1401 may be a combination of the above-described memories.

[0263] The processor 1402 is configured to execute the methods performed by the devices in the various embodiments of this application when calling the computer program stored in the memory 1401.

[0264] In some possible implementations, various aspects of the methods provided in this application may also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of the methods described above according to various exemplary embodiments of this application. For example, the computer device may perform the methods performed by the device in various embodiments of this application.

[0265] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0266] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0267] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A remote procedure call control method characterized by, The method, applied to a first microservice in a microservice cluster, wherein the first microservice is any microservice in the microservice cluster, includes: When the processing of the received first call request involves a call to the second microservice, the predicted call latency for calling the second microservice is determined, wherein the predicted call latency is the predicted duration of the data transmission path required during the call to the second microservice; The reduction duration corresponding to the second microservice is determined based on the predicted call latency. Based on the difference between the first timeout duration corresponding to the first call request and the processing time of the first call request, the second timeout duration corresponding to the second microservice is determined, and the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration. Based on the second timeout duration and the reduced duration, a second call request is initiated to the second microservice; Based on the lossy response returned by the second microservice for the second call request, the first call request is processed; the lossy response is the response returned by the second microservice when the second call request processing times out, based on the processed portion of the second call request, after the second microservice determines the actual timeout duration by subtracting the reduced duration from the second timeout duration. The predicted call latency is determined using at least one of the following methods: The predicted call latency is determined based on the actual call latency consumed by the data transmission path when calling the second microservice during the same period in the historical cycle. The predicted call latency is determined based on the network topology information between the first microservice and the second microservice; Based on the first feature of the request packet that calls the second microservice, the second feature of the corresponding response packet is determined, and based on the first feature and the second feature, the predicted call latency is determined.

2. The method of claim 1, wherein, Before determining the predicted call latency based on the actual call latency consumed by the data transmission path when calling the second microservice during the same period in a historical cycle, the method further includes: When initiating a third call request to the second microservice, record the time when the third call request is sent; When receiving the response message returned by the second microservice in response to the third call request, record the time of receiving the response message; The total call time of the third call request is determined based on the sending time and the receiving time; Based on the response message, the total processing time of the second microservice in processing the third call request is obtained, and the actual call latency of the third call request is determined based on the total call time and the total processing time.

3. The method of claim 2, wherein, Record the time when the third call request was sent, including: When the third call request is sent, the current time data of its corresponding monotonic clock is read and determined as the sending time; wherein, the duration data of the monotonic clock is monotonically increasing; Record the time of receiving the response message, including: When the response message is received, the current time data of its corresponding monotonic clock is read and determined as the receiving time.

4. The method as described in claim 1, characterized in that, The second microservice includes multiple service nodes located in different geographical locations. Based on the network topology information between itself and the second microservice, the predicted call latency is determined, including: When calling the multiple service nodes, obtain the actual call latency of each service node; Based on the obtained actual call latency, the multiple service nodes are divided into network regions to obtain at least one network region category. Each network region category corresponds to a latency interval, and the latency intervals corresponding to different network region categories are different. The predicted call latency is determined based on the latency interval corresponding to the network region category where the service node corresponding to the second call request is located.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Based on the first call request, obtain the first timeout duration; In response to the first call request, obtain the reduction time relative to the first timeout duration when the first microservice processes the first call request; Based on the first timeout duration and the reduced duration, the actual timeout duration for the first microservice to process the first call request is determined; During the processing of the first call request, if it is determined that the processing of the first call request has timed out based on the actual timeout duration, a response is returned to the sender of the first call request based on the processed portion of the first call request.

6. The method as described in claim 5, characterized in that, Based on the first call request, the first timeout duration is obtained, including: The first call request is parsed to obtain the business data included in the first call request, the business data including a timeout duration field indicating the timeout duration; The business parsing method corresponding to the business data is invoked to parse the parameter value corresponding to the timeout duration field in order to obtain the first timeout duration.

7. The method as described in claim 5, characterized in that, Based on the first call request, the first timeout duration is obtained, including: Upon receiving the first call request, the call control attribute information of the first call request is obtained from the sender corresponding to the first call request. The call control attribute information is carried in the header field of the Hypertext Transfer Protocol or in the metadata of the Remote Procedure Call Protocol. The parameter value corresponding to the timeout duration field included in the call control attribute information is parsed to obtain the first timeout duration.

8. The method according to any one of claims 1 to 4, characterized in that, The method further includes: When initiating a second call request to the second microservice, record the time when the second call request is sent; During the invocation of the second invocation request, if it is determined that the difference between the current time and the time when the second invocation request was sent is greater than the second timeout duration, and no response from the second microservice to the second invocation request has been received, then it is determined that the response to the second invocation request has timed out. Based on the processed portion of the first call request, a response is returned to the sender corresponding to the first call request.

9. A remote procedure call control device, characterized in that, An apparatus for use in a microservice cluster, wherein the first microservice is any microservice in the microservice cluster, the apparatus comprising: The determining unit is configured to: determine the predicted call latency of calling the second microservice when the processing of the received first call request involves a call to the second microservice, wherein the predicted call latency is the predicted duration of data transmission path consumption during the call to the second microservice; determine the reduced duration corresponding to the second microservice based on the predicted call latency; and determine the second timeout duration corresponding to the second microservice based on the difference between the first timeout duration corresponding to the first call request and the processing time of the first call request, wherein the timeout time corresponding to the second timeout duration is earlier than the timeout time corresponding to the first timeout duration. The calling unit initiates a second call request to the second microservice based on the second timeout duration and the reduced duration; The timeout execution unit is used to process the first call request based on the lossy response returned by the second microservice in response to the second call request; the lossy response is the response returned by the second microservice based on the processed portion of the second call request when the second call request processing times out, determined by the second microservice subtracting the reduced time from the actual timeout time based on the second timeout timeout duration. The predicted call latency is determined using at least one of the following methods: The predicted call latency is determined based on the actual call latency consumed by the data transmission path when calling the second microservice during the same period in the historical cycle. The predicted call latency is determined based on the network topology information between the first microservice and the second microservice; Based on the first feature of the request packet that calls the second microservice, the second feature of the corresponding response packet is determined, and based on the first feature and the second feature, the predicted call latency is determined.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer storage medium storing computer program instructions thereon, characterized in that, When executed by a processor, the computer program instructions implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product comprising computer program instructions, characterized in that, When executed by a processor, the computer program instructions implement the steps of the method according to any one of claims 1 to 8.

Images