Request processing system and method and electronic equipment

Abstract

The invention provides a request processing system and method and electronic equipment, and relates to the field of computer software development, and the system comprises a service calling end, an asynchronous IO module and a service providing end; the service calling end is used for initiating an asynchronous remote procedure calling request for the to-be-processed target request to the service providing end, and the asynchronous remote procedure calling request is used for indicating the service providing end to process the target request; tracking a processing result of the target request according to the asynchronous handle, and determining a link of the next operation; the asynchronous IO module is used for generating an asynchronous handle according to the target request and returning the asynchronous handle to the service calling end; and the service providing end is used for responding to the asynchronous remote procedure calling request, processing the target request and returning a processing result of the target request to the service calling end through the asynchronous IO module. According to the method, the full-asynchronous architecture is introduced, and the I / O bottleneck problem existing in a traditional synchronous blocking model is effectively solved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer software development, and in particular to a request processing system, method and electronic device. Background Art

[0002] In modern Internet applications, users have increasing requirements for service performance and response speed, especially in the face of massive users and high concurrent requests. How to effectively handle these requests has become an important challenge for technology development. In related technologies, many systems still rely on traditional synchronous blocking models (BIO) and multi-threaded processing to handle high concurrent requests.

[0003] The traditional BIO model has several significant problems when processing high-concurrency requests. First, as the number of requests increases, the system needs to start more threads to handle these requests, resulting in a rapid increase in the number of threads. When the number of threads exceeds the number of CPU cores, multiple threads in the system will compete for limited CPU resources, resulting in frequent context switching. This not only increases the burden on the CPU, but also leads to increased response latency, because each context switch requires saving and restoring the state of the thread, consuming valuable computing resources. Second, in I / O-intensive application scenarios, thread blocking is particularly evident. When a thread is waiting for an I / O operation to complete, other requests cannot be processed in a timely manner, resulting in a decrease in system throughput. Especially in high-concurrency situations, the bottleneck of I / O operations will significantly affect the overall performance of the system and fail to meet users' needs for fast response. Summary of the invention

[0004] The embodiments of the present disclosure provide a request processing system, method and electronic device, aiming to solve the problems existing in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the present disclosure is implemented as follows:

[0006] In a first aspect, an embodiment of the present disclosure provides a request processing system, the system comprising a service calling end, an asynchronous IO module and a service providing end;

[0007] The service calling end is used to initiate an asynchronous remote procedure call request for a target request to be processed to the service providing end, wherein the asynchronous remote procedure call request is used to instruct the service providing end to process the target request; track the processing result of the target request according to the asynchronous handle, and determine the link for the next operation;

[0008] The asynchronous IO module is used to generate the asynchronous handle according to the target request, and return the asynchronous handle to the service call end;

[0009] The service provider is used to respond to the asynchronous remote procedure call request, process the target request, and return the processing result of the target request to the service caller through the asynchronous IO module.

[0010] Optionally, the service call end has multiple container instances deployed via the Kubernetes platform;

[0011] The Kubernetes platform evenly distributes the multiple requests obtained by the service call end to the multiple container instances, and the multiple container instances are used to process asynchronous remote procedure call requests for the multiple requests in parallel.

[0012] Optionally, the number of working threads of the multiple container instances is equal to the number of container CPU cores.

[0013] Optionally, the system further comprises a middleware module and a database;

[0014] The service provider is used to asynchronously call the middleware module and distribute multiple requests from the service caller to multiple business processing instances for parallel processing through the middleware module;

[0015] The business processing instance is used to asynchronously call the database to perform data access operations, and return the processing results to the service calling end through the asynchronous IO module.

[0016] Optionally, the system further includes a network terminal;

[0017] The network end is used to receive the asynchronous remote procedure call request initiated by the service calling end, and forward the asynchronous remote procedure call request to the service providing end to request the service providing end to process the target request.

[0018] In a second aspect, an embodiment of the present disclosure provides a request processing method, which is applied to the system described in the first aspect, and the method includes:

[0019] Obtain the target request to be processed through the service call end;

[0020] Generate an asynchronous handle according to the target request;

[0021] Initiating an asynchronous remote procedure call request for the target request to the service provider, wherein the asynchronous remote procedure call request is used to instruct the service provider to process the target request;

[0022] The processing result of the target request is tracked according to the asynchronous handle, and the link for the next operation is determined.

[0023] Optionally, generating an asynchronous handle according to the target request includes:

[0024] Parsing the content of the target request and extracting request parameters and context information;

[0025] Creating a corresponding asynchronous handle according to the request parameters and the context information;

[0026] The target request and the asynchronous handle corresponding to the target request are cached.

[0027] Optionally, the method further comprises:

[0028] Through the Kubernetes platform, the multiple requests obtained by the service call end are evenly distributed to multiple container instances, wherein the number of working threads of the multiple container instances is equal to the number of container CPU cores;

[0029] The asynchronous remote procedure call requests for the multiple requests are processed in parallel by the multiple container instances.

[0030] Optionally, the initiating an asynchronous remote procedure call request for the target request to the service provider includes:

[0031] Sending an asynchronous remote procedure call request for the target request to a network end through a container instance corresponding to the target request;

[0032] The asynchronous remote procedure call request initiated by the service calling end is received through the network end, and the asynchronous remote procedure call request is forwarded to the service providing end to request the service providing end to process the target request.

[0033] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory, and

[0034] A computer program stored in the memory and capable of running on the processor implements the steps of a request processing method when the computer program is executed by the processor.

[0035] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0036] The present disclosure introduces a fully asynchronous architecture, which effectively solves the I / O bottleneck problem existing in the traditional synchronous blocking model. First, by initiating an asynchronous remote procedure call request, the service call end can quickly initiate a target request to the service provider without blocking the current thread, so that the initiation and processing of the request are independent of each other, avoiding thread blocking caused by I / O operations, thereby significantly improving the response speed and resource utilization of the system. Secondly, according to the request corresponding to the generation of an asynchronous handle and tracking the processing results of the target request, the service call end can continue to process other requests while waiting for the processing results, maximizing the concurrent processing capabilities of the CPU. The present disclosure significantly improves the throughput and response speed of the system through a fully asynchronous link mode, so that it exhibits stronger stability and efficiency when facing high-concurrency requests, and meets the needs of modern Internet applications for fast response and high-concurrency processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0038] Figure 1 It is a schematic diagram of the BIO network programming model;

[0039] Figure 2 It is a schematic diagram of the NIO network programming model;

[0040] Figure 3 It is a schematic diagram of the Netty network programming model;

[0041] Figure 4 is an architectural block diagram of a request processing system provided by an embodiment of the present disclosure;

[0042] Figure 5 It is a schematic diagram of the steps of a request processing method provided by an embodiment of the present disclosure;

[0043] Figure 6 It is a schematic diagram of the overall flow of a request processing method provided by an embodiment of the present disclosure;

[0044] Figure 7 It is a schematic diagram of concurrent time-consuming verification in an embodiment of the present disclosure;

[0045] Figure 8 This is a schematic diagram of a single-machine QPS verification in an embodiment of the present disclosure;

[0046] Fig. 9 It is a schematic diagram of CPU performance verification in an embodiment of the present disclosure;

[0047] Fig.10 It is a schematic diagram of coding complexity verification in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0049] Figure 1 This is a schematic diagram of the BIO network programming model, such as Figure 1 As shown in the figure, in the BIO (Blocking I / O) network programming model, the server starts a separate thread for each client connection request to process. When the client initiates a connection request, the server assigns a thread to the request. This thread will be blocked until the request processing is completed or the I / O operation ends. Since each connection requires a thread, if there are a large number of connection requests, the server will consume a large amount of thread resources, resulting in increased thread overhead. A major flaw in the BIO network programming model is that when the number of threads reaches the maximum limit of the server, new connection requests will be rejected, causing the service to be unavailable. In addition, if a thread is blocked while processing a request (for example, waiting for an I / O operation), other requests will not be processed in time, affecting the response speed and throughput of the entire system. Therefore, the BIO model performs poorly in high-concurrency scenarios and is prone to performance bottlenecks.

[0050] Figure 2 This is a schematic diagram of the NIO network programming model, such as Figure 2 As shown in the figure, the NIO (Non-blocking I / O) network programming model is different from BIO in that NIO allows one thread to handle multiple connection requests. The client's connection request will be registered on a multiplexer (Selector), and the server checks the connection status by polling. In the NIO model, when an I / O request arrives, the Selector notifies the corresponding thread to process it. This method avoids the situation where each connection requires an independent thread, thereby reducing thread overhead and the frequency of context switching. Although the NIO model improves the user experience and avoids blocking at the access layer, in the actual processing process, if an I / O operation is encountered, the thread may still be blocked. Therefore, although NIO is better than BIO in performance, it may still face performance bottlenecks in high concurrency situations.

[0051] Figure 3This is a schematic diagram of the Netty network programming model, such as Figure 3 As shown in the figure, Netty is a high-performance network application framework based on NIO. It further encapsulates and optimizes NIO to simplify the complexity of network programming. Figure 3 The oval part represents some common capabilities handled by the Netty framework or SDK, including receiving (acceptor), reading (read), sending (send), decoding (decode) and encoding (encode). The Netty network programming model makes it easier for developers to build network applications. The cloud part represents the core processing part of the business logic of the service. Netty further optimizes the efficiency of resource usage by distinguishing between receiving and processing thread pools. The receiving thread focuses on processing connection requests, while the processing thread focuses on executing business logic.

[0052] In the above analysis of the BIO, NIO and Netty network programming models, we can see the different challenges and limitations faced by each model when processing connection requests and I / O operations. Since each connection in the BIO model requires an independent thread, it consumes too much resources in high concurrency situations, which easily causes performance bottlenecks. Although NIO allows one thread to handle multiple connection requests through a multiplexing mechanism, thereby reducing thread overhead, it may still encounter thread blocking problems when actually processing I / O operations, affecting the overall performance of the system. With the increase in the complexity of network applications and the amount of user concurrency, the traditional synchronous blocking and non-blocking models can no longer meet the performance requirements of modern high-concurrency scenarios. Therefore, Netty, as a high-performance framework based on NIO, improves the concurrent processing capabilities of the system through modular design and thread pool optimization, but there are still certain limitations, especially when processing I / O intensive tasks, thread blocking may still affect the response speed. This disclosure proposes a fully asynchronous link mode, which aims to completely get rid of the limitations of thread blocking in the traditional model. The fully asynchronous link mode asynchronously processes the operations of the server, business process, and client, ensuring that threads are not blocked due to I / O operations or the execution of business logic during the entire request processing process.

[0053] Figure 4 is an architectural block diagram of a request processing system provided by an embodiment of the present disclosure, such as Figure 4 As shown, the system includes a service calling end, an asynchronous IO module and a service providing end;

[0054] The service calling end is used to initiate an asynchronous remote procedure call request for a target request to be processed to the service providing end, and the asynchronous remote procedure call request is used to instruct the service providing end to process the target request; track the processing result of the target request according to the asynchronous handle, and determine the link for the next operation.

[0055] The service caller is the client or system component that initiates the request, and is responsible for sending a request to the service provider to obtain the required service or data. The target request is the request currently being processed among the multiple I / O requests received by the service caller. When the service caller needs to process the target request, it generates a corresponding asynchronous remote procedure call request (asynchronous RPC call request), which contains the necessary request information of the target request, such as the type, parameters, and address of the target service of the target request, so that the service provider can understand the target request.

[0056] An asynchronous remote procedure call request is a non-blocking request, that is, after initiating an asynchronous remote procedure call request, the service caller does not have to wait for the service provider to complete the processing and can continue to perform other tasks, thus avoiding resource waste and performance bottlenecks caused by waiting for a response. An asynchronous remote procedure call request contains a reference to an asynchronous handle, which is used to uniquely identify the target request and track the processing status and results of the target request in subsequent processing.

[0057] After receiving the asynchronous remote procedure call request, the service provider immediately starts to process the target request. It should be noted that the target request represents the specific operation that the user or other service wants to perform. After receiving the target request, the service caller initiates an asynchronous remote procedure call request to the network end to request the service provider to process the target request. At this time, the target request and the asynchronous remote procedure call request are two different requests. The former is the specific operation requirement of the user, while the latter is a request sent by the service caller to the network end to forward the target request to the service provider for processing. Due to the full asynchronous mode, the service provider can respond to other requests while processing the target request, thereby improving the concurrent processing capability of the system. During the processing, the service provider will not block any thread, but convert the I / O operation (such as database query, external service call, etc.) into an asynchronous operation. When the service provider initiates an I / O request, it immediately returns control to allow other requests to be processed.

[0058] When processing a request, the service provider associates the processing result with the asynchronous handle. When the processing of the target request is completed, the service provider returns the result to the service caller through the asynchronous handle. The service caller can use the asynchronous handle to track the processing status of the request and perform subsequent operations when the result is available. In this way, the service caller can obtain the processing result through query or callback without blocking and waiting. When the service caller receives the asynchronous handle, it determines the next operation link. The next operation link includes but is not limited to the following situations: continue to process other requests, execute specific business logic according to the processing results, feedback the results to the user, or update the interface.

[0059] The asynchronous IO module is used to generate the asynchronous handle according to the target request and return the asynchronous handle to the service call end.

[0060] The asynchronous IO module is responsible for processing asynchronous remote procedure call requests from the service caller and generating corresponding asynchronous handles, aiming to ensure the non-blocking nature of I / O operations to improve the concurrent processing capability and response speed of the system. When the service caller initiates a target request, the asynchronous IO module receives the target request and generates a unique asynchronous handle based on the request parameters and context information. The asynchronous handle is an identifier, an object or data structure that contains the request parameters of the request, such as the request ID, status, timestamp, etc., and is used to track the processing progress and results of the request.

[0061] When the asynchronous handle is generated, the asynchronous IO module returns it to the service caller. At this point, the service caller can use this handle to track and manage subsequent operations of the request. The returned asynchronous handle can contain methods or properties, allowing the service caller to query the status of the request (such as "processing", "completed", "failed", etc.) or obtain the final processing result. In implementation, the asynchronous IO module can use event-driven programming, callback functions, or Promise / Future mode to ensure the asynchronous nature of I / O operations. The asynchronous IO module also needs to interact with the underlying I / O system (such as databases, file systems, network connections, etc.) to ensure that the main thread is not blocked when performing I / O operations.

[0062] The service provider is used to respond to the asynchronous remote procedure call request, process the target request, and return the processing result of the target request to the service caller through the asynchronous IO module.

[0063] The service provider is a system component that responds to the asynchronous remote procedure call request initiated by the service caller. It is responsible for executing the specific business logic for the target request and returning the processing result of the target request to the service caller. When the service provider receives the asynchronous remote procedure call request, it parses the request content of the asynchronous remote procedure call request to understand the specific requirements of the target request, including the request type, parameters, address of the target service and other information.

[0064] In the process of processing asynchronous remote procedure call requests, the service provider executes the corresponding business logic, including but not limited to the following business logic: accessing the database to obtain or store data, calling other microservices or external APIs to obtain the required information, and performing calculations or data processing tasks.

[0065] In order to achieve high concurrency and non-blocking processing, the service provider will design all operations that may cause blocking (such as I / O operations) to be executed asynchronously. That is, when the service provider initiates an I / O request, it will not wait for the request to complete, but will immediately return control to allow other requests to be processed. Once the processing of the target request is completed, the service provider will return the processing result to the service caller through the asynchronous IO module. Specifically, the service provider will encapsulate the processing result (such as success status, return data, or error information) into a response object. The response object will be associated with the previously generated asynchronous handle so that the service caller can recognize and process the result. The service provider calls the asynchronous IO module and sends the response object back to the service caller. At this time, the asynchronous IO module is responsible for passing the result to the corresponding service caller.

[0066] It can be understood that the asynchronous IO module plays a bridging role in this process, ensuring that the communication between the service provider and the service caller is non-blocking.

[0067] The present disclosure introduces a fully asynchronous architecture, which effectively solves the I / O bottleneck problem existing in the traditional synchronous blocking model. First, by initiating an asynchronous remote procedure call request, the service call end can quickly initiate a target request to the service provider without blocking the current thread, so that the initiation and processing of the request are independent of each other, avoiding thread blocking caused by I / O operations, thereby significantly improving the response speed and resource utilization of the system. Secondly, according to the request corresponding to the generation of an asynchronous handle and tracking the processing results of the target request, the service call end can continue to process other requests while waiting for the processing results, maximizing the concurrent processing capabilities of the CPU. The present disclosure significantly improves the throughput and response speed of the system through a fully asynchronous link mode, so that it exhibits stronger stability and efficiency when facing high-concurrency requests, and meets the needs of modern Internet applications for fast response and high-concurrency processing.

[0068] In an optional implementation, the service call end is deployed with multiple container instances through the Kubernetes platform;

[0069] The Kubernetes platform evenly distributes the multiple requests obtained by the service call end to the multiple container instances, and the multiple container instances are used to process asynchronous remote procedure call requests for the multiple requests in parallel.

[0070] In order to improve the availability and concurrent processing capabilities of the system, the service caller deploys multiple container instances on the Kubernetes platform. Each container instance runs the same service caller application and can process requests independently. In this way, the load can be shared among multiple containers to ensure that the system can still maintain good performance under high concurrency. Kubernetes is an open source container orchestration platform that is responsible for automating the deployment, expansion and management of containerized applications. Kubernetes provides powerful scheduling and management functions, enabling multiple container instances to collaborate efficiently. In this embodiment, the Kubernetes platform is used to receive requests from the service caller and evenly distribute these requests to multiple container instances. When the service caller receives multiple asynchronous remote procedure call requests, the Kubernetes platform distributes these requests to different container instances according to the load balancing strategy. This distribution can be based on polling, random, minimum number of connections and other strategies to ensure that the load of each container instance is relatively balanced. The evenly distributed strategy can effectively avoid the performance bottleneck caused by too many requests to a container instance, thereby improving the response speed and processing power of the overall system.

[0071] When a request is assigned to a specific container instance, the container instance processes these asynchronous remote procedure call requests in parallel. Since each container instance is independent, they can process multiple requests at the same time without interfering with each other. During the processing, the container instance uses the asynchronous IO module to perform I / O operations to ensure that other requests are not blocked while waiting for I / O to complete, allowing the system to maintain high throughput in high concurrency situations.

[0072] The Kubernetes platform supports dynamic expansion and reduction of the number of container instances. When the number of requests increases, more container instances can be automatically added according to the load to cope with high concurrency requirements; when the number of requests decreases, the number of container instances can be reduced to save resources, allowing the service caller to flexibly respond to different traffic conditions and ensure that the system can always run efficiently. Kubernetes also provides monitoring and management functions that can monitor the performance, health status and resource usage of container instances in real time. By monitoring data, operation and maintenance personnel can promptly discover and solve potential problems to ensure the stability of the system.

[0073] In the disclosed embodiments, the deployment and management of the service call end on the Kubernetes platform achieves efficient request processing capabilities and can maintain good performance and resource utilization efficiency in high-concurrency scenarios.

[0074] In an optional implementation, the number of working threads of the multiple container instances is equal to the number of container CPU cores.

[0075] The number of worker threads refers to the number of threads that can be executed simultaneously in each container instance. Threads are the basic unit for processing requests, and multiple threads can execute tasks in parallel, thereby improving the throughput of the system.

[0076] The number of CPU cores refers to the number of CPU cores allocated to the container instance. Each CPU core can execute one thread at the same time, so the number of CPU cores directly affects the parallel processing capability of the container instance.

[0077] In the fully asynchronous link mode of the present disclosure, the number of worker threads of multiple container instances is equal to the number of CPU cores of the container, avoiding thread context switching. If the number of worker threads exceeds the number of CPU cores, the system will frequently perform thread context switching, resulting in performance degradation. Each switch requires saving and restoring the state of the thread, which consumes CPU resources. By setting the number of worker threads to the number of CPU cores, it can be ensured that each CPU core is fully utilized without wasting resources due to too many threads.

[0078] Assume that a container instance of a service call end is deployed in the Kubernetes cluster, and each container instance is allocated 4 CPU cores (i.e., 4-core CPU). According to the design principles of the present disclosure, the number of worker threads is set to 4, that is, each container instance will create 4 worker threads. Assume that 10 asynchronous remote procedure call requests arrive at the container instance at the same time. The 4 worker threads of the container instance will be activated and start processing the requests. Since the number of worker threads is equal to the number of CPU cores, 4 threads can process 4 requests in parallel. The first request is processed by thread 1, the second request is processed by thread 2, the third request is processed by thread 3, and the fourth request is processed by thread 4. When processing the remaining requests, when the first request is processed, thread 1 will immediately process the fifth request. Thread 2, thread 3, and thread 4 will also process subsequent requests in turn until all requests are processed.

[0079] During the processing, if a request involves an I / O operation (such as a database query), the thread will initiate an asynchronous I / O request and return control immediately, allowing other threads to continue processing the remaining requests without having to wait for the I / O to complete.

[0080] In an optional embodiment, the system further includes a middleware module and a database;

[0081] The service provider is used to asynchronously call the middleware module and distribute multiple requests from the service caller to multiple business processing instances for parallel processing through the middleware module;

[0082] The business processing instance is used to asynchronously call the database to perform data access operations, and return the processing results to the service calling end through the asynchronous IO module.

[0083] As mentioned above, the service provider is used to respond to the asynchronous remote procedure call request initiated by the service caller, and distributes the request by asynchronously calling the middleware module. As the request distribution center, the middleware module is responsible for distributing multiple requests from the service provider to multiple business processing instances to achieve parallel processing. The business processing instance is responsible for specific business logic processing and asynchronously calls the database to access data. The processing result will be returned to the service provider. The database is used to store business data and supports business processing instances to read and write data.

[0084] When the service caller initiates multiple asynchronous requests, the service provider receives these requests, calls the middleware module asynchronously, and passes the multiple requests to the middleware module. Because it is an asynchronous call, the service provider will not block waiting for the response of the middleware module, but can continue to process other tasks.

[0085] After receiving the request from the service provider, the middleware module distributes these requests to multiple business processing instances according to the load balancing strategy (such as polling, random or least connections). This distribution can be dynamic, allowing the middleware to perform intelligent scheduling based on the load of the current business processing instance. After each business processing instance receives the assigned request, it starts to execute the corresponding business logic. During the processing process, if it is necessary to access the database (such as reading or writing data), the business processing instance will initiate an asynchronous call to the database through the asynchronous IO module. Similarly, after initiating a database request, the business processing instance does not have to wait for the database's response, but can continue to process other tasks or requests.

[0086] After receiving the request, the database performs the corresponding data access operation (such as query, insert, update, etc.). When the database operation is completed, the database returns the result to the corresponding business processing instance. After receiving the response from the database, the business processing instance completes the corresponding business logic processing and returns the processing result to the service provider through the asynchronous IO module.

[0087] In an optional implementation, the system further includes a network end;

[0088] The network end is used to receive the asynchronous remote procedure call request initiated by the service calling end, and forward the asynchronous remote procedure call request to the service providing end to request the service providing end to process the target request.

[0089] The network end is responsible for monitoring and receiving asynchronous remote procedure call requests from the service call end. The network end forwards the received asynchronous remote procedure call requests to the service provider end to ensure that the requests can be processed correctly. In the disclosed embodiment, the network end can also process the response from the service provider end and return it to the service call end.

[0090] After the service caller initiates an asynchronous remote procedure call request, the network end listens to a specific port through a network protocol (such as HTTP, TCP, etc.) and waits to receive a request from the service caller. When the request arrives, the network end immediately receives the asynchronous remote procedure call request and forwards it to the service provider.

[0091] When forwarding asynchronous remote procedure call requests, the network can combine load balancing strategies (such as polling, random, etc.) to distribute asynchronous remote procedure call requests to multiple business processing instances on the service provider side to improve the system's processing capabilities. The network will also receive processing results from the service provider, especially when the results need to be returned to the service caller. If the network receives a response from the service provider, it returns the processing results to the service caller that originally initiated the request, completing the entire request processing process.

[0092] Figure 5 is a schematic diagram of the steps of a request processing method provided by an embodiment of the present disclosure, such as Figure 5 As shown, the method is applied to the request processing system as described above, and the method includes:

[0093] Step S101, obtaining a target request to be processed through a service call end.

[0094] As the initiator of the request, the service caller is responsible for receiving and processing requests from users or other system components. Specifically, the service caller receives multiple I / O requests, including the target request currently to be processed. The multiple I / O requests contain specific operations or data requests that the user wants to perform. The service caller selects the target request from the received requests for processing based on the business logic and the current processing status.

[0095] Step S102: Generate an asynchronous handle according to the target request.

[0096] The service caller calls the asynchronous IO module and passes the content of the target request. The asynchronous IO module generates a unique asynchronous handle based on the content of the target request. After the asynchronous handle is generated, the asynchronous IO module returns it to the service caller. The service caller saves the handle for subsequent use. The service caller uses the asynchronous handle to track the processing progress and results of the target request to ensure that it can respond to the needs of subsequent operations in a timely manner.

[0097] Step S103: Initiate an asynchronous remote procedure call request for the target request to the service provider, wherein the asynchronous remote procedure call request is used to instruct the service provider to process the target request.

[0098] The service caller constructs an asynchronous remote procedure call request based on the target request content and the asynchronous handle. The asynchronous remote procedure call request contains information such as the target request type, parameters, and address of the target service, ensuring that the service provider can understand and process the request. The service caller sends the constructed asynchronous remote procedure call request to the service provider through the network. After receiving the request from the service caller, the network immediately forwards it to the service provider for processing to ensure that the request can be processed in a timely manner.

[0099] Step S104: Track the processing result of the target request according to the asynchronous handle, and determine the link for the next operation.

[0100] The service caller uses the asynchronous handle to track the processing results of the target request and determines the subsequent operations based on the results. Specifically, the service caller uses the asynchronous handle to query the processing status of the target request. The status of the request can be checked through the methods or properties provided by the asynchronous IO module, such as "processing", "completed", "failed", etc. When the processing of the target request is completed, the service provider associates the processing result with the asynchronous handle and returns it to the service caller through the asynchronous IO module. The service caller can use the asynchronous handle to obtain the final processing result. Based on the processing result, the service caller determines the link for the next operation, which may include continuing to process other requests, executing specific business logic based on the processing results, feeding back results to the user, or updating the interface. The service caller flexibly adjusts subsequent operations according to business needs to ensure efficient operation of the system.

[0101] In an optional implementation, generating an asynchronous handle according to the target request includes: parsing the content of the target request to extract request parameters and context information; creating a corresponding asynchronous handle according to the request parameters and the context information; and caching the target request and the asynchronous handle corresponding to the target request.

[0102] After receiving the target request, the service caller parses the target request to extract the necessary information to ensure that the subsequent asynchronous handle can accurately reflect the requirements of the request. Specifically, the service caller parses the content of the target request, including decoding the format of the target request (such as JSON, XML, etc.) and reading each field therein. From the parsed request content, the request parameters of the target request are extracted. The request parameters include the request type, input data, and the target service address. In addition to the request parameters, the service caller also needs to extract context information to help better understand the background and execution environment of the request. The context information includes user information, session information, and timestamp. The service caller will create an asynchronous handle based on the extracted request parameters and context information. The service caller calls the asynchronous IO module and passes the extracted request parameters and context information. The asynchronous IO module generates a unique asynchronous handle based on this information. When creating an asynchronous handle, it is necessary to ensure the uniqueness of each handle to avoid confusion between different requests. This can be achieved by generating a random ID, using a timestamp, or other unique identification methods.

[0103] The service caller caches the target request and its corresponding asynchronous handle, which can be stored in memory or in a database for subsequent processing and status tracking. Stored in memory or in a database for quick access when needed. During the caching process, the generated asynchronous handle needs to be associated with the target request and stored in a data structure (such as a mapping table or dictionary). When caching asynchronous handles, its lifecycle management also needs to be considered. For example, when the request processing is completed, the cache should be cleared in time to avoid memory leaks or cache expiration issues.

[0104] Through the above steps, the service caller can effectively generate and manage asynchronous handles, thereby achieving non-blocking processing and status tracking of target requests, ensuring efficient response to user requests, and improving the overall concurrent processing capability.

[0105] In an optional implementation, the method further includes: through the Kubernetes platform, evenly distributing the multiple requests obtained by the service call end to multiple container instances, wherein the number of working threads of the multiple container instances is equal to the number of container CPU cores; and processing asynchronous remote procedure call requests for the multiple requests in parallel through the multiple container instances.

[0106] The Kubernetes platform evenly distributes the multiple requests obtained by the service call end to the multiple container instances, and the multiple container instances are used to process asynchronous remote procedure call requests for the multiple requests in parallel.

[0107] In order to improve the availability and concurrent processing capabilities of the system, the service caller deploys multiple container instances on the Kubernetes platform. Each container instance runs the same service caller application and can process requests independently. In this way, the load can be shared among multiple containers to ensure that the system can still maintain good performance under high concurrency. Kubernetes is an open source container orchestration platform that is responsible for automating the deployment, expansion and management of containerized applications. Kubernetes provides powerful scheduling and management functions, enabling multiple container instances to collaborate efficiently. In this embodiment, the Kubernetes platform is used to receive requests from the service caller and evenly distribute these requests to multiple container instances. When the service caller receives multiple asynchronous remote procedure call requests, the Kubernetes platform distributes these requests to different container instances according to the load balancing strategy. This distribution can be based on polling, random, minimum number of connections and other strategies to ensure that the load of each container instance is relatively balanced. The evenly distributed strategy can effectively avoid the performance bottleneck caused by too many requests to a container instance, thereby improving the response speed and processing power of the overall system.

[0108] When a request is assigned to a specific container instance, the container instance processes these asynchronous remote procedure call requests in parallel. Since each container instance is independent, they can process multiple requests at the same time without interfering with each other. During the processing, the container instance uses the asynchronous IO module to perform I / O operations to ensure that other requests are not blocked while waiting for I / O to complete, allowing the system to maintain high throughput in high concurrency situations.

[0109] The Kubernetes platform supports dynamic expansion and reduction of the number of container instances. When the number of requests increases, more container instances can be automatically added according to the load to cope with high concurrency requirements; when the number of requests decreases, the number of container instances can be reduced to save resources, allowing the service caller to flexibly respond to different traffic conditions and ensure that the system can always run efficiently. Kubernetes also provides monitoring and management functions that can monitor the performance, health status and resource usage of container instances in real time. By monitoring data, operation and maintenance personnel can promptly discover and solve potential problems to ensure the stability of the system.

[0110] In the disclosed embodiments, the deployment and management of the service call end on the Kubernetes platform achieves efficient request processing capabilities and can maintain good performance and resource utilization efficiency in high-concurrency scenarios.

[0111] In the fully asynchronous link mode of the present disclosure, the number of worker threads of multiple container instances is equal to the number of CPU cores of the container, avoiding thread context switching. If the number of worker threads exceeds the number of CPU cores, the system will frequently perform thread context switching, resulting in performance degradation. Each switch requires saving and restoring the state of the thread, which consumes CPU resources. By setting the number of worker threads to the number of CPU cores, it can be ensured that each CPU core is fully utilized without wasting resources due to too many threads.

[0112] During the processing, if a request involves an I / O operation (such as a database query), the thread will initiate an asynchronous I / O request and return control immediately, allowing other threads to continue processing the remaining requests without having to wait for the I / O to complete.

[0113] In an optional implementation, the initiating an asynchronous remote procedure call request for the target request to the service provider includes: sending the asynchronous remote procedure call request for the target request to the network end through the container instance corresponding to the target request; receiving the asynchronous remote procedure call request initiated by the service call end through the network end, and forwarding the asynchronous remote procedure call request to the service provider to request the service provider to process the target request.

[0114] The service caller sends the prepared asynchronous remote procedure call request to the network end through its corresponding container instance.

[0115] The network side, as an intermediate layer, receives asynchronous remote procedure call requests from the service call side and forwards them to the service provider side. The network side has a monitoring mechanism that can process and receive requests from different service call sides. After receiving the asynchronous remote procedure call request, the network side will parse the asynchronous remote procedure call request to extract the request type, request parameters, target service address and asynchronous handle reference of the target request. The network side forwards the parsed asynchronous remote procedure call request to the corresponding service provider side.

[0116] After receiving the forwarded asynchronous remote procedure call request, the service provider starts to process the target request. The service provider will execute the corresponding business logic according to the content of the request, including accessing the database and calling other services.

[0117] Through the above steps, the service caller can effectively initiate an asynchronous remote procedure call request to the service provider. The entire process realizes the non-blocking nature of the request. After initiating the request, the service caller can continue to perform other tasks without waiting for the response from the service provider. The concurrent processing capability and response speed are improved, so that a large number of requests can be processed more efficiently in high-concurrency scenarios.

[0118] Figure 6 is a schematic diagram of the overall flow of a request processing method provided by an embodiment of the present disclosure, such as Figure 6 As shown, in this process, first, the service caller initiates an asynchronous remote procedure call request for the target request and sends it to the network end. Then, the asynchronous IO module returns an asynchronous handle to the service caller for subsequent request tracking. Subsequently, the service caller caches the target request and its response context for subsequent access. Next, the network end is responsible for forwarding the received asynchronous remote procedure call request to the service provider, requesting it to process the target request. After receiving the request, the service provider executes the corresponding business logic and asynchronously calls the middleware module to complete the processing. After the processing is completed, the service provider sends a response to the asynchronous IO module. Finally, the asynchronous IO module returns the processing result of the target request to the service caller, completing the entire asynchronous processing flow.

[0119] The verification embodiment of the present disclosure uses a Linux operating system and an 8-core CPU to start a single instance for continuous stress testing to evaluate the performance of the fully asynchronous high-concurrency service solution.

[0120] Figure 7 This is a schematic diagram of verification results of a request processing system provided by an embodiment of the present disclosure. Please refer to Figure 7 , the upper broken line is the performance of the synchronous model, and the lower broken line is the performance of the asynchronous model. In the traditional synchronous programming model before optimization, as the number of concurrent requests increases, the system response time increases significantly. This is because the traditional model will cause thread blocking when processing requests, resulting in a waste of system resources and a prolonged response time. After implementing the fully asynchronous solution disclosed in this disclosure, the system can effectively avoid thread blocking, significantly reduce the response time, and ensure that the response time can remain low under high concurrency conditions.

[0121] Figure 8 This is a schematic diagram of the single-machine QPS verification in the embodiment of the present disclosure. Please refer to Figure 8 , the left column pair in the figure shows the portrait interface, and the right column pair in the figure shows the IDMapping. In the two column pairs, the left column chart shows the performance of the synchronous model, and the right column chart shows the performance of the asynchronous model. It can be seen that by implementing the solution disclosed in the present invention, the performance of a single machine has increased by nearly 6 times in IO-intensive capabilities, reaching 10,000 QPS. This result is much higher than the performance of the traditional synchronous programming model. The QPS calculation formula of the traditional model is:

[0122]

[0123] In the implementation data, assuming that the average interface time consumption is 50ms and the number of threads is 100, the calculated QPS is 2000. This shows that after adopting the fully asynchronous solution disclosed in the present invention, the throughput of the system when processing IO-intensive requests is greatly improved.

[0124] Fig. 9 This is a schematic diagram of CPU performance verification in the embodiment of the present disclosure. Please refer to Fig. 9 , the broken line on the lower side is the performance of the asynchronous model, and the broken line in the middle is the performance of the synchronous model. The fully asynchronous solution disclosed in this disclosure significantly reduces the CPU usage by eliminating the switching and waiting consumption between threads. In the traditional synchronous model, due to the frequent creation and destruction of threads, as well as the time waiting for IO operations to complete, a large amount of CPU resources are occupied. After adopting the fully asynchronous solution, the CPU load is effectively reduced, system resources can be used more efficiently, and the overall performance is improved.

[0125] Fig.10 This is a schematic diagram of coding complexity verification in the embodiment of the present disclosure, please refer to Fig.10 Although JDK19 will soon support coroutines and may bring certain performance improvements, the fully asynchronous solution disclosed in this disclosure is still better than the coroutine implementation in terms of performance. This is because the implementation of coroutines requires recording IO interruption points in the heap, which increases coding complexity and runtime overhead. However, this disclosure uses a fully asynchronous link mode to manage IO operations more efficiently, reduce unnecessary overhead, and thus have more advantages in performance.

[0126] In summary, the present disclosure significantly reduces the time consumption of request processing, especially maintaining a stable response time under high concurrency. At the same time, it improves the QPS of a single instance to 10,000 / s, far exceeding the performance of the traditional synchronization model, and also reduces the CPU usage and optimizes the efficiency of resource utilization. In comparison with the upcoming JDK19 coroutine, it shows better performance.

[0127] An embodiment of the present disclosure also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes in the above-mentioned request processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0128] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, devices, electronic devices, and storage media. Therefore, the embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] The embodiments of the present disclosure are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0130] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present disclosure.

[0131] Finally, it should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. A request processing system, method and electronic device provided by the present disclosure are described in detail above. The principles and implementation methods of the present disclosure are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present disclosure; at the same time, for those of ordinary skill in the art, according to the idea of ​​the present disclosure, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A request processing system, characterized in that: The system includes a service calling end, an asynchronous IO module and a service providing end; The service calling end is used to initiate an asynchronous remote procedure call request for a target request to be processed to the service providing end, wherein the asynchronous remote procedure call request is used to instruct the service providing end to process the target request; Track the processing result of the target request according to the asynchronous handle, and determine the link for the next operation; The asynchronous IO module is used to generate the asynchronous handle according to the target request, and return the asynchronous handle to the service call end; The service provider is used to respond to the asynchronous remote procedure call request, process the target request, and return the processing result of the target request to the service caller through the asynchronous IO module.

2. The system according to claim 1, characterized in that The service call end has multiple container instances deployed through the Kubernetes platform; The Kubernetes platform evenly distributes the multiple requests obtained by the service call end to the multiple container instances, and the multiple container instances are used to process asynchronous remote procedure call requests for the multiple requests in parallel.

3. The system according to claim 2, characterized in that The number of worker threads of the multiple container instances is equal to the number of container CPU cores.

4. The system according to claim 1, characterized in that The system also includes a middleware module and a database; The service provider is used to asynchronously call the middleware module and distribute multiple requests from the service caller to multiple business processing instances for parallel processing through the middleware module; The business processing instance is used to asynchronously call the database to perform data access operations, and return the processing results to the service calling end through the asynchronous IO module.

5. The system according to claim 1, characterized in that The system also includes a network end; The network end is used to receive the asynchronous remote procedure call request initiated by the service calling end, and forward the asynchronous remote procedure call request to the service providing end to request the service providing end to process the target request.

6. A request processing method, characterized in that: Applied to the system according to any one of claims 1 to 5, the method comprises: Obtain the target request to be processed through the service call end; Generate an asynchronous handle according to the target request; Initiating an asynchronous remote procedure call request for the target request to the service provider, wherein the asynchronous remote procedure call request is used to instruct the service provider to process the target request; The processing result of the target request is tracked according to the asynchronous handle, and the link for the next operation is determined.

7. The method according to claim 6, characterized in that Generating an asynchronous handle according to the target request includes: Parsing the content of the target request and extracting request parameters and context information; Creating a corresponding asynchronous handle according to the request parameters and the context information; The target request and the asynchronous handle corresponding to the target request are cached.

8. The method according to claim 6, characterized in that The method further comprises: Through the Kubernetes platform, the multiple requests obtained by the service call end are evenly distributed to multiple container instances, wherein the number of working threads of the multiple container instances is equal to the number of container CPU cores; The asynchronous remote procedure call requests for the multiple requests are processed in parallel by the multiple container instances.

9. The method according to claim 8, characterized in that The initiating an asynchronous remote procedure call request for the target request to the service provider includes: Sending an asynchronous remote procedure call request for the target request to a network end through a container instance corresponding to the target request; The asynchronous remote procedure call request initiated by the service calling end is received through the network end, and the asynchronous remote procedure call request is forwarded to the service providing end to request the service providing end to process the target request.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method according to any one of claims 6 to 9 are implemented.

Images