A request processing method and device

Abstract

The present disclosure provides a request processing method and device. The embodiments of the present disclosure process service call requests through multi-threading, thereby utilizing different threads to run cached codes, thereby preventing a single process error from causing all services to be paralyzed. In addition, through shared memory and queues, the communication efficiency between multiple processes is improved, thereby ensuring the startup efficiency of the serverless computing platform.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a request processing method and device. Background Art

[0002] Serverless computing is a cloud computing model. Based on Platform as a Service (PaaS), the serverless computing platform provides infrastructure so that users can publish their own applications without deploying, configuring and managing servers. The operating environment of the program is provided by the platform. The client can send a service call request to the serverless computing platform to call the code on the serverless computing platform.

[0003] Typically, serverless computing platforms are implemented based on managed language virtual machines. Managed languages ​​are programming languages ​​that are hosted and run in a runtime environment, and whose execution is managed and controlled by the runtime environment. Managed language virtual machines run user code through a single-process, multi-instance approach, with multiple instances running in a single process, thereby reducing instance performance overhead and startup delays. However, in this way, errors in different instances within a process are difficult to isolate. In the same process, if an instance generates an unrecoverable error, all other instances will be forced to shut down. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a request processing method and device.

[0005] In a first aspect, an embodiment of the present disclosure provides a request processing method, including:

[0006] In response to a service call request sent by a client, a target work process is allocated for the service call request from multiple work processes, and the service call request is written into a submission queue corresponding to the target work process;

[0007] Using the target working process to read the service call request from the submission queue, and searching for memory address information of the cache code matching the service call request from the shared memory;

[0008] Using the target working process, processing the service call request based on the memory address information of the cache code found from the shared memory, and writing the response data obtained by processing the service call request into the completion queue corresponding to the target working process;

[0009] Reading the response data in the completion queue and sending the response data to the client;

[0010] The submission queue and the completion queue are stored in the shared memory.

[0011] In an optional implementation manner, allocating a target work process for the service call request from multiple work processes, and writing the service call request to a submission queue corresponding to the target work process, includes:

[0012] Based on the load status of the multiple work processes, select a target work process assigned to the service call request from the multiple work processes;

[0013] The service call request is written into the submission queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0014] In an optional implementation manner, the using the target working process to read the service call request from the submission queue and searching for a cache code matching the service call request from a shared memory includes:

[0015] Using a fast user space mutual exclusion mechanism, reading the service call request from the submission queue;

[0016] Utilizing a fast user space mutual exclusion mechanism, based on the request-cache mapping relationship stored in the shared memory, the memory address information of the cache code that matches the service call request is searched from the shared memory; wherein the request-cache mapping relationship includes a key-value pair with the identification information of the service call request as the key and the memory address information of the cache code that matches the service call request as the value.

[0017] In an optional implementation, the using the target working process to process the service call request based on the memory address information of the cache code found from the shared memory, and writing the response data obtained by processing the service call request into the completion queue corresponding to the target working process, includes:

[0018] Using the thread under the target working process to establish an instance of the cache code;

[0019] Using the memory address information of the cache code and the service call request, the cache code is run in the instance of the cache code to obtain the response data;

[0020] The response data is written into the completion queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0021] In an optional implementation manner, reading the response data in the completion queue and sending the response data to the client includes:

[0022] Using a fast user space mutual exclusion mechanism, reading the response data in the completion queue;

[0023] The response data is sent to the client.

[0024] In an optional implementation, the using a fast user space mutual exclusion mechanism to search the shared memory for memory address information of a cache code matching the service call request based on the request-cache mapping relationship stored in the shared memory includes:

[0025] Adding a mutual exclusion lock to the shared memory; the mutual exclusion lock is implemented based on the fast user space mutual exclusion mechanism;

[0026] Searching the request-cache mapping relationship from the shared memory after adding the mutex lock;

[0027] Finding the memory address information of the cache code matching the service call request from the request-cache mapping relationship, and updating the reference count of the found memory address information of the cache code;

[0028] Release the mutex lock of the shared memory.

[0029] In an optional implementation manner, the method further includes:

[0030] If the memory address information of the cache code matching the service call request is not found, requesting the cache code matching the service call request from the code database;

[0031] Using a fast user space mutual exclusion mechanism, writing the cache code matching the service call request into the shared memory, and updating the request-cache mapping relationship;

[0032] The instruction of searching the shared memory for the memory address information of the cache code matching the service call request is executed again.

[0033] In an optional implementation manner, the method further includes:

[0034] Determine the timestamp of the last use of each cache code stored in the shared memory with a preset time period as a period;

[0035] If the time interval between the timestamp of any cached code and the current time node is greater than or equal to the preset cleaning time, the fast user space mutual exclusion mechanism is used to delete the any cached code from the shared memory.

[0036] In an optional implementation manner, the method further includes:

[0037] When an error occurs in any thread in the target working process, adding a write lock to the target working process;

[0038] After the threads other than the any one thread in the target working process release the lock added to the shared memory, the target working process is closed.

[0039] In a second aspect, the present disclosure also provides a request processing device, including:

[0040] An allocation module, configured to respond to a service call request sent by a client, allocate a target work process to the service call request from multiple work processes, and write the service call request to a submission queue corresponding to the target work process;

[0041] A search module, used to read the service call request from the submission queue using the target working process, and search for memory address information of a cache code matching the service call request from a shared memory;

[0042] A processing module, used to use the target working process to process the service call request based on the memory address information of the cache code found from the shared memory, and write the response data obtained by processing the service call request into the completion queue corresponding to the target working process;

[0043] A sending module, used for reading the response data in the completion queue and sending the response data to the client;

[0044] The submission queue and the completion queue are stored in the shared memory.

[0045] In an optional implementation manner, when the allocation module allocates a target work process to the service call request from multiple work processes and writes the service call request to a submission queue corresponding to the target work process, it is used to:

[0046] Based on the load status of the multiple work processes, select a target work process assigned to the service call request from the multiple work processes;

[0047] The service call request is written into the submission queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0048] In an optional implementation manner, the search module is specifically used to:

[0049] Using a fast user space mutual exclusion mechanism, reading the service call request from the submission queue;

[0050] Utilizing a fast user space mutual exclusion mechanism, based on the request-cache mapping relationship stored in the shared memory, the memory address information of the cache code that matches the service call request is searched from the shared memory; wherein the request-cache mapping relationship includes a key-value pair with the identification information of the service call request as the key and the memory address information of the cache code that matches the service call request as the value.

[0051] In an optional implementation manner, the processing module is specifically used to:

[0052] Using the thread under the target working process to establish an instance of the cache code;

[0053] Using the memory address information of the cache code and the service call request, the cache code is run in the instance of the cache code to obtain the response data;

[0054] The response data is written into the completion queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0055] In an optional implementation manner, the sending module is specifically used to:

[0056] Using a fast user space mutual exclusion mechanism, reading the response data in the completion queue;

[0057] The response data is sent to the client.

[0058] In an optional implementation, the search module uses a fast user space mutual exclusion mechanism to search the shared memory for memory address information of the cache code matching the service call request based on the request-cache mapping relationship stored in the shared memory, and is used to:

[0059] Adding a mutual exclusion lock to the shared memory; the mutual exclusion lock is implemented based on the fast user space mutual exclusion mechanism;

[0060] Searching the request-cache mapping relationship from the shared memory after adding the mutex lock;

[0061] Finding the memory address information of the cache code matching the service call request from the request-cache mapping relationship, and updating the reference count of the found memory address information of the cache code;

[0062] Release the mutex lock of the shared memory.

[0063] In an optional implementation manner, the device further includes a request module, configured to:

[0064] If the memory address information of the cache code matching the service call request is not found, requesting the cache code matching the service call request from the code database;

[0065] Using a fast user space mutual exclusion mechanism, writing the cache code matching the service call request into the shared memory, and updating the request-cache mapping relationship;

[0066] The instruction of searching the shared memory for the memory address information of the cache code matching the service call request is executed again.

[0067] In an optional implementation manner, the device further includes a deletion module, configured to:

[0068] Determine the timestamp of the last use of each cache code stored in the shared memory with a preset time period as a period;

[0069] If the time interval between the timestamp of any cached code and the current time node is greater than or equal to the preset cleaning duration, the fast user space mutual exclusion mechanism is used to delete the any cached code from the shared memory.

[0070] In an optional embodiment, the device further comprises a closing module, which is used to:

[0071] When an error occurs in any thread in the target working process, adding a write lock to the target working process;

[0072] After the threads other than the any one thread in the target working process release the lock added to the shared memory, the target working process is closed.

[0073] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0074] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0075] The request processing method and device provided by the embodiment of the present disclosure first respond to the service call request sent by the client, allocate a target work process to the service call request from multiple work processes, and write the service call request to the submission queue corresponding to the target work process; then, use the target work process to read the service call request from the submission queue, and find the memory address information of the cache code matching the service call request from the shared memory; then, use the target work process to process the service call request based on the memory address information of the cache code found from the shared memory, and write the response data obtained by processing the service call request to the completion queue corresponding to the target work process; finally, read the response data in the completion queue, and send the response data to the client; wherein the submission queue and the completion queue are stored in the shared memory. The present disclosure processes the service call request through multi-threading, thereby using different threads to run the cache code, preventing a single process error from causing the entire service to be paralyzed. In addition, through shared memory and queues, the communication efficiency between multiple processes is improved, thereby ensuring the startup efficiency of the serverless computing platform.

[0076] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0078] Figure 1 A schematic diagram of a request processing system provided by an embodiment of the present disclosure is shown;

[0079] Figure 2 A flowchart of a request processing method provided by an embodiment of the present disclosure is shown;

[0080] Figure 3 A schematic diagram showing a shared memory and a private memory provided by an embodiment of the present disclosure is shown;

[0081] Figure 4 A schematic diagram showing the relationship between the monitoring process and the working process provided by an embodiment of the present disclosure;

[0082] Figure 5 A flowchart showing the steps of searching for a memory address provided by an embodiment of the present disclosure is shown;

[0083] Figure 6 A schematic diagram showing the communication between the gateway process and the working process provided by the embodiment of the present disclosure;

[0084] Figure 7 A flowchart of a work process processing a service call request provided by an embodiment of the present disclosure is shown;

[0085] Figure 8 A flowchart of the target work process exit provided by the embodiment of the present disclosure is shown;

[0086] Fig. 9 A schematic diagram of a request processing device provided by an embodiment of the present disclosure is shown;

[0087] Fig.10 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0089] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0090] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0091] Research has found that serverless computing platforms are implemented based on managed language virtual machines. Managed languages ​​are programming languages ​​that are managed and controlled by the runtime environment. Managed language virtual machines run user code through a single-process multi-instance method, with multiple instances running in a single process, thereby reducing instance performance overhead and startup delays. However, in this way, errors in different instances within a process are difficult to isolate. In the same process, if an instance generates an unrecoverable error, all other instances will be forced to interrupt.

[0092] Based on the above research, the present disclosure provides a request processing method, which processes service call requests through multi-threading, thereby utilizing different threads to run cached code to prevent a single process error from causing the entire service to be paralyzed. In addition, through shared memory and queues, the communication efficiency between multiple processes is improved, thereby ensuring the startup efficiency of the serverless computing platform.

[0093] To facilitate understanding of this embodiment, a request processing method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the request processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example, a terminal device or a server or other processing device. In some possible implementations, the request processing method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0094] See also Figure 1 As shown, it is a schematic diagram of a request processing system provided by an embodiment of the present disclosure. A monitoring process, a gateway process and a working process can be run on the request processing system. At least one thread can be run under the working process. An instance can be run in the thread. After the request processing system detects a service call request, the request processing method provided by the embodiment of the present disclosure can be executed. The monitoring process, the gateway process and the working process can read and write shared memory and private memory deployed on the request processing system.

[0095] See also Figure 2 FIG. 2 is a flowchart of a request processing method provided by an embodiment of the present disclosure, wherein the method includes steps S201 to S204, wherein:

[0096] S201. In response to a service call request sent by a client, a target work process is allocated to the service call request from multiple work processes, and the service call request is written into a submission queue corresponding to the target work process.

[0097] In this step, the request processing system can receive the service call request sent by the client, and select a suitable target work process from multiple work processes to process the service call request. Among them, at least one thread can be run in the work process, and an instance can be run in the thread, and the instance is used to process the service call request. The work process can read the shared memory and its corresponding private memory. The submission queue of each work process can be deployed in the shared memory. The gateway process can write the service call request to the submission queue in the shared memory. The work process can read the data in the corresponding submission queue to obtain the written service call request.

[0098] Specifically, step S201 may include:

[0099] Based on the load status of the multiple work processes, a target work process assigned to the service call request is screened out from the multiple work processes; and the service call request is written into a submission queue corresponding to the target work process by using a fast user space mutual exclusion mechanism.

[0100] In this step, the load status of the work process can be obtained, which may include the number of currently available threads of the work process, the number of currently running instances, the total number of threads, etc. The process with the largest number of available threads can be preferentially selected as the target work process to balance the load of each process and prevent a thread from being overloaded.

[0101] Among them, the fast user space mutual exclusion mechanism is also called Futex, which is a mixed synchronization mechanism of user state and kernel state. The synchronized processes can share a section of memory. The Futex variable is located in this shared memory. When the process tries to enter or exit the mutual exclusion area, it can first check the Futex variable in the shared memory. If there is no other thread competing for the memory segment, only the Futex variable is modified to directly access the shared memory without executing a system call. If the Futex variable indicates that there are other threads competing for the memory segment, the system call is executed to complete the processing required for shared memory access, such as waiting, waking up, etc. When there are no other competing threads, there is no need to execute a system call, which is more efficient. The Futex variable here is equivalent to applying a mutual exclusion lock to the shared memory. Before writing the service call request, first determine whether the submission queue corresponding to the target working process is occupied by other threads. If it is occupied, the system call is executed and waits for the release of the shared memory normally. If it is not occupied, the service call request is directly written to the submission queue.

[0102] Here, the above working process can access the shared memory and its own corresponding private memory at the same time, see Figure 3, which is a schematic diagram of the shared memory and private memory provided by the embodiment of the present disclosure. The shared memory includes a shared buffer, and the working process can access the shared buffer and its corresponding private memory. At the same time, the monitoring process can also access the shared buffer and its corresponding private memory.

[0103] See also Figure 4 As shown, it is a schematic diagram of the relationship between the monitoring process and the working process provided in the embodiment of the present disclosure. The above-mentioned monitoring process can be used to allocate shared memory through memory mapping mmap, initialize shared data structures (such as cache code, submission queue, completion queue), start the working process, execute cache expiration policy, etc.

[0104] S202: Using the target working process to read the service call request from the submission queue, and searching the shared memory for memory address information of the cache code matching the service call request.

[0105] When there is a service call request in the submission queue, the target working process can be controlled to read the written service call request from the submission queue. The service code corresponding to the service call request is required to respond to the service call request. When there is no thread holding the service code under the target working process, it is necessary to obtain the service code corresponding to the service call request. The service code can be pre-cached in the shared memory and stored in the form of cached code. The target working thread can search for the memory address information of the cached code from the shared memory, and then read the cached code from the found memory address information. Here, since the cached code is stored in the shared memory, the above memory address information is also within the address range of the shared memory.

[0106] The address information of the cache code may be stored in the shared memory in the form of a key-value pair, the key of the key-value pair may be the identification information of the service call request, and the value of the key-value pair may be the memory address information of the cache code corresponding to the service call request.

[0107] Specifically, through the fast user space mutual exclusion mechanism, the request-cache mapping relationship stored in the shared memory can be quickly accessed. The mapping relationship can be the key-value pair mentioned above. By traversing the keys in the request-cache mapping relationship, the mapping relationship that matches the identification information of the service call request is found, and then the value in the found key-value pair is used as the memory address information of the cache code that matches the service call request.

[0108] Exemplarily, the target working thread can add a mutex lock to the shared memory, which is implemented based on a fast user space mutex mechanism, and then search for the request-cache mapping relationship from the shared memory after the mutex lock is added, and then search for the memory address information of the cache code that matches the service call request from the request-cache mapping relationship, and update the reference count of the memory address information of the cache code found, so that the cache code is in a referenced state to avoid cache cleanup, and finally release the mutex lock of the shared memory after completion, so that other working threads can access the shared memory.

[0109] See also Figure 5 As shown, it is a flowchart of the steps of searching for memory addresses provided in an embodiment of the present disclosure. This step first locks the cache in the shared memory, and then searches for a key matching the service call request in the shared memory. If the key is found, the reference count of the value corresponding to the key is increased by 1, and then the cache in the shared memory is unlocked, and the found value, that is, the memory address information of the cache code, is returned; if the key does not exist, the cache in the shared memory is unlocked, and a null value is returned.

[0110] S203: Using the target working process, processing the service call request based on the memory address information of the cache code found in the shared memory, and writing the response data obtained by processing the service call request into the completion queue corresponding to the target working process.

[0111] In this step, the above-mentioned memory address information is the memory address where the cache code is stored in the shared memory. After finding the memory address information of the cache code, the target working process can be controlled to access the address information of the cache code, read the cache code, and use the cache code to process the service call request. After using the cache code to process the service call request, the obtained response data can be written to the completion queue corresponding to the target working process.

[0112] Similar to the submission queue, the completion queue is also stored in shared memory and can be read and written by the gateway process and the worker process. The worker process writes the response data to the submission queue, and the gateway process reads the response data from the submission queue. Reading and writing data can also utilize the fast user space mutual exclusion mechanism.

[0113] Specifically, the thread under the target working process can be controlled to establish an instance of the cache code, and then the cache code is read according to the memory address information, and the parameters indicated by the service call request are used to run the cache code in the instance to process the service call request and obtain the response data of the service call request. Finally, the response data is written into the completion queue of the target working process through the fast user space mutual exclusion mechanism.

[0114] In this way, by caching the code in shared memory, multiple caching of the cache code by different threads can be avoided, reducing memory overhead and thus improving operating performance.

[0115] S204: Read the response data in the completion queue, and send the response data to the client.

[0116] In this step, the gateway thread can use the fast user space mutual exclusion mechanism to read the response data in the completion queue, and then send the response data to the client corresponding to the service call request, thereby realizing a response to the service call request.

[0117] Furthermore, if the address information of the cache code matching the service call request cannot be found, it means that the cache code is not stored in the shared memory. The cache code matching the service call can be requested from the code database, and the cache code matching the service call request can be written into the shared memory by utilizing the fast user space mutual exclusion mechanism. At the same time, the request-cache mapping relationship can be updated. After that, the instruction to search the shared memory for the memory address information of the cache code matching the service call request can be executed again, and the service request instruction can be responded to according to the found memory address information, or the service request instruction can be directly responded to using the cache code requested from the code database.

[0118] In order to alleviate the pressure on shared memory and improve the system's operating performance, cached codes can be cleaned up regularly. For example, the Least Recently Used (LRU) strategy can be adopted. For example, the timestamp of the last use of each cached code stored in the shared memory can be determined based on a preset time period, that is, the timestamp of the last reference. If the time interval between the above timestamp of a cached code and the current time node is greater than or equal to the preset cleaning time, the user space mutual exclusion mechanism can be used to delete the above cached code from the shared memory.

[0119] Here, before deleting the cache code, the reference count of the pointer of the cache code may be determined first. If the reference count is not 0, the cache code may not be deleted to prevent errors in the thread that is referencing the cache code.

[0120] See also Figure 6 As shown, it is a schematic diagram of the communication between the gateway process and the working process provided by the embodiment of the present disclosure. The above-mentioned gateway process can write the information sent by the client to the submission queue of the working process in the shared memory, and then the working process reads the data in the submission queue; the gateway process can also feed back the information written by the working process to the completion queue to the client, realizing the bridge of communication between the client and the working process.

[0121] For further information, see Figure 8As shown, it is a flowchart of exiting the target work process provided by the embodiment of the present disclosure. During the operation of the system, there may be a situation where a thread in the target work process has an error, and the target work thread needs to be restarted. If an error occurs in a thread, a write lock can be added to the private read-write lock RwLock of the target work process to prevent the target work process from being written to data. After adding the write lock, you can wait for other threads in the target work process to complete the access to the shared memory, and finally release the read-write lock RwLock added to the shared memory. After that, run the exit-related logic, close the target work process, and restart to eliminate the error that occurred.

[0122] In this way, by adding a write lock to the target worker process first, data can be prevented from being written to the target worker process. In this way, no new tasks will be generated. After other threads release the shared memory, the target worker process can be restarted to avoid the problem that other worker processes cannot access the shared memory due to the shared memory not being released during the restart.

[0123] In one possible implementation, see Figure 7 As shown, it is a flowchart of a work process processing a service call request provided by an embodiment of the present disclosure. In this step, the work process can read the task corresponding to the service call request from the submission queue, and then determine whether there is an idle instance matching the service call request in its own thread. If not, the cached code package is obtained from the shared memory, and the instance matching the service call request is started by using the code package, and then the execution environment is initialized. If it exists, the task is directly distributed to the thread corresponding to the instance, and the execution environment is initialized. After that, the application code of the code package can be executed, and then the obtained response data is sent to the completion queue for the gateway thread to read and feed back to the client; when a thread error is found, it can be determined whether the error can be recovered. If it can be recovered, the instance of the thread where the error occurred is recycled to the instance pool for thread recovery. If it cannot be recovered, a write lock is added to the work process to realize the offline of the work process, and the monitoring process is notified to prevent the work process from being assigned new tasks, and then wait for the requests being processed by other threads of the work process to end, release the shared memory, and exit the current work process where the error occurs.

[0124] In this way, the present invention realizes serverless service through multiple processes and multiple threads and shared memory under each process, thereby improving the efficiency of cross-process scheduling of managed language programs. Multiple instances of managed language programs can run in a thread, and the instances are isolated by logical methods. Through the process-thread hybrid design, errors are dispersed to different processes, thereby improving the reliability of the request processing system and preventing other users' processes from being interrupted due to errors in a certain thread.

[0125] The request processing method provided by the embodiment of the present disclosure first responds to the service call request sent by the client, allocates a target work process to the service call request from multiple work processes, and writes the service call request to the submission queue corresponding to the target work process; then, the target work process is used to read the service call request from the submission queue, and the memory address information of the cache code matching the service call request is found from the shared memory; thereafter, the target work process is used to process the service call request based on the memory address information of the cache code found from the shared memory, and the response data obtained by processing the service call request is written to the completion queue corresponding to the target work process; finally, the response data in the completion queue is read, and the response data is sent to the client; wherein the submission queue and the completion queue are stored in the shared memory. The present disclosure processes the service call request through multi-threading, thereby using different threads to run the cache code, preventing a single process error from causing the entire service to be paralyzed. In addition, through shared memory and queues, the communication efficiency between multiple processes is improved, thereby ensuring the startup efficiency of the serverless computing platform.

[0126] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0127] Based on the same inventive concept, a request processing device corresponding to the request processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned request processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0128] Reference Fig. 9 FIG. 1 is a schematic diagram of a request processing device provided by an embodiment of the present disclosure, wherein the device includes:

[0129] The allocation module 910 is used to respond to the service call request sent by the client, allocate a target work process to the service call request from multiple work processes, and write the service call request to the submission queue corresponding to the target work process;

[0130] A search module 920 is used to read the service call request from the submission queue using the target working process, and search for memory address information of a cache code matching the service call request from a shared memory;

[0131] The processing module 930 is used to use the target working process to process the service call request based on the memory address information of the cache code found in the shared memory, and write the response data obtained by processing the service call request into the completion queue corresponding to the target working process;

[0132] A sending module 940 is used to read the response data in the completion queue and send the response data to the client;

[0133] The submission queue and the completion queue are stored in the shared memory.

[0134] In an optional implementation, when the allocation module 910 allocates a target work process to the service call request from multiple work processes and writes the service call request to a submission queue corresponding to the target work process, it is used to:

[0135] Based on the load status of the multiple work processes, select a target work process assigned to the service call request from the multiple work processes;

[0136] The service call request is written into the submission queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0137] In an optional implementation manner, the search module 920 is specifically used to:

[0138] Using a fast user space mutual exclusion mechanism, reading the service call request from the submission queue;

[0139] Utilizing a fast user space mutual exclusion mechanism, based on the request-cache mapping relationship stored in the shared memory, the memory address information of the cache code that matches the service call request is searched from the shared memory; wherein the request-cache mapping relationship includes a key-value pair with the identification information of the service call request as the key and the memory address information of the cache code that matches the service call request as the value.

[0140] In an optional implementation manner, the processing module 930 is specifically configured to:

[0141] Using the thread under the target working process to establish an instance of the cache code;

[0142] Using the memory address information of the cache code and the service call request, the cache code is run in the instance of the cache code to obtain the response data;

[0143] The response data is written into the completion queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0144] In an optional implementation manner, the sending module 940 is specifically configured to:

[0145] Using a fast user space mutual exclusion mechanism, reading the response data in the completion queue;

[0146] The response data is sent to the client.

[0147] In an optional implementation, the search module 920 uses a fast user space mutual exclusion mechanism to search the shared memory for memory address information of the cache code matching the service call request based on the request-cache mapping relationship stored in the shared memory, and is used to:

[0148] Adding a mutual exclusion lock to the shared memory; the mutual exclusion lock is implemented based on the fast user space mutual exclusion mechanism;

[0149] Searching the request-cache mapping relationship from the shared memory after adding the mutex lock;

[0150] Finding the memory address information of the cache code matching the service call request from the request-cache mapping relationship, and updating the reference count of the found memory address information of the cache code;

[0151] Release the mutex lock of the shared memory.

[0152] In an optional implementation manner, the device further includes a request module, configured to:

[0153] If the memory address information of the cache code matching the service call request is not found, requesting the cache code matching the service call request from the code database;

[0154] Using a fast user space mutual exclusion mechanism, writing the cache code matching the service call request into the shared memory, and updating the request-cache mapping relationship;

[0155] The instruction of searching the shared memory for the memory address information of the cache code matching the service call request is executed again.

[0156] In an optional implementation manner, the device further includes a deletion module, configured to:

[0157] Determine the timestamp of the last use of each cache code stored in the shared memory with a preset time period as a period;

[0158] If the time interval between the timestamp of any cached code and the current time node is greater than or equal to the preset cleaning time, the fast user space mutual exclusion mechanism is used to delete the any cached code from the shared memory.

[0159] In an optional embodiment, the device further comprises a closing module, which is used to:

[0160] When an error occurs in any thread in the target working process, adding a write lock to the target working process;

[0161] After the threads other than the any one thread in the target working process release the lock added to the shared memory, the target working process is closed.

[0162] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0163] Corresponds to Figure 1 The request processing method in the present disclosure also provides an electronic device 1000, such as Fig.10 FIG. 1 is a schematic diagram of the structure of an electronic device 1000 provided in an embodiment of the present disclosure, including:

[0164] Processor 101, memory 102, and bus 103; memory 102 is used to store execution instructions, including internal memory 1021 and external memory 1022; the internal memory 1021 is also called internal memory, which is used to temporarily store the operation data in the processor 101 and the data exchanged with the external memory 1022 such as a hard disk. The processor 101 exchanges data with the external memory 1022 through the internal memory 1021. When the electronic device 1000 is running, the processor 101 communicates with the memory 102 through the bus 103, so that the processor 101 executes the following instructions:

[0165] In response to a service call request sent by a client, a target work process is allocated for the service call request from multiple work processes, and the service call request is written into a submission queue corresponding to the target work process;

[0166] Using the target working process to read the service call request from the submission queue, and searching for memory address information of the cache code matching the service call request from the shared memory;

[0167] Using the target working process, processing the service call request based on the memory address information of the cache code found from the shared memory, and writing the response data obtained by processing the service call request into the completion queue corresponding to the target working process;

[0168] Reading the response data in the completion queue and sending the response data to the client;

[0169] The submission queue and the completion queue are stored in the shared memory.

[0170] In an optional implementation manner, in the instructions executed by the processor 101, allocating a target work process for the service call request from multiple work processes and writing the service call request to a submission queue corresponding to the target work process includes:

[0171] Based on the load status of the multiple work processes, select a target work process assigned to the service call request from the multiple work processes;

[0172] The service call request is written into the submission queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0173] In an optional implementation, in the instructions executed by the processor 101, using the target work process to read the service call request from the submission queue and searching for a cache code matching the service call request from a shared memory includes:

[0174] Using a fast user space mutual exclusion mechanism, reading the service call request from the submission queue;

[0175] Utilizing a fast user space mutual exclusion mechanism, based on the request-cache mapping relationship stored in the shared memory, the memory address information of the cache code that matches the service call request is searched from the shared memory; wherein the request-cache mapping relationship includes a key-value pair with the identification information of the service call request as the key and the memory address information of the cache code that matches the service call request as the value.

[0176] In an optional implementation manner, in the instructions executed by the processor 101, using the target work process to process the service call request based on the memory address information of the cache code found from the shared memory, and writing the response data obtained by processing the service call request to the completion queue corresponding to the target work process, includes:

[0177] Using the thread under the target working process to establish an instance of the cache code;

[0178] Using the memory address information of the cache code and the service call request, the cache code is run in the instance of the cache code to obtain the response data;

[0179] The response data is written into the completion queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

[0180] In an optional implementation manner, in the instructions executed by the processor 101, reading the response data in the completion queue and sending the response data to the client includes:

[0181] Using a fast user space mutual exclusion mechanism, reading the response data in the completion queue;

[0182] The response data is sent to the client.

[0183] In an optional implementation manner, in the instructions executed by the processor 101, using a fast user space mutual exclusion mechanism, based on a request-cache mapping relationship stored in the shared memory, searching from the shared memory for memory address information of a cache code matching the service call request, includes:

[0184] Adding a mutual exclusion lock to the shared memory; the mutual exclusion lock is implemented based on the fast user space mutual exclusion mechanism;

[0185] Searching the request-cache mapping relationship from the shared memory after adding the mutex lock;

[0186] Finding the memory address information of the cache code matching the service call request from the request-cache mapping relationship, and updating the reference count of the found memory address information of the cache code;

[0187] Release the mutex lock of the shared memory.

[0188] In an optional implementation manner, the instructions executed by the processor 101 further include:

[0189] If the memory address information of the cache code matching the service call request is not found, requesting the cache code matching the service call request from the code database;

[0190] Using a fast user space mutual exclusion mechanism, writing the cache code matching the service call request into the shared memory, and updating the request-cache mapping relationship;

[0191] The instruction of searching the shared memory for the memory address information of the cache code matching the service call request is executed again.

[0192] In an optional implementation manner, the instructions executed by the processor 101 further include:

[0193] Determine the timestamp of the last use of each cache code stored in the shared memory with a preset time period as a period;

[0194] If the time interval between the timestamp of any cached code and the current time node is greater than or equal to the preset cleaning time, the fast user space mutual exclusion mechanism is used to delete the any cached code from the shared memory.

[0195] In an optional implementation manner, the instructions executed by the processor 101 further include:

[0196] When an error occurs in any thread in the target working process, adding a write lock to the target working process;

[0197] After the threads other than the any one thread in the target working process release the lock added to the shared memory, the target working process is closed.

[0198] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the request processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0199] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the request processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0200] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0201] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0202] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0203] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0204] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0205] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A request processing method, characterized in that: include: In response to a service call request sent by a client, a target work process is allocated to the service call request from multiple work processes, and the service call request is written into a submission queue corresponding to the target work process; Using the target working process to read the service call request from the submission queue, and searching for memory address information of the cache code matching the service call request from the shared memory; Using the target working process, processing the service call request based on the memory address information of the cache code found from the shared memory, and writing the response data obtained by processing the service call request into the completion queue corresponding to the target working process; Reading the response data in the completion queue and sending the response data to the client; The submission queue and the completion queue are stored in the shared memory.

2. The method according to claim 1, characterized in that The allocating a target work process for the service call request from a plurality of work processes, and writing the service call request into a submission queue corresponding to the target work process, comprises: Based on the load status of the multiple work processes, select a target work process assigned to the service call request from the multiple work processes; The service call request is written into the submission queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

3. The method according to claim 1, characterized in that The using the target working process to read the service call request from the submission queue and searching for a cache code matching the service call request from a shared memory includes: Using a fast user space mutual exclusion mechanism, reading the service call request from the submission queue; Utilizing a fast user space mutual exclusion mechanism, based on the request-cache mapping relationship stored in the shared memory, the memory address information of the cache code that matches the service call request is searched from the shared memory; wherein the request-cache mapping relationship includes a key-value pair with the identification information of the service call request as the key and the memory address information of the cache code that matches the service call request as the value.

4. The method according to claim 1, characterized in that: The process of using the target working process to process the service call request based on the memory address information of the cache code found in the shared memory, and writing the response data obtained by processing the service call request into the completion queue corresponding to the target working process, includes: Using the thread under the target working process to establish an instance of the cache code; Using the memory address information of the cache code and the service call request, the cache code is run in the instance of the cache code to obtain the response data; The response data is written into the completion queue corresponding to the target working process by utilizing a fast user space mutual exclusion mechanism.

5. The method according to claim 1, characterized in that The reading the response data in the completion queue and sending the response data to the client includes: Using a fast user space mutual exclusion mechanism, reading the response data in the completion queue; The response data is sent to the client.

6. The method according to claim 3, characterized in that The method of utilizing a fast user space mutual exclusion mechanism to search the shared memory for memory address information of a cache code matching the service call request based on a request-cache mapping relationship stored in the shared memory includes: Adding a mutual exclusion lock to the shared memory; the mutual exclusion lock is implemented based on the fast user space mutual exclusion mechanism; Searching the request-cache mapping relationship from the shared memory after adding the mutex lock; Finding the memory address information of the cache code matching the service call request from the request-cache mapping relationship, and updating the reference count of the found memory address information of the cache code; Release the mutex lock of the shared memory.

7. The method according to claim 3, characterized in that The method further comprises: If the memory address information of the cache code matching the service call request is not found, requesting the cache code matching the service call request from the code database; Using a fast user space mutual exclusion mechanism, writing the cache code matching the service call request into the shared memory, and updating the request-cache mapping relationship; The instruction of searching the shared memory for the memory address information of the cache code matching the service call request is executed again.

8. The method according to claim 1, characterized in that The method further comprises: Determine the timestamp of the last use of each cache code stored in the shared memory with a preset time period as a period; If the time interval between the timestamp of any cached code and the current time node is greater than or equal to the preset cleaning time, the fast user space mutual exclusion mechanism is used to delete the any cached code from the shared memory.

9. The method according to claim 1, characterized in that: The method further comprises: When an error occurs in any thread in the target working process, adding a write lock to the target working process; After the threads other than the any one thread in the target working process release the lock added to the shared memory, the target working process is closed.

10. A request processing device, characterized in that: include: An allocation module, configured to respond to a service call request sent by a client, allocate a target work process to the service call request from multiple work processes, and write the service call request to a submission queue corresponding to the target work process; A search module, used to read the service call request from the submission queue using the target working process, and search for memory address information of a cache code matching the service call request from a shared memory; A processing module, used to use the target working process to process the service call request based on the memory address information of the cache code found from the shared memory, and write the response data obtained by processing the service call request into the completion queue corresponding to the target working process; A sending module, used for reading the response data in the completion queue and sending the response data to the client; The submission queue and the completion queue are stored in the shared memory.

11. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the request processing method as described in any one of claims 1 to 9 are performed.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the request processing method according to any one of claims 1 to 9 are executed.

Images