Request processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a request processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method is applied to a server, a server side is deployed on the server, transmission control connection is established between the server side and a client side in advance, and the server side comprises a working thread group; the method comprises the following steps: receiving request data sent by the client; under the condition that a target data channel corresponding to the transmission control connection is registered on a target working thread in the working thread group, obtaining a load state index value of the server; under the condition that the load state index value meets a preset request receiving condition, transmitting the request data to the target working thread through the target data channel; and processing the request data through the target working thread. By adopting the method, the request response speed can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a request processing method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of Internet applications and high-concurrency systems, thread pools, as a basic resource management mechanism, are widely used in server-side architectures to effectively control and reuse execution threads to support the system in handling massive concurrent requests.

[0003] In the traditional processing model, whenever a new request arrives at the server, the system allocates a separate thread to execute the entire processing flow of the request until the response is returned.

[0004] However, in high-concurrency or long-connection scenarios, the effective thread resources for processing actual computing tasks are sharply reduced because a large number of processing threads are occupied and blocked in I / O (Input / Output) waiting. This leads to a bottleneck in overall throughput, making it impossible to fully utilize CPU (Central Processing Unit) resources, and consequently increasing request response latency. Summary of the Invention

[0005] Therefore, it is necessary to provide a request processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the request response speed in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a request processing method applied to a server, wherein a server-side component is deployed on the server, and the server-side component and the client have pre-established a transport control connection, and the server-side component includes a group of worker threads; the method includes:

[0007] Receive request data sent by the client;

[0008] If a target data channel corresponding to a transmission control connection is registered on the target worker thread in the worker thread group, obtain the load status index value of the server.

[0009] If the load status indicator value meets the preset request acceptance conditions, the request data will be transmitted to the target worker thread through the target data channel;

[0010] The requested data is processed by the target worker thread.

[0011] In one embodiment, the server includes a central processing unit (CPU) and memory, the CPU being used to provide computing resources for the worker thread group and the memory being used to provide data storage space for the worker thread group; the load status metric values ​​include a first utilization rate of the CPU and a second utilization rate of the memory; the preset request acceptance conditions include the first utilization rate being less than a preset first utilization rate threshold and the second utilization rate being less than a preset second utilization rate threshold.

[0012] After obtaining the server's load status metrics, the method also includes:

[0013] If the first utilization rate is less than a preset first utilization rate threshold and the second utilization rate is less than a preset second utilization rate threshold, the load status index value is determined to meet the preset request acceptance condition.

[0014] If the first utilization rate is not less than a preset first utilization rate threshold, or the second utilization rate is not less than a preset second utilization rate threshold, the load status indicator value is determined to not meet the preset request acceptance condition.

[0015] In one embodiment, the server further includes a management thread group; the worker thread group includes at least one worker thread; after receiving request data sent by the client, the method further includes:

[0016] Obtain the server's load status metric value when no target data channel corresponding to the transmission control connection is registered on any of the worker threads;

[0017] When the load status index value meets the preset request acceptance conditions, the target worker thread is determined from each worker thread through the management thread group, and the target data channel corresponding to the transmission control connection is registered on the target worker thread.

[0018] The request data sent by the client is transmitted to the target worker thread through the target data channel;

[0019] The requested data is processed by the target worker thread.

[0020] In one embodiment, the load status metric includes the average processing time of each worker thread and the total number of current tasks; determining the target worker thread from among the worker threads includes:

[0021] Based on the average processing time of each worker thread and the total number of current tasks, estimate the estimated processing time required for each worker thread to process the requested data.

[0022] The worker thread corresponding to the shortest estimated processing time is determined as the target worker thread.

[0023] In one embodiment, the worker thread corresponding to the shortest estimated processing time is determined as the target worker thread, including:

[0024] If the shortest estimated processing time is less than or equal to the preset timeout threshold, the worker thread corresponding to the shortest estimated processing time will be determined as the target worker thread.

[0025] After estimating the estimated processing time required for each worker thread to process the requested data based on the average processing time of each worker thread and the total number of current tasks, the method further includes:

[0026] If the shortest estimated processing time exceeds the preset timeout threshold, generate response data to reject the request.

[0027] The response data is returned to the client.

[0028] In one embodiment, after obtaining the server's load status metric value, the method further includes:

[0029] If the load status indicator value does not meet the preset conditions for accepting the request, response data is generated based on the load status indicator value to reject the request.

[0030] The response data is returned to the client.

[0031] Secondly, this application also provides a request processing apparatus, applied to a server, wherein the server and client pre-establish a transport control connection, and the server includes a group of worker threads; comprising:

[0032] The receiving module is used to receive request data sent by the client;

[0033] The acquisition module is used to acquire the load status index value of the server when the target data channel corresponding to the transmission control connection is registered on the target worker thread in the worker thread group.

[0034] The transmission module is used to transmit the request data to the target worker thread through the target data channel when the load status index value meets the preset request acceptance conditions;

[0035] The processing module is used to process the requested data through the target worker thread.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0037] Receive request data sent by the client;

[0038] If a target data channel corresponding to a transmission control connection is registered on the target worker thread in the worker thread group, obtain the load status index value of the server.

[0039] If the load status indicator value meets the preset request acceptance conditions, the request data will be transmitted to the target worker thread through the target data channel;

[0040] The requested data is processed by the target worker thread.

[0041] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0042] Receive request data sent by the client;

[0043] If a target data channel corresponding to a transmission control connection is registered on the target worker thread in the worker thread group, obtain the load status index value of the server.

[0044] If the load status indicator value meets the preset request acceptance conditions, the request data will be transmitted to the target worker thread through the target data channel;

[0045] The requested data is processed by the target worker thread.

[0046] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0047] Receive request data sent by the client;

[0048] If a target data channel corresponding to a transmission control connection is registered on the target worker thread in the worker thread group, obtain the load status index value of the server.

[0049] If the load status indicator value meets the preset request acceptance conditions, the request data will be transmitted to the target worker thread through the target data channel;

[0050] The requested data is processed by the target worker thread.

[0051] The aforementioned request processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product achieve immediate reception and response preparation for client requests by receiving request data sent by the client; by obtaining the server's load status index value when the target data channel corresponding to the transmission control connection is registered on the target worker thread in the worker thread group, they achieve real-time monitoring and quantitative evaluation of the system's load status; and by transmitting the request data to the target worker thread through the target data channel when the load status index value meets the preset request acceptance conditions, the target worker thread processes the request data, achieving efficient scheduling and execution of requests within the system's capabilities. In this process, on the one hand, by reusing the established transmission control connection and corresponding target data channel, it is unnecessary to allocate a separate thread for each request, reducing resource consumption caused by frequent thread switching. On the other hand, through the load status judgment mechanism, requests are received and transmitted only when the load status allows, reducing request backlog during overload and lowering the risk of upstream and downstream blockage due to untimely responses. This enhances the overall throughput and stability in high-concurrency scenarios, effectively improves request response speed, and significantly reduces response latency. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a diagram illustrating the application environment of a request processing method in one embodiment of this application.

[0054] Figure 2 This is a flowchart illustrating a request processing method in one embodiment of this application;

[0055] Figure 3 This is a flowchart illustrating a request processing method in another embodiment of this application;

[0056] Figure 4 This is a structural block diagram of a request processing device in one embodiment of this application;

[0057] Figure 5 This is an internal structural diagram of a computer device in one embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] The request processing method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0060] In one exemplary embodiment, the method is applied to Figure 1 Taking server 104 as an example, the server is deployed on server 104, and the client can be deployed on terminal 102. The server can refer to an application that provides network services. The client can refer to an application or device that initiates a request to the server. A transport control connection is pre-established and maintained between the server and the client.

[0061] A transmission control connection refers to a network connection used for transmitting data. Transmission control connections are typically established based on reliable transport layer protocols (such as TCP) to ensure that data can be transmitted in an orderly and reliable manner bidirectionally between the client and the server.

[0062] In some feasible implementations, the client and server can establish a TCP connection during the initial communication and reuse that connection in subsequent requests.

[0063] The server creates and manages a set of worker threads for concurrent processing of business logic. Each worker thread can register one or more data channels, and each data channel corresponds to a client's transport control connection, used to implement non-blocking reception and response of requested data on that transport control connection.

[0064] A server includes a central processing unit (CPU) and memory. The CPU provides computing resources for the worker thread group, and the memory provides data storage space for the worker thread group.

[0065] In some feasible implementations, the number of worker threads on the server is greater than or equal to the number of CPU cores, and the difference between the number of worker threads and the number of CPU cores is less than a preset threshold. This configuration aims to ensure that the number of worker threads is roughly matched to the hardware's parallelism capabilities, avoiding frequent thread suspension and wake-up due to the number of threads far exceeding the number of cores. A larger number of worker threads results in greater context switching overhead, which in turn affects the efficiency of the CPU in executing actual computational tasks.

[0066] In this embodiment, each request is not processed exclusively on a dedicated worker thread. Instead, the request data is asynchronously transmitted to the corresponding worker thread for processing via a data channel registered on the worker thread. Therefore, the same worker thread can process requests from multiple different connections sequentially, avoiding the idle and wasted thread resources caused by thread blocking and waiting for I / O in the traditional "one thread, one request" model. This significantly improves the overall request throughput and resource utilization efficiency of the system under the same hardware resources.

[0067] When a single worker thread processes requests from multiple different connections, it can poll and process each data unit sequentially according to the order in which the requests arrive at their corresponding data channels. If an I / O wait (such as a database query or calling an external interface) occurs during the processing of a request, the worker thread will not remain blocked. Instead, it will immediately suspend the current processing context of that request and retrieve the next pending request from another ready data channel to begin processing. Once the previously suspended I / O operation completes and returns a result, the processing context of that request will be rescheduled, and the remaining logic will continue to be executed in subsequent polls by the worker thread. Through this mechanism, the computing resources of the worker thread are continuously and fully utilized, effectively avoiding resource idleness caused by thread blocking while waiting for I / O, thereby improving the overall concurrency processing capability and resource utilization of the server.

[0068] This embodiment provides a request processing method, such as Figure 2 As shown, the method includes steps 202 to 208. Wherein:

[0069] Step 202: Receive request data sent by the client.

[0070] For example, the server-side network listening component continuously listens for established transport control connections. When a client sends request data through this transport control connection, the server-side network framework triggers a read event, reads the arriving request data packet, and parses it into structured request data.

[0071] Step 204: If the target data channel corresponding to the transmission control connection is registered on the target worker thread in the worker thread group, obtain the load status index value of the server.

[0072] The target worker thread can refer to the worker thread that has registered a target data channel corresponding to the transmission control connection.

[0073] A target data channel can refer to a logical or physical data transmission path identified or established on top of an established transmission control connection for the efficient transmission of specific requested data. Registering a target data channel on a target worker thread means that the target worker thread is configured to listen for and process data incoming through the target data channel.

[0074] Load status indicators can refer to quantitative data that reflects the current workload of a worker thread group. These can include the usage status of each worker thread, the number of core threads currently executing tasks, the average processing time of each worker thread's tasks and the total number of tasks over a period of time, CPU utilization, memory utilization, etc., or a comprehensive value obtained by weighting, comparing, or calculating the above basic indicators using functions.

[0075] For example, if the client is not sending a request to the server for the first time, the server's worker thread group usually already contains the registration information for the target data channel corresponding to the transport control connection. In this case, it is not necessary to re-register the channel; instead, the current load status metric value can be obtained by calling the monitoring module or accessing the server's internal data structure. Alternatively, the raw data used to calculate the load status metric value can be obtained by calling the monitoring module or accessing the server's internal data structure, and then processed according to predefined rules or formulas to obtain the current load status metric value.

[0076] Step 206: If the load status indicator value meets the preset request acceptance conditions, transmit the request data to the target worker thread through the target data channel.

[0077] The preset acceptance conditions can refer to predefined logical judgment rules for determining whether a new request should be accepted and processed. The specific rules can be determined based on actual conditions or test results, and this embodiment does not impose any restrictions on them.

[0078] For example, the obtained load status index value is substituted into the preset request acceptance conditions for logical judgment. If the judgment result is that the preset request acceptance conditions are met, the request data is transmitted to the corresponding target worker thread through the registered target data channel.

[0079] In some feasible embodiments, each worker thread in the worker thread group can be bound to an event handler, which polls the read and write events of multiple data channels registered to it.

[0080] Step 208: Process the requested data through the target worker thread.

[0081] For example, after receiving the request data, the target worker thread executes the corresponding business logic processing flow according to the request type and content. After processing, it generates the corresponding response data and returns the response result to the client through the established transport control connection.

[0082] In the above request processing method, by receiving request data sent by the client, real-time reception and response preparation for client requests are achieved. By obtaining the server's load status indicator value when the target data channel corresponding to the transmission control connection is registered on the target worker thread in the worker thread group, real-time monitoring and quantitative evaluation of the system's load status are achieved. When the load status indicator value meets the preset request acceptance conditions, the request data is transmitted to the target worker thread through the target data channel, and the target worker thread processes the request data, achieving efficient scheduling and execution of requests within the system's capabilities. In this process, on the one hand, by reusing the established transmission control connection and the corresponding target data channel, it is not necessary to allocate a separate thread for each request, reducing resource consumption caused by frequent thread switching. On the other hand, through the load status judgment mechanism, requests are received and transmitted only when the load status allows, reducing request backlog during overload and lowering the risk of upstream and downstream blocking due to untimely responses. This enhances the overall throughput and stability in high-concurrency scenarios, effectively improves request response speed, and significantly reduces response latency.

[0083] In an exemplary embodiment, the server includes a central processing unit (CPU) and memory. The CPU provides computing resources to a group of worker threads, and the memory provides data storage space for the worker thread group. Load status metrics include a first utilization rate of the CPU and a second utilization rate of the memory. Preset request acceptance conditions include a first utilization rate less than a preset first utilization rate threshold and a second utilization rate less than a preset second utilization rate threshold. After obtaining the server's load status metrics, the method further includes:

[0084] If the first utilization rate is less than a preset first utilization rate threshold and the second utilization rate is less than a preset second utilization rate threshold, the load status indicator value is determined to meet the preset request acceptance condition; if the first utilization rate is not less than the preset first utilization rate threshold or the second utilization rate is not less than the preset second utilization rate threshold, the load status indicator value is determined to not meet the preset request acceptance condition.

[0085] It's important to note that while channel switching significantly reduces time and resource consumption compared to thread switching, there's a clear upper limit to the computing and storage resources a server can provide to a group of worker threads. When system resources (such as CPU or memory) are already under high load, indiscriminately accepting and processing new requests can easily lead to resource exhaustion. Sustained CPU saturation will cause numerous threads to frequently switch due to competition for computing resources, resulting in a decrease in overall system throughput; excessive memory usage may directly trigger memory overflow errors, causing service processes to crash. If this state is not controlled, it can further cause a sharp increase in response time and even trigger cascading failures, leading to a service avalanche.

[0086] The first utilization rate can refer to the percentage of CPU resources utilized at a specific point in time or within a specific time period.

[0087] The second utilization rate can refer to the percentage of memory resources currently in use relative to the total available resources.

[0088] The preset first utilization threshold can refer to a pre-set upper limit percentage reference value used to determine whether there is still spare CPU resources available for new data channels.

[0089] The preset second utilization threshold can refer to a pre-set reference value that determines whether there is still a surplus of memory resources available for new data channels to allocate the required stack space and runtime data.

[0090] The specific values ​​of the preset first usage rate threshold and the preset second usage rate threshold can be determined according to the actual situation and test results, etc., and this embodiment does not impose any restrictions.

[0091] For example, the performance monitoring interface provided by the server can be called periodically or in an event-driven manner to obtain the current first CPU utilization and the current second memory utilization. This monitoring process can continue, providing the latest underlying resource status data for subsequent decisions. After obtaining the latest first and second utilization, the first utilization is compared with a preset first utilization threshold, and the second utilization is compared with a second utilization threshold.

[0092] If the first utilization rate is less than the preset first utilization rate threshold and the second utilization rate is less than the preset second utilization rate threshold, it indicates that the server's current CPU and memory resources have sufficient spare capacity. Therefore, the load status index value can be output as a result that the preset request acceptance condition is met.

[0093] If the first utilization rate is not less than the preset first utilization rate threshold, or the second utilization rate is not less than the preset second utilization rate threshold, it indicates that at least one type of critical hardware resource of the server has been used to the point of approaching or reaching the preset safety limit, and the resource reserve is insufficient. Therefore, the load status index value can be output as a judgment result that does not meet the preset request acceptance conditions.

[0094] In this embodiment, when the utilization rate of either CPU or memory exceeds the corresponding safety threshold, it is immediately determined that the conditions for accepting a request are not met, thereby preventing new requests from entering the processing flow at the architectural level. This effectively alleviates the continuous pressure on the system when resources are nearly exhausted, reduces the sharp drop in thread scheduling efficiency caused by CPU overload, and reduces the risk of crashes that may be caused by memory overload. Therefore, it can maintain the stability and availability of core services in high-concurrency scenarios, achieve self-protection of its own resources, and improve overall resilience and controllability.

[0095] In one exemplary embodiment, such as Figure 3 As shown, the server also includes a management thread group; the worker thread group includes at least one worker thread; after receiving request data sent by the client, the method further includes steps 302 to 308. Wherein:

[0096] Step 302: If no target data channel corresponding to the transmission control connection is registered on any of the worker threads, obtain the load status index value of the server.

[0097] In this context, the management thread group can refer to a group of threads on the server that are specifically responsible for coordinating and managing worker threads and connection resources.

[0098] For example, when the client sends a request to the server for the first time, the target data channel registration information corresponding to the transmission control connection does not exist in the server's worker thread group. Therefore, it is necessary to register the target data channel corresponding to the transmission control connection on one of the worker threads in the worker thread group first.

[0099] In this scenario, before registration, the current load status metric value can be obtained by calling the monitoring module or accessing the server's internal data structure. Alternatively, the raw data used to calculate the load status metric value can be obtained by calling the monitoring module or accessing the server's internal data structure, and then processed according to predefined rules or formulas to obtain the current load status metric value.

[0100] Step 304: If the load status index value meets the preset request acceptance conditions, the target worker thread is determined from each worker thread through the management thread group, and the target data channel corresponding to the transmission control connection is registered on the target worker thread.

[0101] For example, the obtained load status metric value is substituted into the preset request acceptance conditions for logical judgment. If the judgment result is that the preset request acceptance conditions are met, the management thread group selects one of the worker threads as the target worker thread according to the preset load balancing strategy. After selection, the management thread group performs the channel registration operation.

[0102] In some feasible embodiments, the preset load balancing strategy may include at least one of the following: a round-robin strategy based on the current number of connections of each worker thread or a least-connections strategy.

[0103] In some feasible embodiments, the channel registration operation may include adding the socket descriptor of the transmission control connection or its encapsulated object to the event listener loop of the target worker thread to complete the registration of the target data channel on the target worker thread.

[0104] Step 306: Transmit the request data sent by the client to the target worker thread through the target data channel.

[0105] For example, after registration is completed, since the load status has been determined to meet the conditions for accepting requests, the request data can be directly transmitted to the corresponding target worker thread through the registered target data channel.

[0106] Step 308: Process the requested data through the target worker thread.

[0107] For example, after receiving the request data, the target worker thread executes the corresponding business logic processing flow according to the request type and content. After processing, it generates the corresponding response data and returns the response result to the client through the established transport control connection.

[0108] In this embodiment, by only allowing the allocation of worker threads and registration of data channels for new connections when the load status index value meets the preset request acceptance conditions, the risk of resource overload that may result from blindly accepting new connections under high load is effectively reduced. Simultaneously, by managing the thread group to dynamically select target worker threads based on a load balancing strategy, a relatively balanced task allocation among worker threads is ensured, optimizing overall resource utilization. This allows the server to more smoothly respond to dynamic fluctuations in the number of connections, enhancing the system's self-protection capabilities and overall service stability while maintaining high connection processing capacity.

[0109] In an exemplary embodiment, the load status metric values ​​include the average processing time of each task corresponding to each worker thread and the total number of current tasks; determining the target worker thread from among the worker threads includes:

[0110] Based on the average processing time of each worker thread and the total number of current tasks, estimate the estimated processing time required for each worker thread to process the requested data; determine the worker thread with the shortest estimated processing time as the target worker thread.

[0111] It's important to note that randomly selecting target worker threads, or selecting them solely based on the number of registered data channels, can easily lead to uneven load distribution among these threads. Some worker threads may remain under heavy load for extended periods due to continuously receiving long-running requests, causing their task queues to accumulate; while other worker threads may be relatively idle due to receiving shorter-running requests. This uneven load distribution can result in underutilization of system resources, with high-load threads potentially becoming performance bottlenecks, leading to a significant increase in the response latency of the requests they process.

[0112] The average task processing time can be used to calculate the average processing time of multiple tasks (i.e., requests) recently completed by a specific worker thread. This time reflects the typical speed at which the thread processes tasks or its processing efficiency under the current load type. For example, it can be calculated by maintaining a sliding time window and dividing the total time of all tasks completed by the worker thread by the number of tasks.

[0113] The total number of current tasks refers to the total number of tasks that a particular worker thread has been assigned to but not yet completed. This includes tasks that are currently being executed and tasks waiting in their queue. This number directly reflects the current backlog of tasks for that thread.

[0114] Estimated processing time can refer to the approximate time required for a worker thread to complete a new task if it is assigned a new task, based on the average processing time of a worker thread's tasks and the total number of current tasks.

[0115] In some feasible embodiments, the estimated processing time can be equal to the product of the average processing time of the task and the total number of current tasks, plus the average processing time of the task.

[0116] In some feasible embodiments, tasks can also be classified according to their computation time. The average processing time of a task can include the average processing time of each type of task, and the total number of current tasks can also include the total number of current tasks of each type. In this way, the estimated processing time for each type of task can be calculated first, and then the estimated processing time for each worker thread can be determined by summing these values. For example, assuming tasks are divided into types A1 and A2, the average processing time for thread L to process type A1 tasks is calculated to be T1, and the average processing time for processing type B1 tasks is calculated to be T2. Furthermore, the total number of current tasks of type A1 currently being executed by thread L and the total number of current tasks of type A2 waiting in its queue can be counted as N1, and the total number of current tasks of type A2 as N2. In this case, if a new task belongs to type A1, the estimated processing time required for thread L to process the requested data can be estimated as T1 × N1 + T2 × N2 + T1.

[0117] For example, for each worker thread in the worker thread group, its average task processing time and the current total number of tasks can be obtained first. Then, the estimated processing time required for each worker thread to process the requested data can be calculated using a preset estimation model (e.g., multiplying the average task processing time by the current total number of tasks, or using a more complex weighted calculation). After determining the estimated processing time required for each worker thread to process the requested data, the estimated processing times can be compared to determine the smallest estimated processing time. The worker thread corresponding to the smallest estimated processing time is then selected as the target worker thread for this data request.

[0118] In this embodiment, not only is the current backlog of tasks for each thread considered, but also its historical processing efficiency is taken into account to comprehensively estimate the completion time of each worker thread in processing new requests, and the worker thread expected to complete the task the fastest is always selected. In this way, by assigning new requests to threads with high processing efficiency or light current load, the load is proactively directed to resource units with stronger processing capabilities. This helps reduce the risk of a single thread becoming a bottleneck due to overload, promotes balanced utilization of the processing capacity of each worker thread, reduces the overall average response latency of client requests, improves the system's processing efficiency and throughput under high concurrency, and achieves better service quality and resource utilization.

[0119] In an exemplary embodiment, the worker thread corresponding to the shortest estimated processing time is determined as the target worker thread, including:

[0120] If the shortest estimated processing time is less than or equal to the preset timeout threshold, the worker thread corresponding to the shortest estimated processing time will be selected as the target worker thread.

[0121] It's important to note that under high load, even if the fastest-equipped worker thread is selected, if its estimated completion time is still too long, continuing to accept and process the request will cause the actual response time to far exceed acceptable limits, harming the user experience. More seriously, under sustained high pressure, the prolonged persistence of a large number of requests will exacerbate system resource consumption and queue accumulation, creating a vicious cycle that may ultimately lead to a comprehensive deterioration of system responsiveness.

[0122] The preset timeout threshold can refer to a pre-set maximum time limit used to determine whether the estimated processing time of a request is within the acceptable service time range. The preset timeout threshold can be set based on service level agreements, user experience requirements, or system performance baselines, etc., and this embodiment does not impose any restrictions on it.

[0123] For example, after calculating the estimated processing time for all worker threads and determining the shortest estimated processing time, the shortest estimated processing time is compared with a preset timeout threshold. If the shortest estimated processing time is less than or equal to the preset timeout threshold, it indicates that the request can be processed within an acceptable service commitment range. Therefore, the request is deemed acceptable, and the worker thread corresponding to the shortest estimated processing time is identified as the target worker thread. Subsequently, the request data is assigned to the target worker thread for processing.

[0124] After estimating the estimated processing time required for each worker thread to process the requested data based on the average processing time of the task corresponding to each worker thread and the total number of tasks, the method further includes: generating response data to reject the requested data if the shortest estimated processing time is greater than a preset timeout threshold; and returning the response data to the client.

[0125] For example, if the minimum estimated processing time exceeds a preset timeout threshold, it indicates that under the current system load, no worker thread can complete the processing of the request within an acceptable time, and therefore the request should be rejected. Subsequently, the server-side response generation module can construct a response data packet containing a rejection status and optional explanatory information according to a preset rejection protocol format, and send the response data packet back to the requesting client through the transmission control connection corresponding to the request. Upon receiving this response, the client will be aware that the request has not been processed and can take subsequent actions such as retrying, downgrading, or prompting the user.

[0126] In this embodiment, by introducing a comparison between the estimated processing time and a timeout threshold, along with a corresponding rejection mechanism, service quality assurance can be added on top of load balancing. When it is predicted that even optimal allocation cannot complete a request within the promised time, the request is proactively rejected to prevent it from consuming system resources and experiencing prolonged waiting. This protects the user experience by preventing users from experiencing failures or timeouts after long waits, allowing clients to take alternative solutions promptly through a quick and clear rejection response. Furthermore, it protects the system itself by preventing performance collapse due to continuously accepting requests exceeding processing capacity at overload thresholds, thus maintaining the stability and overall availability of core services.

[0127] In one exemplary embodiment, after obtaining the server's load status metric value, the method further includes:

[0128] If the load status metric value does not meet the preset conditions for accepting requests, generate response data to reject the request based on the load status metric value, and return the response data to the client.

[0129] For example, after substituting the obtained load status metric value into the preset acceptance request conditions for logical judgment, if the judgment result indicates that the preset acceptance request conditions are not met, the response generation module is invoked. This module can combine the load status metric value, the preset acceptance request conditions, and a predefined response template to generate response data containing an appropriate status code and explanatory information, informing the client that the request data has been rejected and the reason for rejection. For instance, an HTTP response with a 503 status code can be generated, and the response body can include the message "System busy, please try again later." Subsequently, the response data packet is sent back to the requesting client through the transmission control connection corresponding to the request. Upon receiving this response, the client can promptly know that the request has not been processed and can take subsequent actions such as retrying, downgrading, or prompting the user.

[0130] In this embodiment, when the system detects that its load has reached a dangerous level, it no longer allows requests to enter the processing queue, but immediately generates and returns a rejection response. On the one hand, for the client, it can quickly obtain clear feedback, avoid long and meaningless waiting times, and improve the predictability of the interaction. On the other hand, for the server itself, by actively discarding requests that exceed its processing capacity, it reduces the immediate pressure on the system, preserves critical resources to process requests already in the queue, effectively prevents the overall service from crashing due to resource exhaustion, and significantly enhances the system's resilience, availability, and service quality reliability under high load scenarios.

[0131] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0132] Based on the same inventive concept, this application also provides a request processing apparatus for implementing the request processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more request processing apparatus embodiments provided below can be found in the limitations of the request processing method described above, and will not be repeated here.

[0133] In one exemplary embodiment, such as Figure 4 As shown, a request processing apparatus is provided, applied to a server. The server and client pre-establish a transmission control connection, and the server includes a group of worker threads. The apparatus includes: a receiving module 402, an acquiring module 404, a transmission module 406, and a processing module 408, wherein:

[0134] The receiving module 402 is used to receive request data sent by the client;

[0135] The acquisition module 404 is used to acquire the load status index value of the server when the target data channel corresponding to the transmission control connection is registered on the target worker thread in the worker thread group.

[0136] Transmission module 406 is used to transmit request data to target worker thread through target data channel when the load status index value meets the preset request acceptance conditions;

[0137] Processing module 408 is used to process the requested data through the target worker thread.

[0138] In an exemplary embodiment, the server includes a central processing unit (CPU) and memory. The CPU provides computing resources to the worker thread group, and the memory provides data storage space for the worker thread group. Load status indicators include a first utilization rate of the CPU and a second utilization rate of the memory. Preset request acceptance conditions include a first utilization rate less than a preset first utilization rate threshold and a second utilization rate less than a preset second utilization rate threshold. The request processing device further includes a judgment module, which, after obtaining the server's load status indicators, performs the following:

[0139] If the first utilization rate is less than a preset first utilization rate threshold and the second utilization rate is less than a preset second utilization rate threshold, the load status index value is determined to meet the preset request acceptance condition.

[0140] If the first utilization rate is not less than a preset first utilization rate threshold, or the second utilization rate is not less than a preset second utilization rate threshold, the load status indicator value is determined to not meet the preset request acceptance condition.

[0141] In one exemplary embodiment, the server further includes a management thread group; the worker thread group includes at least one worker thread; the request processing apparatus further includes a registration module, which, after receiving request data sent by the client, is configured to:

[0142] Obtain the server's load status metric value when no target data channel corresponding to the transmission control connection is registered on any of the worker threads;

[0143] When the load status index value meets the preset request acceptance conditions, the target worker thread is determined from each worker thread through the management thread group, and the target data channel corresponding to the transmission control connection is registered on the target worker thread.

[0144] The request data sent by the client is transmitted to the target worker thread through the target data channel;

[0145] The requested data is processed by the target worker thread.

[0146] In an exemplary embodiment, the load status metric values ​​include the average processing time of each worker thread's task and the current total number of tasks; the registration module is also used for:

[0147] Based on the average processing time of each worker thread and the total number of current tasks, estimate the estimated processing time required for each worker thread to process the requested data.

[0148] The worker thread corresponding to the shortest estimated processing time is determined as the target worker thread.

[0149] In one exemplary embodiment, the registration module is further configured to:

[0150] If the shortest estimated processing time is less than or equal to the preset timeout threshold, the worker thread corresponding to the shortest estimated processing time will be determined as the target worker thread.

[0151] The registration module is also used for:

[0152] If the shortest estimated processing time exceeds the preset timeout threshold, generate response data to reject the request.

[0153] The response data is returned to the client.

[0154] In one exemplary embodiment, the request processing apparatus further includes a rejection module, which, after obtaining the server's load status indicator value, is configured to:

[0155] If the load status indicator value does not meet the preset conditions for accepting the request, response data is generated based on the load status indicator value to reject the request.

[0156] The response data is returned to the client.

[0157] Each module in the aforementioned request processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can invoke and execute the operations corresponding to each module.

[0158] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a request processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0159] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0162] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0164] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0165] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0166] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A request processing method, characterized in that, Applied to a server, wherein a server-side component is deployed on the server, the server-side component and the client have a pre-established transport control connection, and the server-side component includes a group of worker threads; the method includes: Receive request data sent by the client; If the target data channel corresponding to the transmission control connection is registered on the target worker thread in the worker thread group, the load status index value of the server is obtained. If the load status indicator value meets the preset request acceptance conditions, the request data is transmitted to the target worker thread through the target data channel; The requested data is processed by the target worker thread.

2. The method according to claim 1, characterized in that, The server includes a central processing unit (CPU) and memory. The CPU provides computing resources for the worker thread group, and the memory provides data storage space for the worker thread group. The load status metric includes a first utilization rate of the CPU and a second utilization rate of the memory. The preset conditions for accepting requests include that the first utilization rate is less than a preset first utilization rate threshold and the second utilization rate is less than a preset second utilization rate threshold. After obtaining the load status metric value of the server, the method further includes: If the first utilization rate is less than a preset first utilization rate threshold and the second utilization rate is less than a preset second utilization rate threshold, the load status index value is determined to meet the preset request acceptance condition. If the first utilization rate is not less than a preset first utilization rate threshold, or the second utilization rate is not less than a preset second utilization rate threshold, it is determined that the load status indicator value does not meet the preset request acceptance condition.

3. The method according to claim 1, characterized in that, The server also includes a management thread group; the worker thread group includes at least one worker thread; after receiving the request data sent by the client, the method further includes: If no target data channel corresponding to the transmission control connection is registered on any of the aforementioned worker threads, obtain the load status index value of the server. When the load status index value meets the preset acceptance request conditions, the target working thread is determined from each of the working threads through the management thread group, and the target data channel corresponding to the transmission control connection is registered on the target working thread; The request data sent by the client is transmitted to the target worker thread through the target data channel. The requested data is processed by the target worker thread.

4. The method according to claim 3, characterized in that, The load status index values ​​include the average processing time of each task corresponding to each worker thread and the total number of current tasks. Determining the target worker thread from each of the worker threads includes: Based on the average processing time of the task corresponding to each worker thread and the total number of current tasks, the estimated processing time required for each worker thread to process the request data is estimated. The worker thread corresponding to the shortest estimated processing time is determined as the target worker thread.

5. The method according to claim 4, characterized in that, The step of determining the worker thread corresponding to the shortest estimated processing time as the target worker thread includes: If the shortest estimated processing time is less than or equal to the preset timeout threshold, the worker thread corresponding to the shortest estimated processing time will be determined as the target worker thread. After estimating the estimated processing time required for each worker thread to process the request data based on the average processing time of the task corresponding to each worker thread and the total number of current tasks, the method further includes: If the shortest estimated processing time exceeds a preset timeout threshold, generate response data that rejects the requested data. The response data is returned to the client.

6. The method according to any one of claims 1 to 5, characterized in that, After obtaining the load status metric value of the server, the method further includes: If the load status indicator value does not meet the preset conditions for accepting the request, response data that rejects the request data is generated based on the load status indicator value. The response data is returned to the client.

7. A request processing apparatus, characterized in that, Applied to the server side, the server and client pre-establish a transmission control connection, the server includes a group of worker threads; the device includes: A receiving module is used to receive request data sent by the client; The acquisition module is used to acquire the load status index value of the server when the target data channel corresponding to the transmission control connection is registered on the target worker thread in the worker thread group. The transmission module is used to transmit the request data to the target worker thread through the target data channel when the load status index value meets the preset request acceptance conditions; The processing module is used to process the requested data through the target worker thread.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images