High-concurrency request processing method and device, computer equipment and storage medium
By acquiring system load and request credit scores, dynamically adjusting thread pools and rate limiting strategies, predicting request timeouts, and interrupting low-credit requests, the stability issues in high-concurrency scenarios in traditional technologies are solved, and stable processing of high-concurrency requests is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-01-14
- Publication Date
- 2026-04-17
AI Technical Summary
Traditional request processing technologies are prone to problems such as request queuing timeouts, response failures, and core business being squeezed out by non-core requests in high-concurrency, high-traffic scenarios, resulting in low request processing stability.
By acquiring system load and request credit scores, the thread pool size and rate limiting strategy are dynamically adjusted to predict whether a request will time out. When the load is too high, low-credit requests are allocated to the request queuing queue. If a timeout occurs, the request is interrupted and a preset response result is returned.
It effectively prevents issues such as request queuing timeouts and response loss, improves the stability of high-concurrency request processing, ensures that core requests are processed first, and achieves a stable response for each request.
Smart Images
Figure CN121887871A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of request and response technology, and in particular to a high-concurrency request processing method, apparatus, computer device, and storage medium. Background Technology
[0002] With the rapid development of the internet and large-scale model services, the demands on system request processing capabilities in high-concurrency, high-traffic scenarios are becoming increasingly stringent. Common high-concurrency scenarios include e-commerce flash sales, payment settlements, large-scale model API calls, and "Double Eleven" promotional events. Online e-commerce systems, banking transaction systems, and other business systems need to efficiently and stably handle high-concurrency requests to ensure user experience and business continuity. However, traditional request processing technologies rely on fixed thread pool sizes, static rate limiting strategies, and rigid priority mechanisms. Under high-traffic, high-concurrency scenarios, these technologies are prone to problems such as request queuing timeouts, response failures, and core business processes being squeezed out by non-core requests, resulting in low request processing stability. Therefore, improving the stability of high-concurrency request processing has become an urgent problem to solve. Summary of the Invention
[0003] This application provides a method, apparatus, computer device, and storage medium for processing high-concurrency requests, so as to improve the stability of high-concurrency request processing.
[0004] Firstly, this application provides a high-concurrency request processing method, the method comprising: Upon receiving concurrent requests from at least one client, the system load is obtained, and each request is analyzed to determine a credit score for each request. When the system load is greater than or equal to a preset load threshold, requests with credit scores lower than a preset score threshold are assigned to a request queuing queue, and it is predicted whether the request will time out. If the request times out, the request is interrupted and a preset response result is returned.
[0005] Secondly, this application also provides a high-concurrency request processing apparatus, the apparatus comprising: The credit scoring determination module is used to obtain the system load and analyze each request to determine the credit score of each request when receiving concurrent requests from at least one client. The prediction result acquisition module is used to allocate requests with credit scores lower than a preset score threshold to a request queuing queue when the system load is greater than or equal to a preset load threshold, and to predict whether the request will time out. The response result return module is used to interrupt the request and return a preset response result if the request will time out.
[0006] Thirdly, this application also provides a computer device, the computer device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the high-concurrency request processing method as described above when executing the computer program.
[0007] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the high-concurrency request processing method described above.
[0008] This application discloses a high-concurrency request processing method, apparatus, computer device, and storage medium. Upon receiving concurrent requests from at least one client, the method acquires the system load and analyzes each request to determine its credit score. When the system load is greater than or equal to a preset load threshold, requests with credit scores lower than the preset threshold are allocated to a request queuing queue, and the method predicts whether the request will time out. If the request is expected to time out, it is interrupted, and a preset response result is returned. This application determines the request processing strategy based on the system load status. When the load is too high, low-credit requests are allocated to a request queuing queue, and the method predicts whether they will time out. If a timeout is predicted, the request is immediately interrupted, and a preset response result is returned. This avoids timed-out requests continuously occupying system resources, effectively preventing problems such as request queuing timeouts, request swallowing, and response loss, thus improving the stability of concurrent request processing. Attached Figure Description
[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 This is a first schematic flowchart of a high-concurrency request processing method provided by an embodiment of this application; Figure 2 This is a second schematic flowchart of a high-concurrency request processing method provided in an embodiment of this application; Figure 3 This is a third schematic flowchart of a high-concurrency request processing method provided in an embodiment of this application; Figure 4 A schematic block diagram of a high-concurrency request processing apparatus provided for embodiments of this application; Figure 5 A schematic block diagram of the structure of a computer device provided for an embodiment of this application. Detailed Implementation
[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0012] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0013] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0014] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0015] This application provides a high-concurrency request processing method, apparatus, computer device, and storage medium. The high-concurrency request processing method can be applied to a server. Based on the system load status, a request processing strategy is determined. When the load is too high, low-credit requests are allocated to a request queuing queue, and the system predicts whether a timeout will occur. If a timeout is predicted, the process is immediately interrupted, and a preset response result is returned. This avoids timeout requests continuously consuming system resources, effectively preventing problems such as request queuing timeouts, request swallowing, and response loss, thus improving the stability of concurrent request processing. The server can be a standalone server or a server cluster.
[0016] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0017] Please see Figure 1 , Figure 1This is a schematic flowchart illustrating a high-concurrency request processing method provided in an embodiment of this application. This high-concurrency request processing method can be applied to a server to determine a request processing strategy based on system load status. When the load is too high, low-credit requests are allocated to a request queuing queue, and the system predicts whether a timeout will occur. If a timeout is predicted, the process is immediately interrupted, and a preset response result is returned. This avoids timeout requests from continuously occupying system resources, effectively preventing problems such as request queuing timeouts, request swallowing, and response loss, thus improving the stability of concurrent request processing.
[0018] like Figure 1 As shown, the high-concurrency request processing method specifically includes steps S101 to S103.
[0019] S101. Upon receiving concurrent requests from at least one client, obtain the system load, analyze each request, and determine the credit score of each request. In one embodiment, concurrent requests refer to multiple requests arriving at the system simultaneously or nearly simultaneously within a very short time interval (typically milliseconds or microseconds), competing for limited system resources (such as CPU (Central Processing Unit), memory, database connections, network bandwidth, etc.). For example, at midnight on "Double Eleven," during flash sales of popular products, or when large-scale APIs (Application Programming Interfaces) are called frequently, massive amounts of requests flood the online shopping system or server almost simultaneously.
[0020] In one embodiment, the current system load is obtained in real time, including but not limited to CPU utilization, QPS (Queries-per-second) and RT (Response Time).
[0021] In one embodiment, the request is parsed to obtain its characteristic information, including but not limited to request type, business identifier, and whether it carries a latency-sensitive flag. The priority of the request is matched based on the parsed characteristic information, and the credit score of the request is determined by combining the priority and system load.
[0022] Furthermore, the step of analyzing each request to determine the credit score of each request includes: analyzing each request to obtain the priority of each request; and performing a weighted analysis based on the priority and the system load to obtain the credit score of each request.
[0023] In one embodiment, for each of the received concurrent requests, based on the formula... The system calculates the real-time credit score corresponding to the request. Here, RCS is the request credit score, L is the current system load (including CPU, QPS, RT), P is the request priority, and T is the waiting time of the request in the queue. , , These are the weighting coefficients for system load, request priority, and wait time, respectively. It's understandable that for new requests not waiting in the queue, the wait time is zero.
[0024] In one embodiment, request priorities are assigned based on business importance, categorized as high, medium, and low. Core business requests (such as payment transactions and risk control verification) have high priority; routine business requests (such as user information queries and order status updates) have medium priority; and non-core, non-routine business requests (such as ad loading and log reporting) have low priority. The priority of each business request can be set according to actual needs.
[0025] Different priority levels are converted into numerical values, such as high priority P=3, medium priority P=2, and low priority P=1.
[0026] In one embodiment, the weighting coefficients for system load and request priority can be set according to actual needs.
[0027] In another embodiment, the weighting coefficients for system load and request priority can be adaptively adjusted based on system performance metrics. Specifically, key performance metrics are continuously collected, such as CPU utilization, QPS (queries per second, reflecting throughput), RT (average response time for requests, reflecting system processing speed), and ERR (error rate, reflecting system stability).
[0028] The collected key indicators are compared with the corresponding standards to determine how each indicator changes relative to the standard value, and the corresponding weight adjustment strategy is matched according to the changes.
[0029] For example, when the error rate is higher than the standard value, it indicates that the system is unstable and a conservative strategy needs to be adopted to protect the core business. The weight adjustment strategy is to increase the weight of priority P to ensure that the credit score of high-priority requests (such as payment and risk control) is higher than that of ordinary requests, so as to ensure that the core business can be processed in a timely manner when the system is unstable.
[0030] Furthermore, after receiving concurrent requests from at least one client, obtaining the system load, analyzing each request, and determining the credit score of each request, the method further includes: when the system load is greater than or equal to a preset load threshold, obtaining anomaly indicators of the system load; adjusting the system processing parameters corresponding to the anomaly indicators according to the concurrent requests to obtain an adjusted system load; and when the adjusted system load is less than the preset load threshold, allocating each request to a thread pool for processing to obtain a response result.
[0031] In one embodiment, when the system load is greater than or equal to a preset load threshold, it indicates the presence of one or more abnormal indicators, such as QPS exceeding the standard value, average response time exceeding the business-specified upper limit, or error rate exceeding the standard value.
[0032] Specifically, the adaptive feedback loop module monitors core metrics in real time to identify anomalies, such as QPS(t): whether the current number of requests per second exceeds the system's maximum processing capacity (e.g., a preset QPS threshold); RT(t): whether the average response time exceeds the specified upper limit (e.g., payment request RT > 2 seconds); and ERR(t): whether the request error rate is increasing (e.g., timeout error rate > 5%). When any of the above metrics exceeds the preset threshold, the system load is deemed abnormal. For example, if QPS(t) is greater than the preset QPS threshold, it indicates traffic overload; if RT(t) is greater than the specified upper limit, it indicates insufficient system processing capacity; and if ERR(t) is greater than the preset error rate threshold, it indicates decreased system stability.
[0033] In one embodiment, based on anomaly indicators and characteristics of concurrent requests (such as request type and priority distribution), system parameters are dynamically adjusted through an adaptive feedback loop (AFR), including adjusting the thread pool size: if QPS(t) is too high and RT(t) increases, the thread pool size is increased (to improve concurrent processing capability); if ERR(t) increases due to thread resource contention, the thread pool queue length is optimized (to reduce queuing blocking).
[0034] It can also optimize the weighting coefficients of various factors in the dynamic request credit score. Specifically, if high-priority requests (such as payments) account for a high proportion of concurrent requests, the β weight is increased (strengthening the impact of request priority P); if the request waiting time T is too long, the γ weight is increased (to avoid core requests queuing for a long time).
[0035] Alternatively, the rate limiting threshold can be adjusted. For low-priority requests (such as regular queries), the rate limiting threshold can be lowered (to reduce the resource consumption of non-core requests); for high-priority requests, the rate limiting threshold can be raised (to ensure the normal processing of core business processes).
[0036] In one embodiment, the aforementioned dynamic adjustment of system parameters, weighting coefficients, and flow limiting thresholds can be implemented through an Adaptive Feedback Loop (AFR). An Adaptive Feedback Loop (AFR) is an intelligent control mechanism that dynamically adjusts system parameters based on real-time monitoring indicators. It continuously collects core system performance indicators and automatically optimizes configurations to cope with traffic fluctuations in high-concurrency scenarios, replacing the traditional fixed-parameter mode and achieving millisecond-level adaptive adjustment. The function AFR(t) = f(QPS(t), RT(t), ERR(t)) analyzes indicator changes and triggers corresponding adjustments. All adjustments are completed within milliseconds, ensuring the system quickly adapts to real-time conditions.
[0037] In one embodiment, if the adjusted system load is less than the preset load threshold, it indicates that the system has recovered its normal processing capacity; if it still does not meet the threshold, the parameter adjustment process is repeated (such as further expanding the thread pool and optimizing the RCS weight) until the load drops below the threshold.
[0038] In one embodiment, when the adjusted load is normal, all requests are sorted in descending order of credit score, with requests that have higher credit scores being allocated first (e.g., payment requests take precedence over regular queries). The sorted requests are then distributed to the adjusted thread pool in sequence. After processing the requests, the thread pool returns a stable response.
[0039] In the above embodiments, the adaptive feedback loop enables millisecond-level dynamic parameter adjustment to adapt to instantaneous traffic fluctuations, allowing the system to remain stable under high load and achieve a stable response for each request even in high-traffic, high-concurrency scenarios.
[0040] S102. When the system load is greater than or equal to a preset load threshold, requests with credit scores lower than a preset score threshold are assigned to a request queuing queue, and it is predicted whether the request will time out. In one embodiment, when the system load is greater than or equal to a preset load threshold, queue allocation logic is triggered. Specifically, requests with credit scores lower than a preset score threshold are allocated to an interruptible queuing queue. At the same time, their estimated completion time is predicted and compared with the upper limit of response time to determine whether a timeout will occur.
[0041] In another embodiment, requests with credit scores higher than or equal to a preset score threshold are typically core, high-priority requests (such as payments), so they are allocated to a thread pool for processing even when the system load is high, to ensure that critical business operations are not affected.
[0042] Furthermore, predicting whether the request will time out includes: obtaining the upper limit of the response time corresponding to the request; analyzing the system load and the request based on a preset prediction model to obtain the predicted response time of the request; and comparing the predicted response time with the upper limit of the response time to determine whether the request will time out.
[0043] In one embodiment, the upper limit of the response time for a request can be carried in the request header, indicating the maximum time the client is willing to wait, or it can be set according to business requirements.
[0044] In one embodiment, the predicted response time is the system's estimate of the total time from the current moment to the completion of processing the request. The prediction model is trained based on historical request and response time data to estimate the total time from the current moment to the completion of processing the request.
[0045] In a specific embodiment, the predicted response time includes the estimated queuing time and the estimated processing time. The prediction model predicts the queuing time of a request based on the requests in the queue and the processing speed parameters in the system load, and predicts the request processing time based on the system load (such as CPU utilization, current QPS, average response time, thread pool status) and the request's own characteristic information (such as request type, historical processing time, etc.). The sum of the queuing time and the request processing time is the predicted response time. In one embodiment, the predicted response time is compared with the upper limit of the response time. If the predicted response time is greater than the upper limit of the response time, the request is determined to time out. If the predicted response time is not greater than the upper limit of the response time, the request is determined not to time out.
[0046] S103. If the request will time out, the request will be interrupted and a preset response result will be returned.
[0047] In one embodiment, if the request times out, an interruption operation is performed to actively terminate the request and return a preset response result. The preset response result includes, but is not limited to, a non-failure message, such as "The current product query volume is high, please try again later," to avoid resource consumption and system crashes.
[0048] In the above implementation, the request processing strategy is determined by the system load status. When the load is too high, low-credit requests are allocated to the request queuing queue, and it is predicted whether a timeout response will occur. If a timeout response is predicted, the process is immediately interrupted and a preset response result is returned. This avoids timeout requests from continuously occupying system resources, effectively preventing problems such as request queuing timeout, request swallowing, and response loss, improving the stability of concurrent request processing, and achieving a stable response for each request.
[0049] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating a high-concurrency request processing method provided in an embodiment of this application. This high-concurrency request processing method can be applied to a server to determine the queuing order by comparing the credit score of a request with the credit scores of existing requests in the queue. This ensures that high-priority requests are processed first even when in the queue, avoiding being squeezed out by low-priority requests and guaranteeing the stability of critical links in high-concurrency scenarios.
[0050] like Figure 2 As shown, the high-concurrency request processing method specifically includes steps S201 to S203.
[0051] S201. If the request will not time out, the queuing order of the request is determined based on the credit score of the request. In one embodiment, if a request will not time out, its credit score is sorted against the credit scores of existing requests in the queue, and the queuing order of the requests is determined based on the sorting result. It is understood that the higher the credit score of the current request is in the queue, the closer it is to the head of the queue, and the more likely it is to receive processing opportunities.
[0052] S202. When the latest system load is detected to be less than the preset load threshold, the requests in the request queue are allocated to the thread pool for processing according to the queuing order to obtain the response result.
[0053] In one embodiment, the system load is continuously monitored. When the latest monitored system load is less than a preset load threshold, it indicates that there are idle system resources. At this time, the requests in the queue are allocated to the thread pool for processing according to the queuing order to obtain the response result.
[0054] In another embodiment, when the system load is greater than or equal to a preset load threshold, the system processing parameters can be dynamically adjusted based on abnormal indicators in the system load, such as dynamically increasing the thread pool size to raise the system load threshold so that the system load is less than the load threshold.
[0055] Further, the step of allocating requests in the request queue to a thread pool for processing according to the queuing order to obtain the response result includes: extracting pending requests from the request queue according to the queuing order; obtaining the waiting time of the pending requests, and determining the credit score of the pending requests based on the latest system load, the waiting time, and the priority of the pending requests; determining the processing order of the pending requests based on the credit score of the pending requests, and allocating the pending requests to a thread pool for processing according to the processing order of the pending requests to obtain the response result.
[0056] In one embodiment, requests in the queue are sorted in descending order based on credit score; the higher the credit score, the closer the request is to the head of the queue. When retrieving requests to be processed, requests are retrieved sequentially starting from the head of the queue, prioritizing requests with high credit scores to ensure that requests for core business functions (such as payments and risk control) are processed first.
[0057] In one embodiment, there may be more than one pending request extracted, and the specific number is determined based on the number of idle threads in the thread pool.
[0058] In one embodiment, a credit score is calculated based on the latest system load, the waiting time of extracted pending requests, and their priority. The credit score calculation formula is as follows:
[0059] Where RCS is the request credit score, L is the current system load (including CPU, QPS, RT), P is the request priority, and T is the waiting time of the request in the queue. , , These are the weighting coefficients for system load, request priority, and waiting time, respectively.
[0060] In one embodiment, the original credit score of each pending request is replaced with the latest calculated credit score, and the pending requests are sorted according to the latest credit score to obtain the processing order. The pending requests are then assigned to idle threads in the thread pool according to the processing order.
[0061] In another embodiment, requests with higher processing order can be assigned to idle threads with higher processing capacity based on the processing order and the processing capacity of idle threads, ensuring that high-priority requests are responded to quickly.
[0062] In the above embodiments, by comparing the credit score of a request with the credit scores of existing requests in the queue, the queuing order is determined, ensuring that high-priority requests are processed first even if they are in the queue, avoiding being squeezed out by low-priority requests, and ensuring the stability of critical links in high-concurrency scenarios.
[0063] Please see Figure 3 , Figure 3 This is a schematic flowchart illustrating a high-concurrency request processing method provided in an embodiment of this application. This high-concurrency request processing method can be applied to servers to ensure that core requests remain in a priority processing position regardless of load changes by updating the credit score and processing order of requests in real time. This solves the problem of core requests being squeezed out by non-core requests in traditional fixed-order processing, thus improving the stability of core requests.
[0064] like Figure 3As shown, the high-concurrency request processing method specifically includes steps S301 to S304.
[0065] S301. When the system load is less than the preset load threshold, determine the current processing request among the requests based on the credit score of each request. S302. Allocate the current processing request to the thread pool for processing, and obtain the system load during the processing; S303. Based on the system load during the processing, update the credit score of each pending request, and update the processing order of each pending request based on the credit score of each pending request. S304. According to the updated processing order, each of the pending requests is assigned to a thread pool for processing, and the response result of each pending request is obtained.
[0066] In one embodiment, when the system load is less than the load threshold, it indicates that there are idle system resources. In this case, the request with the highest credit score is processed first to ensure that core business requests are responded to first.
[0067] The request with the highest credit score is selected as the current processing request, and the current processing request is assigned to an idle thread in the thread pool for processing.
[0068] In one embodiment, the credit score calculation formula is adaptively adjusted because the system load changes during request processing. Weighting coefficients in , , Therefore, during request processing, system load is continuously monitored. The credit scores of other pending requests are dynamically updated based on the real-time monitored system load, and the processing order of the remaining pending requests is readjusted according to the credit scores.
[0069] For example, there are three concurrent requests: a regular query, a VIP user request, and a report generation request, with corresponding original credit scores of 60, 85, and 50, respectively. The VIP user request is processed first based on its credit score. However, processing the VIP user request involves calling a slow credit scoring interface, causing increased load and slower response. Therefore, the credit scores for the regular query and report generation requests are recalculated based on the current load, and the processing order is re-determined.
[0070] In one embodiment, requests are retrieved sequentially from the updated processing order header and assigned to a thread pool for processing. Once the thread pool has completed its processing, a stable response result is generated.
[0071] In another embodiment, if the system load is monitored in real time during the request processing and is greater than or equal to a preset load threshold, the remaining pending requests are sequentially allocated to the queuing queue. At the same time, timeout response prediction is performed. For requests that will time out, the system is immediately interrupted and a preset response result is fed back to the user. For requests that will not time out, they are queued according to their credit score and the credit score of the original requests in the queuing queue.
[0072] In the above embodiments, by updating the credit score and processing order of requests in real time, core requests are guaranteed to remain in a priority position as the load changes, which solves the problem of core requests being squeezed out by non-core requests in the traditional fixed order and improves the stability of core requests.
[0073] Please see Figure 4 , Figure 4 This embodiment of the present application provides a schematic block diagram of a high-concurrency request processing device, which is used to execute the aforementioned high-concurrency request processing method. The high-concurrency request processing device can be configured on a server.
[0074] like Figure 4 As shown, the high-concurrency request processing device 400 includes: The credit score determination module 401 is used to obtain the system load and analyze each request to determine the credit score of each request when receiving concurrent requests sent by at least one client. The prediction result acquisition module 402 is used to allocate requests with credit scores lower than a preset score threshold to a request queuing queue when the system load is greater than or equal to a preset load threshold, and to predict whether the request will time out. The response result return module 403 is used to interrupt the request and return a preset response result if the request will time out.
[0075] Furthermore, the prediction result obtaining module 402 includes: The duration limit acquisition unit is used to acquire the response duration limit corresponding to the request; The response time prediction unit is used to analyze the system load and the request based on a preset prediction model to obtain the predicted response time of the request. The timeout response determination unit is used to compare the predicted response time with the upper limit of the response time to determine whether the request will time out.
[0076] Furthermore, the high-concurrency request processing device 400 also includes: A queuing sequence determination module is used to determine the queuing order of the request based on the credit score of the request if the request will not time out. The response result acquisition module is used to allocate the requests in the request queue to the thread pool for processing according to the queuing order when the latest system load is detected to be less than the preset load threshold, and obtain the response result.
[0077] Furthermore, the response result acquisition module includes: A pending request extraction unit is used to extract pending requests from the request queue according to the queuing order. A credit scoring determination unit is used to obtain the waiting time of the pending request and determine the credit score of the pending request based on the latest system load, the waiting time, and the priority of the pending request. The response result acquisition unit is used to determine the processing order of the pending requests based on the credit score of the pending requests, and to allocate the pending requests to the thread pool for processing according to the processing order of the pending requests, so as to obtain the response result.
[0078] Furthermore, the high-concurrency request processing device 400 also includes: The current processing request determination module is used to determine the current processing request among the requests based on the credit score of each request when the system load is less than the preset load threshold. The system load acquisition module is used to allocate the currently processed request to the thread pool for processing and to acquire the system load during the processing. The processing order update module is used to update the credit score of each pending request according to the system load during the processing, and update the processing order of each pending request based on the credit score of each pending request. The response result acquisition module is used to allocate each of the pending requests to a thread pool for processing according to the updated processing order, and obtain the response result of each of the pending requests.
[0079] Furthermore, the high-concurrency request processing device 400 also includes: An abnormal indicator acquisition module is used to acquire abnormal indicators of the system load when the system load is greater than or equal to a preset load threshold. The system parameter adjustment module is used to adjust the system processing parameters corresponding to the abnormal indicators according to the concurrent requests, so as to obtain the adjusted system load. The response result acquisition module is used to allocate each of the requests to a thread pool for processing and obtain response results when the adjusted system load is less than the preset load threshold.
[0080] Furthermore, the credit scoring determination module 401 includes: The request analysis unit is used to analyze each of the requests and obtain the priority of each request; The credit score acquisition unit is used to perform a weighted analysis based on the priority and the system load to obtain a credit score for each request.
[0081] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and modules can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0082] The aforementioned device can be implemented as a computer program, which can be used in, for example... Figure 5 It runs on the computer device shown.
[0083] Please see Figure 5 , Figure 5 This is a schematic block diagram illustrating the structure of a computer device according to an embodiment of this application. The computer device may be a server.
[0084] See Figure 5 The computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.
[0085] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any high-concurrency request processing method.
[0086] The processor provides computing and control capabilities, supporting the operation of the entire computer device.
[0087] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When these computer programs are executed by the processor, the processor can perform any high-concurrency request processing method.
[0088] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0089] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.
[0090] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Upon receiving concurrent requests from at least one client, the system load is obtained, and each request is analyzed to determine a credit score for each request. When the system load is greater than or equal to a preset load threshold, requests with credit scores lower than a preset score threshold are assigned to a request queuing queue, and it is predicted whether the request will time out. If the request times out, the request is interrupted and a preset response result is returned.
[0091] In one embodiment, the processor, in order to predict whether the request will time out, implements the following: Get the upper limit of the response time corresponding to the request; The system load and the request are analyzed based on a preset prediction model to obtain the predicted response time of the request; The predicted response time is compared with the upper limit of the response time to determine whether the request will time out.
[0092] In one embodiment, after the processor allocates requests with credit scores lower than a preset score threshold to a request queuing queue when the system load is greater than or equal to a preset load threshold, and predicts whether the request will time out, it is further configured to: If the request will not time out, the queuing order of the request will be determined based on the credit score of the request. When the latest system load is detected to be less than the preset load threshold, the requests in the request queue are allocated to the thread pool for processing according to the queuing order to obtain the response result.
[0093] In one embodiment, when the processor allocates requests in the request queue to a thread pool for processing according to the queuing order and obtains the response result, it is configured to: Requests to be processed are extracted from the request queue according to the queuing order. The waiting time of the pending request is obtained, and the credit score of the pending request is determined based on the latest system load, the waiting time, and the priority of the pending request. Based on the credit score of the pending requests, the processing order of the pending requests is determined, and the pending requests are allocated to the thread pool for processing according to the processing order to obtain the response result.
[0094] In one embodiment, after analyzing each request and determining the credit score of each request, the processor is further configured to: When the system load is less than the preset load threshold, the current processing request is determined from among the requests based on the credit score of each request; The current processing request is assigned to a thread pool for processing, and the system load during the processing is obtained; Based on the system load during the processing, update the credit score of each pending request, and based on the credit score of each pending request, update the processing order of each pending request. According to the updated processing order, each pending request is assigned to a thread pool for processing, and the response result of each pending request is obtained.
[0095] In one embodiment, after the processor receives concurrent requests from at least one client, acquires the system load, analyzes each request, and determines the credit score of each request, it is further configured to: When the system load is greater than or equal to a preset load threshold, obtain abnormal indicators of the system load. Based on the concurrent requests, the system processing parameters corresponding to the abnormal indicators are adjusted to obtain the adjusted system load; When the adjusted system load is less than the preset load threshold, each request is assigned to a thread pool for processing to obtain a response result.
[0096] In one embodiment, when the processor analyzes each request and determines a credit score for each request, it is configured to: Each request is analyzed to determine its priority. A weighted analysis is performed based on the priority and the system load to obtain a credit score for each request.
[0097] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement any of the high-concurrency request processing methods provided in the embodiments of this application.
[0098] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.
[0099] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A high-concurrency request processing method, characterized in that, include: Upon receiving concurrent requests from at least one client, the system load is obtained, and each request is analyzed to determine a credit score for each request. When the system load is greater than or equal to a preset load threshold, requests with credit scores lower than a preset score threshold are assigned to a request queuing queue, and it is predicted whether the request will time out. If the request times out, the request is interrupted and a preset response result is returned.
2. The high-concurrency request processing method of claim 1, wherein, The prediction of whether the request will time out includes: Get the upper limit of the response time corresponding to the request; The system load and the request are analyzed based on a preset prediction model to obtain the predicted response time of the request; The predicted response time is compared with the upper limit of the response time to determine whether the request will time out.
3. The high-concurrent request processing method of claim 1, wherein, After assigning requests with credit scores lower than a preset score threshold to a request queuing queue when the system load is greater than or equal to a preset load threshold, and predicting whether the request will time out, the method further includes: If the request will not time out, the queuing order of the request will be determined based on the credit score of the request. When the latest system load is detected to be less than the preset load threshold, the requests in the request queue are allocated to the thread pool for processing according to the queuing order to obtain the response result.
4. The high-concurrency request processing method according to claim 3, characterized in that, The step of allocating requests in the request queue to a thread pool for processing according to the queuing order to obtain the response result includes: Requests to be processed are extracted from the request queue according to the queuing order. The waiting time of the pending request is obtained, and the credit score of the pending request is determined based on the latest system load, the waiting time, and the priority of the pending request. Based on the credit score of the pending requests, the processing order of the pending requests is determined, and the pending requests are allocated to the thread pool for processing according to the processing order to obtain the response result.
5. The high-concurrency request processing method according to claim 1, characterized in that, After analyzing each request and determining the credit score for each request, the method further includes: When the system load is less than the preset load threshold, the current processing request is determined from among the requests based on the credit score of each request; The current processing request is assigned to a thread pool for processing, and the system load during the processing is obtained; Based on the system load during the processing, update the credit score of each pending request, and based on the credit score of each pending request, update the processing order of each pending request. According to the updated processing order, each pending request is assigned to a thread pool for processing, and the response result of each pending request is obtained.
6. The high-concurrency request processing method according to claim 1, characterized in that, After receiving concurrent requests from at least one client, obtaining the system load, analyzing each request, and determining the credit score of each request, the method further includes: When the system load is greater than or equal to a preset load threshold, obtain abnormal indicators of the system load. Based on the concurrent requests, the system processing parameters corresponding to the abnormal indicators are adjusted to obtain the adjusted system load; When the adjusted system load is less than the preset load threshold, each request is assigned to a thread pool for processing to obtain a response result.
7. The high-concurrency request processing method according to any one of claims 1 to 6, characterized in that, The analysis of each request to determine the credit score for each request includes: Each request is analyzed to determine its priority. A weighted analysis is performed based on the priority and the system load to obtain a credit score for each request.
8. A high-concurrency request processing device, characterized in that, include: The credit scoring determination module is used to obtain the system load and analyze each request to determine the credit score of each request when receiving concurrent requests from at least one client. The prediction result acquisition module is used to allocate requests with credit scores lower than a preset score threshold to a request queuing queue when the system load is greater than or equal to a preset load threshold, and to predict whether the request will time out. The response result return module is used to interrupt the request and return a preset response result if the request will time out.
9. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the high-concurrency request processing method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the high-concurrency request processing method as described in any one of claims 1 to 7.