A method and system for dynamically scheduling microservice gateway based on coroutine

Abstract

The present invention discloses a method for dynamically scheduling a microservice gateway based on a coroutine, including: 1. creating a coroutine pool to process network requests, the gateway accepts network requests and adds network requests to a request queue; 2. identifying the request type and assigning a preset static priority to the network request, and adjusting the dynamic priority based on the static priority and the configured preliminary dynamic priority in combination with the characteristics; 3. collecting data in the coroutine operation in real time, and according to the data operation status, by calculating the timeout threshold, adopting timeout differentiation strategy, low priority discard strategy and data status update, these measures reduce the coroutine processing delay. The present invention also discloses a dynamically scheduling microservice gateway system based on a coroutine. The present invention can significantly improve gateway performance, reduce system latency, and intelligently manage request queues under high load conditions to ensure the normal operation of key businesses, and can be widely used in the field of computer data.

Description

Technical Field

[0001] The present invention relates to the field of computer data, and in particular to a method and system for dynamically scheduling microservice gateways based on coroutines. Background Art

[0002] In modern software development, microservice architecture has become the mainstream of enterprise application development due to its high degree of decoupling and scalability. In the current microservice architecture, the gateway, as the entrance to the system, undertakes important functions such as request routing, load balancing, and security authentication. Traditional microservice gateway systems usually use a multi-threaded model to handle concurrent requests. In scenarios where the hardware server performance is insufficient, this method has some limitations and shortcomings when handling a large number of concurrent connections.

[0003] When processing concurrent requests, the multi-threaded model creates multiple threads to handle different tasks. The separation of network threads and business threads has become increasingly prominent, which has led to the following problems: a. Increased system complexity: two different types of thread pools need to be managed; b. Sharp increase in inter-thread communication: data transfer between network threads and business threads requires additional synchronization mechanisms; c. Increased latency: additional latency is introduced when requests are transferred between network threads and business threads.

[0004] The multi-threaded model itself has great limitations: a. A large amount of resources are occupied: each thread requires an independent stack space. When the concurrency is large, a large amount of system resources will be consumed; b. The context switching cost is high: frequent thread switching will lead to higher CPU overhead; c. Limited scalability: an increase in the number of threads will lead to a decrease in system performance.

[0005] At the same time, in actual application scenarios, some interfaces of background services, especially those involving complex data processing or historical data query, may take much longer to execute than regular requests. For example, historical report data query may take 10 seconds or more to complete. Such long-executed interfaces face significant challenges when processed in microservice gateways. If the microservice gateway does not implement differentiated timeout strategies for interfaces in different business scenarios, interface requests that take too long to execute may fail due to timeout, especially in high concurrency situations. If the microservice gateway sets a unified longer timeout, although it can accommodate requests with long execution times, this approach may cause the entire request queue to quickly become blocked when the system encounters an abnormal situation (such as a sudden abnormality in a high-frequency, low-time interface causing an excessively long execution time), thereby affecting the normal operation of other businesses.

[0006] In this case, when business requests are unavailable due to large-scale timeouts, the request queue of the microservice gateway is easily filled up quickly under high concurrency conditions, resulting in system denial of service. The existing processing method discards requests indiscriminately: when the queue is full, simply discarding newly arrived requests may result in the loss of important high-priority requests; low-priority requests may occupy queue resources for a long time, affecting the overall system efficiency. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the above-mentioned background technology and provide a method and system for dynamically scheduling microservice gateways based on coroutines, so that by introducing coroutine models, merging network and business processing threads, realizing dynamic priority scheduling mechanisms and differentiated timeout strategies and other innovative methods, the present invention can significantly improve gateway performance, reduce system latency, and intelligently manage request queues under high load conditions to ensure the normal operation of key businesses.

[0008] The present invention provides a coroutine-based dynamic scheduling microservice gateway method, comprising the following steps: step 1, creating a coroutine pool to process network requests, the gateway accepts the network request and adds the network request to the request queue, and assigns an idle coroutine to process and forward the network request; step 2, identifying the request type and assigning a preset static priority to the network request, and adjusting the dynamic priority based on the static priority and the configured preliminary dynamic priority, combining the characteristics of request frequency, response time and system load; step 3, collecting the request frequency, response time, queue length, system load and interface execution data in the coroutine operation in real time, and according to the data operation status, by calculating the timeout threshold, adopting the timeout differentiation strategy, low priority discard strategy and data status update to reduce the coroutine processing delay.

[0009] In the above technical scheme, in the step 2, the adjustment of dynamic priority is to calculate the dynamic priority based on the comprehensive characteristics and the ridge regression model, and the specific process is as follows: configure the dynamic priority weights of each characteristic of the static priority, request frequency, response time and system load of the interface; calculate the contribution values ​​of the individual characteristics of static priority, request frequency, response time and system load respectively; calculate the combined characteristic value according to the individual characteristic contribution values ​​of static priority, request frequency, response time and system load; based on the respective coefficients and combined characteristic values ​​of static priority, request frequency, response time and system load, use the ridge regression model to calculate the average response time and use it as the dynamic priority fitting coefficient; output the dynamic priority of the network request based on the respective coefficients of static priority, request frequency, response time and system load and the configured respective dynamic priority weights.

[0010] In the above technical solution, in step 2, the dynamic priority calculation formula based on ridge regression is as follows: ,in, Indicates dynamic priority, represents the intercept term, Indicates static priority, Indicates the request frequency, Indicates the response time, Indicates the system load, represents the regression coefficient of the static priority feature, represents the regression coefficient of the request frequency feature, represents the regression coefficient of the response time characteristic, Regression coefficient representing the system load characteristics, represents the error term.

[0011] In the above technical solution, in step three, the specific process of calculating the timeout threshold is as follows: according to the historical monitoring data of the interface, the timeout time of each interface is calculated based on the wavelet decomposition of Daubechies; based on the multivariate data obtained after the Daubechies wavelet decomposition, the iTransformer model is reversely reconstructed to obtain the final timeout threshold.

[0012] In the above technical scheme, the specific process of the Daubechies wavelet decomposition introduced in the calculation of the timeout threshold in step three is as follows: obtain the interface historical monitoring data, and use Daubechies wavelet decomposition to split various types of historical monitoring data; fix the number of layers of Daubechies wavelet decomposition to a preset value; perform discrete wavelet transform on each parameter sequence to obtain approximate coefficients and detail coefficients of various scales, and set the step size of the approximate coefficients and detail coefficients to T; analyze the approximate coefficients and detail coefficients of each scale, identify the characteristics of each type of parameters, and set the number of parameter characteristics to N.

[0013] In the above technical solution, the Daubechies wavelet decomposition algorithm introduced in the calculation of the timeout threshold in step 3 is as follows: Input: Input the backtracking time series ; input length T; prediction length S; number of variables N; token dimension D; number of iTransformer blocks L; wavelet decomposition level J; then for the variable , perform the following steps: 1) , extract the time series of the nth variable; 2) = Wavelet_Daubechies(X_n Level=J); 3) X[: n] = concat([ ]) Connect the decomposed coefficients; 4) X = X.Transpose.

[0014] In the above technical solution, the specific process of the iTransformer algorithm used in calculating the timeout threshold in step three is as follows: convert the multivariate data after Daubechies wavelet decomposition through the Embedding model and input it into TrmBlock to obtain a sequence value; use a multilayer perceptron to embed the sequence value into the variable Tokens in the last dimension; loop the following operations for the variable Tokens: use the first layer normalization and self-attention mechanism to update the output value of the previous layer to obtain a first update value; use the first feedforward network sequence to represent the first update value and broadcast it to each variable token; use the second layer normalization and the second feedforward network to update each variable token to obtain a second update value; use the third layer normalization on the sequence table to reduce variable differences; use a multilayer perceptron to process and output the final variable, transpose the final variable and use the transposed result as the prediction result.

[0015] In the above technical solution, in step three, the specific process of the timeout differentiation strategy is as follows: according to the interface historical monitoring data and the expected execution time, the interfaces are classified into fast interfaces, ordinary interfaces and long execution interfaces; by calculating the timeout threshold, the corresponding initial timeout time is set for various categories of interfaces, and dynamically adjusted according to the real-time monitoring data; when the execution time of each interface is close to the timeout threshold, an early warning is triggered and consideration is given to extending the timeout time or transferring the request to the long execution queue.

[0016] In the above technical solution, in step three, the specific process of the low priority discard strategy is as follows: when it is detected that the queue length is close to a certain proportion of the set value, the low priority discard strategy is started; according to the current system status, a priority threshold is dynamically set; the dynamic priority of the newly arrived request is calculated; the dynamic priority of the request is compared with the current priority threshold, and requests below the priority threshold will be directly discarded; for requests with a longer expected execution time, the request is transferred to a special long-time execution queue and processed by an independent coroutine pool.

[0017] The present invention also provides a coroutine-based dynamic scheduling microservice gateway system, which has a computer program that can execute a coroutine-based dynamic scheduling microservice gateway method.

[0018] The present invention provides a method and system for dynamically scheduling microservice gateways based on coroutines, which has the following beneficial effects:

[0019] 1. Use coroutines instead of multithreading to reduce thread switching overhead and improve system performance;

[0020] 2. Unify network I / O and business logic into the coroutine to reduce processing delay;

[0021] 3. Through dynamic priority scheduling, important requests can be processed first under high load conditions, improving the availability and stability of the system;

[0022] 4. The differentiated timeout strategy effectively solves the problem of large differences in interface execution time in different business scenarios, improving the system's adaptability and resource utilization efficiency;

[0023] 5. Intelligent request queue management and discard strategy ensure that the system can maintain normal operation of core functions under extreme circumstances. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a schematic diagram of the architecture of each function of the dynamic scheduling microservice gateway based on coroutines of the present invention;

[0025] Figure 2 This is a functional architecture diagram of a coroutine pool unit in a coroutine-based dynamic scheduling microservice gateway of the present invention;

[0026] Figure 3 This is a functional architecture diagram of a request queue unit in a dynamic scheduling microservice gateway based on coroutines of the present invention;

[0027] Figure 4 This is a functional architecture diagram of a priority scheduling unit in a dynamic scheduling microservice gateway based on coroutines of the present invention;

[0028] Figure 5 This is a functional architecture diagram of a threshold prediction calculation unit in a dynamic scheduling microservice gateway based on coroutines of the present invention;

[0029] Figure 6 This is a functional architecture diagram of a low priority discard strategy unit in a dynamic scheduling microservice gateway based on coroutines of the present invention;

[0030] Figure 7 This is a functional architecture diagram of the timeout differentiation strategy unit in the dynamic scheduling microservice gateway based on coroutines of the present invention;

[0031] Figure 8 The overall flow chart of the method for dynamically scheduling microservice gateway based on coroutines of the present invention;

[0032] Fig. 9 A schematic diagram of the key component architecture of each TrmBlock of the iTransformer algorithm in the threshold prediction calculation step in the dynamic scheduling microservice gateway method based on coroutines of the present invention;

[0033] Fig.10 It is a structural diagram of the dynamic scheduling microservice gateway system based on coroutines of the present invention. DETAILED DESCRIPTION

[0034] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the embodiments should not be construed as limiting the present invention.

[0035] In order to solve the performance bottlenecks and complexity problems that traditional multi-threaded solutions often face when facing high concurrency and large-scale data processing, as well as to cope with the challenge of large differences in interface execution time in different business scenarios, the present invention proposes a method and system for dynamically scheduling microservice gateways based on coroutines. By introducing innovative methods such as coroutine models, merging network and business processing threads, implementing dynamic priority scheduling mechanisms, and timeout differentiation strategies, the present invention can significantly improve gateway performance, reduce system latency, and intelligently manage request queues under high load conditions to ensure the normal operation of key businesses.

[0036] See also Figure 1 The architecture of the dynamic scheduling microservice gateway based on coroutines of the present invention is as follows:

[0037] The coroutine pool unit is used to create and manage multiple coroutines, each of which can handle network I / O and business logic at the same time.

[0038] The request queue unit is used to store pending requests.

[0039] The priority scheduling unit is used to dynamically schedule requests based on the priority of the requests and the system load.

[0040] The business processing unit is used to process specific business logic, including service routing, load balancing, etc.

[0041] The monitoring unit is used to monitor various indicators of the system in real time and provide a basis for priority scheduling.

[0042] The threshold prediction calculation unit dynamically predicts the timeout threshold based on the monitoring data.

[0043] The low priority discard policy unit is used to selectively discard low priority requests according to the dynamic priority of the requests when the system is under high load, to ensure that high priority requests can be successfully responded to.

[0044] The timeout differentiation policy unit is used to set different timeout policies according to the characteristics of different interfaces.

[0045] See also Figure 2 , the coroutine pool unit includes:

[0046] The coroutine creation subunit is used to create a specified number of coroutines according to the system configuration;

[0047] The coroutine scheduling subunit is used to assign pending requests to idle coroutines;

[0048] The coroutine recycling subunit is used to recycle the coroutine that has completed the task and put it into the idle state.

[0049] See also Figure 3 , the request queue unit comprises:

[0050] The enqueue subunit is used to add the newly arrived request to the queue.

[0051] The dequeue subunit is used to take requests out of the queue for processing.

[0052] join Figure 4 , the priority scheduling unit comprises:

[0053] A static priority calculation subunit, used to calculate the static priority of the request based on the request type factor;

[0054] Dynamically adjust the sub-units, based on the static priorities, and dynamically adjust the priorities based on the following factors:

[0055] Request frequency (request_frequency): the number of requests of this type per unit time;

[0056] Response time (response_time): the average response time for this type of request;

[0057] System load (system_load): The current system CPU and memory usage.

[0058] The scheduling execution subunit is used to discard low-priority requests according to the priority when the queue is close to full load, so as to ensure the normal response of high-priority requests;

[0059] Static priority, pre-assign static priorities to different interfaces or services, for example: transaction quotation interface = highest priority; business processing interface: medium priority; visitor interface: low priority.

[0060] Dynamic priority calculation formula:

[0061] Algorithm: Dynamic priority calculation of comprehensive features and ridge regression (see below for the specific calculation process)

[0062] Business processing unit: decompresses and decrypts the received client request, obtains the original request content, makes routing judgment based on the interface request type, and selects the corresponding background service for forwarding routing.

[0063] The monitoring unit collects system operation data in real time, including indicators such as request frequency, response time, queue length, system load, and interface execution time. These data are coordinated and linked with the priority scheduling unit in the following ways:

[0064] Data update: The monitoring unit updates the values ​​of the above indicators at regular time intervals. In one or more embodiments, the time interval is 30 seconds.

[0065] Threshold trigger: When a metric exceeds the timeout threshold, the priority is recalculated.

[0066] Periodic recalculation: Even if the threshold is not triggered, the dynamic priority of all requests is recalculated at a fixed period (such as 60 seconds).

[0067] Execution time analysis: Continuously analyze the actual execution time of different interfaces to optimize the prediction of expected execution time.

[0068] See also Figure 5 , threshold prediction calculation unit:

[0069] In order to meet the real-time diagnosis of the timeout threshold, Daubechies-based wavelet decomposition is first introduced to make a first prediction for monitoring data such as the number of calls, average execution time, number of timeouts, number of errors, etc. In one or more embodiments, the first prediction is to calculate the timeout time of each interface. In the present invention, Daubechies wavelet is selected as the basis function because Daubechies wavelet has good localization characteristics and orthogonality, and can effectively capture subtle changes and mutations in the signal. The present invention fixes the number of layers of wavelet decomposition to a predetermined value, which can ensure the consistency and comparability of the analysis. The fixed number of decomposition layers enables standardized comparisons between different time points and different parameters, and also simplifies the implementation and tuning process of the algorithm.

[0070] The specific scheme is as follows: 1) In this project, the number of layers of Daubechies wavelet decomposition is fixed to 4; 2) Discrete wavelet transform is performed on each parameter sequence to obtain approximate coefficients and detail coefficients of different scales, with a step size of T; 3) The coefficients of each scale are analyzed to identify the characteristics of different types of parameters, with a number of N. For example, static priority may be mainly reflected in the low-frequency part, while request frequency and response time may be more reflected in the high-frequency part.

[0071] The iTransformer architecture is used to perform a second prediction based on the multivariate data after Daubechies wavelet decomposition. In one or more embodiments, the second prediction is the reverse reconstruction of the iTransformer model to obtain the final timeout threshold. The second prediction is embedded in each patch from the perspective of multivariate regression, and the intrinsic correlation between multiple variables is deeply mined through the transposed matrix combined with the self-attention mechanism. Combined with the introduction of layer normalization and feedforward network, the model's ability to learn and represent the global features of the time series is enhanced to improve the accuracy of the final time series data prediction.

[0072] See also Figure 6 , low priority drop policy unit:

[0073] As a key component that works with the priority scheduling unit and the request queue unit, the low priority discard strategy plays a vital role under high system load conditions.

[0074] Trigger condition: When the request queue unit detects that the queue length is close to 90% of the set value, it notifies the priority scheduling unit to start the low priority discard strategy.

[0075] Priority threshold setting: The priority scheduling unit dynamically sets a priority threshold based on the current system status.

[0076] Request screening: The newly arrived request is first calculated by the priority calculation subunit and the dynamic adjustment subunit to calculate its dynamic priority.

[0077] Discard execution: The scheduling execution subunit compares the dynamic priority of the request with the current priority threshold. Requests below the priority threshold will be directly discarded and will not enter the request queue.

[0078] Priority Guarantee: This mechanism ensures that high-priority requests (such as core transaction operations) are processed promptly even when the system is under high load.

[0079] Long-running request processing: For requests with a long expected execution time (such as large-scale historical data queries), the system transfers them to a dedicated long-running queue for processing, which is managed by an independent coroutine pool to avoid affecting the processing of other requests in the main request queue.

[0080] See also Figure 7 , timeout differentiation strategy unit:

[0081] Interface classification: Based on historical data and expected execution time, interfaces are divided into fast interfaces, normal interfaces, and long-execution interfaces.

[0082] Dynamic timeout setting: Through the threshold prediction calculation unit, different initial timeout periods are set for different types of interfaces and dynamically adjusted according to real-time monitoring data.

[0083] Timeout warning mechanism: When the interface execution time is close to the timeout threshold, an alert is triggered and consideration is given to extending the timeout period or transferring the request to a long execution queue.

[0084] The present invention provides an embodiment, which describes the application of a coroutine-based dynamic scheduling microservice gateway method in the securities industry. The securities platform includes multiple types of requests, such as stock trading, market query, account management, etc. Figure 8 , the following is the detailed operation process of the dynamic scheduling microservice gateway method based on coroutines of the present invention combined with this embodiment:

[0085] Step 1: Create a coroutine pool to process network requests and debug parameters. The specific process is as follows:

[0086] When the gateway system starts, the following components are initialized:

[0087] Coroutine pool unit: creates 50 threads, each of which can create up to 1,000 coroutines for concurrent processing of requests.

[0088] Request Queue Unit: Set the maximum capacity to 100,000.

[0089] Priority scheduling unit: configures the static priority weight and dynamic factor weight of the interface.

[0090] Business processing unit: obtains service reporting information from the registration center and obtains relevant information such as service routing.

[0091] Monitoring unit: Set the data collection interval to 30 seconds and the priority recalculation interval to 60 seconds.

[0092] Threshold prediction calculation unit: Initializes the threshold prediction calculation unit and obtains historical interface monitoring data to calculate the timeout threshold.

[0093] Low priority drop policy unit: Set the queue full load threshold to 90% and define the calculation rule for the dynamic priority threshold.

[0094] Timeout differentiation strategy unit: The predicted initial timeout time of different interfaces is calculated based on the threshold prediction calculation unit, and dynamically adjusted through the threshold prediction calculation unit based on monitoring data.

[0095] The system pre-sets static priorities for different types of requests:

[0096] Stock trading request: 10 points (highest priority)

[0097] Real-time market query: 10 minutes

[0098] Account fund operation: 10 points

[0099] Historical data query: 9 points

[0100] Customer service: 7 points

[0101] News and information enquiry: 5 points (lowest priority)

[0102] Step 2: The gateway accepts the network request and adds it to the request queue, and assigns an idle coroutine to handle and forward the network request. The specific process is as follows:

[0103] When a new request arrives at the gateway, the system processes it as follows:

[0104] Request received: The gateway receives a stock trading request and the request is immediately added to the request queue.

[0105] Step 3: Identify the request type and assign a preset static priority to the network request. Based on the static priority and the configured preliminary dynamic priority, the dynamic priority is adjusted in combination with the request frequency, response time, and system load. The specific process is as follows:

[0106] Coroutine pool unit:

[0107] An idle coroutine in the coroutine pool is assigned to handle forwarding the stock trading request.

[0108] This coroutine handles network I / O and business logic simultaneously without thread switching.

[0109] Business Processing:

[0110] After decrypting and decompressing the received request, the coroutine forwards the stock trading request to the corresponding trading microservice to execute the stock trading logic, including verifying orders, checking account balances, updating positions, generating transaction records, etc.

[0111] The dynamic priority calculation process of comprehensive features and ridge regression is as follows:

[0112] Input: Static priority , Maximum static priority ,

[0113] Request frequency , maximum request frequency ,

[0114] Maximum frequency factor , response time ,

[0115] Maximum response time , maximum response time factor ,

[0116] Current system load , maximum system load ,

[0117] Ridge regression model parameters , regularization parameter lambda, average response time:

[0118] Output: Dynamic Priority

[0119] 1: Calculate the contribution of a single feature

[0120]

[0121] 2: Combination of features

[0122]

[0123] 3: Calculate dynamic priority fitting coefficients using ridge regression model

[0124]

[0125] Static priority coefficient

[0126] Request frequency factor

[0127] Response time factor

[0128] System load factor

[0129] intercept

[0130] 4: Return dynamic priority

[0131]

[0132] The dynamic priority calculation of comprehensive features and ridge regression is implemented by taking the following example as an embodiment, and the steps of the embodiment are as follows:

[0133] Static priority calculation: The priority calculation subunit identifies the request type as "stock trading request". According to the preset, the static priority of the stock trading request is 10 points (assuming the maximum weight is 10 points).

[0134] Dynamic priority adjustment: The monitoring unit provides current system status data:

[0135] Request frequency: 5000 times / second (assuming the maximum frequency is 10000 times / second)

[0136] Average response time: 10ms (assuming maximum response time is 100ms)

[0137] Current queue length: 50000

[0138] System load: 80%

[0139] The dynamic adjustment subunit uses this data, combined with the static priority, to calculate the dynamic priority:

[0140] dynamic_priority = ((10 / 10) * 0.5 + ((10000 / 5000) / 10000) * 0.2 + ((100 / 10) 100) * 0.2 + ((1-0.8) / 1) * 0.1) = 0.54

[0141] Among them, the dynamic priority calculation formula based on ridge regression is: ,in, Indicates dynamic priority. represents the intercept term, represents the regression coefficient of the static priority feature, represents the regression coefficient of the request frequency feature, represents the regression coefficient of response time, Regression coefficient representing the system load characteristics, Indicates static priority, Indicates the request frequency, Indicates the response time, Indicates the system load, represents the error term.

[0142] The regression coefficients are estimated using the Ridge Regression method to minimize the following objective function: .Where: y is the dependent variable vector (dynamic priority), X is the independent variable matrix (feature matrix), is the regression coefficient vector, is the regularization parameter. By comprehensively considering the adaptive weight allocation of multiple factors, the regularization term introduced It effectively prevents overfitting problems and efficient calculation, and realizes accurate, stable and flexible priority calculation.

[0143] Step 4: Collect the request frequency, response time, queue length, system load, and interface execution data of the coroutine in real time, and calculate the timeout threshold according to the data operation status, and adopt the timeout differentiation strategy, low priority discard strategy, and data status update to reduce the coroutine processing delay. The specific process is as follows:

[0144] Business Processing:

[0145] For requests that are expected to take a long time to execute (such as large-scale historical data queries), the system will transfer them to a dedicated long-running queue for processing.

[0146] Timeout Differentiation Strategy:

[0147] The threshold prediction calculation unit calculates the timeout thresholds of different interfaces based on the historical monitoring data provided by the monitoring unit:

[0148] Stock trading request: 500ms

[0149] Real-time market query: 200ms

[0150] Account fund operation: 1000ms

[0151] Historical data query: 10000ms.

[0152] Timeout Differentiation Strategy:

[0153] If the request execution time is close to the timeout threshold, the system will trigger the timeout warning mechanism.

[0154] Response returned:

[0155] After processing is completed, the coroutine directly sends the transaction results back to the client.

[0156] The coroutine then returns to the idle state, ready to handle the next request.

[0157] Low priority drop policy:

[0158] Suppose that at the market opening time, the system suddenly encounters a large number of user requests:

[0159] The monitoring unit detects that the queue length reaches a certain proportion of the set value. In one or more embodiments, the certain proportion of the set value is 90% of the maximum capacity. In a preferred embodiment, when the maximum setting capacity of the request queue is 100,000, the priority threshold mechanism is triggered when the queue length reaches 90,000.

[0160] Triggering the priority threshold mechanism:

[0161] The system sets the priority threshold to 0.5.

[0162] New incoming requests with a dynamic priority lower than 0.5 will be discarded directly.

[0163] Request Scheduling:

[0164] High priority requests (e.g., stock trades, options trades) continue to be accepted and processed.

[0165] Low-priority requests (such as historical data queries) are temporarily discarded to ensure the normal operation of core trading business.

[0166] Long-running request processing:

[0167] For long-running requests such as historical data queries, the system places them in a dedicated long-running queue.

[0168] These requests are handled using a separate coroutine pool to avoid occupying the main request processing resources.

[0169] As trading volume changes, the system continues to adapt:

[0170] The monitoring unit updates the system status data every 30 seconds.

[0171] Every 60 seconds, the gateway system recalculates the dynamic priorities of all request types.

[0172] The priority scheduling unit adjusts the request processing order according to the latest dynamic priority.

[0173] Dynamic adjustment of overtime differentiation strategy:

[0174] The system dynamically adjusts the timeout thresholds for different types of requests based on real-time monitoring data.

[0175] Threshold prediction calculation unit:

[0176] Obtain historical monitoring data of the interface through the monitoring unit, including the number of interface calls, average duration, number of timeouts, and number of errors per minute;

[0177] Based on the cleaned data set W above, Daubechies wavelet analysis is used to split different types of parameters. The specific process is as follows:

[0178] Input: Input backtracking time series ; Input length T; Prediction length S; Number of variables N; Token dimension D; Number of iTransformer blocks L; Wavelet decomposition level J.

[0179] 1) For variables , perform the following steps:

[0180] 2) Extract the time series of the nth variable;

[0181] 3) = Wavelet_Daubechies(X_n Level = J);

[0182] 4) X[: n] = concat([ ]) Connect the coefficients after decomposition;

[0183] 5) X = X.Transpose.

[0184] Furthermore, the working principle of the iTransformer algorithm is as follows:

[0185] Using the data of different frequencies obtained in the above steps, set it as , where T is the current step size and N is the number of frequencies. The core goal of this algorithm is to predict the value for S time steps:

[0186] For each Xn, it needs to be converted into , then input into the TrmBlock part to get ,in , contains N-dimensional embeddings with D dimensions, and the superscripts represent the layer index. Both the embedding and projection parts are implemented by multi-layer perceptrons.

[0187] See also Fig. 9 , TrmBlock (short for Transformer Block) is the core of the model. Each TrmBlock contains the following three key components:

[0188] 1) Layer Normalization: Layer normalization helps stabilize model training. By normalizing the representation of each series, the difference between different time steps can be reduced. The specific formula is as follows, where H is the hidden layer representation of the input, and Mean and Var are the mean and variance, respectively.

[0189]

[0190] 2) Self-Attention Mechanism: The self-attention mechanism allows the model to take into account the information of the entire sequence when processing each time step, thereby capturing long-term dependencies. The specific formula is as follows:

[0191] in, is the output of the previous layer, Q represents the query, K represents the key, and V represents the value, where A is the attention score matrix, is the dimension of the key.

[0192] ,O=AV

[0193] 3) Feed-Forward Network (FFN): A feed-forward network is a simple linear transformation followed by an activation function, which is used to learn nonlinear representations of time series data. Output of the previous layer.

[0194]

[0195] Based on the three key components of TrmBlock, the specific steps of the iTransformer algorithm are as follows:

[0196] 6. Use a multilayer perceptron (MLP) to embed the sequence into the variable Tokens in the last dimension

[0197] 7. Initialization ,in

[0198] 8. For l ∈ {1, ..., L}, perform the following loop steps:

[0199] 9. Apply the first layer normalization and self-attention mechanism update, where

[0200] 10. Use the first feed-forward network for sequence representation and broadcast to each token.

[0201] 11. Apply a second layer normalization and a second feed-forward network update, where

[0202] 12. A third layer of normalization is used in sequence representation to reduce variable differences.

[0203] 13. End the loop

[0204] 14. Use a multi-layer perceptron to process the final output

[0205] 15. Will Transpose

[0206] 16. Return prediction results

[0207] A different timeout period is set for each interface according to the prediction result of the threshold prediction calculation unit.

[0208] For interfaces that frequently time out, the system automatically extends their timeout period and issues a warning.

[0209] See also Fig.10The present invention provides a dynamic scheduling microservice gateway system based on coroutines, including the following contents:

[0210] Network request initial processing module: Create a coroutine pool to process network requests. The gateway accepts network requests and adds them to the request queue, and allocates an idle coroutine to process and forward the network request.

[0211] Priority dynamic processing module: Identifies the request type and assigns a preset static priority to the network request. It also adjusts the dynamic priority based on the static priority and the configured preliminary dynamic priority, combined with the request frequency, response time, and system load.

[0212] Processing measures module: collects request frequency, response time, queue length, system load and interface execution data in coroutine operation in real time, and calculates timeout thresholds based on data operation status, and adopts timeout differentiation strategy, low priority discard strategy and data status update to reduce coroutine processing delay.

[0213] With this implementation, the system exhibits the following performance advantages:

[0214] Ultra-high concurrent processing: The coroutine model allows the system to process tens of thousands of transaction requests simultaneously, significantly improving transaction throughput.

[0215] Ultra-low latency: By merging network I / O and business logic processing, the request processing latency is reduced to milliseconds.

[0216] Transaction priority: Dynamic priority scheduling ensures the normal operation of core transaction business under high load conditions and effectively prevents system paralysis.

[0217] Real-time market information: While ensuring that transaction requests are given priority, a large number of real-time market information query requests can also be processed in a timely manner.

[0218] Adaptive capability: The system can automatically adjust according to market fluctuations and changes in trading volume, improving overall stability and efficiency.

[0219] Flexible processing of long-running requests: Through dedicated long-running execution queues and timeout differentiation strategies, the system can effectively handle long-running requests such as large-scale historical data queries without affecting the processing of other high-frequency, low-running requests.

[0220] Intelligent timeout management: The timeout differentiation strategy ensures that different types of requests can get a reasonable execution time, improving the overall efficiency of the system and user experience.

[0221] This embodiment describes in detail the operation process of the system in the securities trading platform, from initialization to request processing, to the processing of high-load situations and the system's adaptive capabilities. It shows how the present invention can significantly improve system performance, optimize resource utilization, and ensure the stable operation of core trading services in practical applications in the securities industry, while effectively processing a large number of auxiliary requests such as market inquiries. This solution is particularly suitable for handling high concurrency, low latency, and high reliability requirements in the securities market.

[0222] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0223] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A method for dynamically scheduling microservice gateway based on coroutines, characterized in that: The steps include: Step 1: Create a coroutine pool to process network requests. The gateway accepts network requests and adds them to the request queue, and assigns an idle coroutine to process and forward the network request. Step 2: Identify the request type and assign a preset static priority to the network request, and adjust the dynamic priority based on the static priority and the configured preliminary dynamic priority, combined with the characteristics of request frequency, response time and system load; Step 3: Collect the request frequency, response time, queue length, system load, and interface execution data of the coroutine in real time, and calculate the timeout threshold according to the data operation status, and adopt timeout differentiation strategy, low priority discard strategy, and data status update to reduce the coroutine processing delay.

2. The method for dynamically scheduling microservice gateways based on coroutines according to claim 1 is characterized in that: In the step 2, the dynamic priority is adjusted based on the comprehensive features and the ridge regression model to calculate the dynamic priority. The specific process is as follows: Configure the static priority of the interface, request frequency, response time, and dynamic priority weights of each feature of the system load; Calculate the contribution values ​​of individual features of static priority, request frequency, response time, and system load respectively; Calculate the combined feature value based on the individual feature contribution values ​​of static priority, request frequency, response time and system load; Based on the coefficients and combined eigenvalues ​​of static priority, request frequency, response time and system load, the ridge regression model is used to calculate the average response time and use it as the dynamic priority fitting coefficient. Outputs the dynamic priority of network requests based on the respective coefficients of static priority, request frequency, response time and system load and the configured respective dynamic priority weights.

3. The method for dynamically scheduling microservice gateways based on coroutines according to claim 2 is characterized in that: In step 2, the dynamic priority calculation formula based on ridge regression is as follows: ,in, Indicates dynamic priority, represents the intercept term, Indicates static priority, Indicates the request frequency, Indicates the response time, Indicates the system load, represents the regression coefficient of the static priority feature, represents the regression coefficient of the request frequency feature, represents the regression coefficient of the response time characteristic, Regression coefficient representing the system load characteristics, represents the error term.

4. The method for dynamically scheduling microservice gateway based on coroutine according to claim 3 is characterized in that: In step 3, the specific process of calculating the timeout threshold is as follows: According to the historical monitoring data of the interface, the timeout time of each interface is calculated based on the wavelet decomposition of Daubechies; Based on the multivariate data obtained after Daubechies wavelet decomposition, the iTransformer model is reversely reconstructed to obtain the final timeout threshold.

5. The method for dynamically scheduling microservice gateway based on coroutine according to claim 4 is characterized in that: The specific process of the Daubechies wavelet decomposition introduced by calculating the timeout threshold in step 3 is as follows: Obtain interface historical monitoring data and use Daubechies wavelet decomposition to split various types of historical monitoring data; Fix the number of layers of Daubechies wavelet decomposition to the preset value; Perform discrete wavelet transform on each parameter sequence to obtain approximate coefficients and detail coefficients of various scales, and set the step size of the approximate coefficients and detail coefficients to T; Analyze the approximate coefficients and detail coefficients of each scale, identify the characteristics of each type of parameters, and set the number of parameter features to N.

6. The method for dynamically scheduling microservice gateway based on coroutine according to claim 5 is characterized in that: The Daubechies wavelet decomposition algorithm introduced in the calculation of the timeout threshold in step 3 is as follows: Input: Input backtracking time series ; Input length T; Prediction length S; Number of variables N; Token dimension D; Number of iTransformer blocks L; Wavelet decomposition level J; Then for the variable , perform the following steps: 1) , extract the time series of the nth variable; 2) = Wavelet_Daubechies(X_n Level=J); 3) X[: n] = concat([ ]) Connect the coefficients after decomposition; 4) X = X.Transpose.

7. The method for dynamically scheduling microservice gateway based on coroutine according to claim 6 is characterized in that: The specific process of the iTransformer algorithm used in calculating the timeout threshold in step 3 is as follows: The multivariate data after Daubechies wavelet decomposition is converted through the Embedding model and input into TrmBlock to obtain the sequence value; Use a multi-layer perceptron to embed sequence values ​​into the variable Tokens in the last dimension; The following operations are performed for the variable Tokens loop: the first layer normalization and self-attention mechanism are used to update the output value of the previous layer to obtain the first update value; the first feedforward network sequence is used to represent the first update value and broadcast it to each variable token; the second layer normalization and the second feedforward network are used to update each variable token to obtain the second update value; the third layer normalization is used on the sequence table to reduce the variable difference; Use a multilayer perceptron to process and output the final variable, transpose the final variable and use the transposed result as the prediction result.

8. The method for dynamically scheduling microservice gateways based on coroutines according to claim 7 is characterized in that: In step 3, the specific process of the timeout differentiation strategy is as follows: Based on the interface historical monitoring data and expected execution time, interfaces are classified into fast interfaces, normal interfaces, and long-execution interfaces; By calculating the timeout threshold, the corresponding initial timeout period is set for various types of interfaces, and dynamically adjusted according to real-time monitoring data; When the execution time of each interface approaches the timeout threshold, an alert is triggered and consideration is given to extending the timeout period or transferring the request to a long execution queue.

9. The method for dynamically scheduling microservice gateways based on coroutines according to claim 8, characterized in that: In step 3, the specific process of the low priority discard strategy is as follows: When it is detected that the queue length is close to a certain percentage of the set value, the low priority discard strategy is activated; Dynamically set a priority threshold based on the current system status; Calculate the dynamic priority of newly arrived requests; Compare the dynamic priority of the request with the current priority threshold. Requests below the priority threshold will be discarded directly. For requests that are expected to take a long time to execute, the request is transferred to a dedicated long-running queue and processed by an independent coroutine pool.

10. A coroutine-based dynamic scheduling microservice gateway system, having a computer program, characterized in that: The computer program can execute the coroutine-based dynamic scheduling microservice gateway method as described in any one of claims 1 to 9.

Images