Financial transaction system resource high availability method and related device

Abstract

The invention relates to a financial transaction system resource high availability method and a related device. The method comprises the following steps: assigning a special request header to a to-be-sent request, and sending the to-be-sent request to a server; the method comprises the following steps of: isolating a processing request resource pool and dividing the resource pool into a high priority resource pool and a low priority resource pool during the operation of a server, and after the server receives a client request, acquiring a special request header, recording initial information and a service module entering the server, and marking the arrival time; dividing to different resource pools for processing according to requests, and preferentially preempting resources of a low-priority resource pool for service; and after the request is completely executed, a request processing result is fed back to the calculation module, and the calculation module performs backtracking calculation on the data so as to support subsequent resource allocation. The problem that a financial transaction system cannot ensure high availability of a single service node in a low-cost and friendly mode under the condition of limited resources is solved, the system is high in real-time performance and low in cost, high-availability design is achieved, and meanwhile service is kept stable and available.

Description

Technical Field

[0001] The present invention relates to an Internet service technology, and in particular to a method for high availability of financial transaction system resources and a related device. Background Art

[0002] With the increasing popularity and development of mobile Internet, the number of Internet participants is increasing, and the number of requests transmitted on the network is also rapidly expanding. This is a challenge to the performance and stability of Internet service providers. For core systems such as financial transaction systems that involve the flow of funds, high availability should be achieved under large traffic.

[0003] Theoretically, there is an upper limit to the server machine's ability to process requests. For a server machine, if the number of visits received exceeds the capacity limit within a unit time, the requests will not be processed and responded to in a timely manner, resulting in a backlog of requests, causing the machine to be overloaded until it crashes. A single point of failure will cause a chain reaction, further dragging down the entire system and causing the global service to be paralyzed.

[0004] Currently, the relevant technologies to solve this problem are:

[0005] Solution 1: Horizontally expand bottleneck services by adding machines and service nodes. During the duration of the traffic peak, it is often difficult to prepare enough physical machines and deploy the corresponding service nodes in a short period of time. The handling of large traffic bursts is poor and the real-time performance is not good enough.

[0006] Solution 2: Server-side current limiting and fuse setting, setting a threshold for the number of requests processed per unit time, and discarding all requests that exceed the upper threshold. Discarding requests will lead to a poor user experience, and the threshold is set as a static parameter, which makes it difficult to calculate the boundary threshold based on the actual operating status of the service node, maximize the performance of the service node, or set the threshold too large, resulting in fuse failure and service downtime.

[0007] shortcoming:

[0008] 1. Plan 1 is costly and time-sensitive.

[0009] 2. Solution 2 has inaccurate threshold calculation and poor user experience, resulting in low efficiency and insecurity. Summary of the invention

[0010] Aiming at the problem that the financial transaction system cannot ensure the high availability of a single service node in a low-cost and friendly manner under limited resource conditions, a method and related devices for high availability of financial transaction system resources are proposed. The solution ideas are as follows:

[0011] 1. The present invention enhances node processing performance by adjusting the request processing method of a single service node, and solves the problems of high cost and poor timeliness brought by solution 1.

[0012] 2. The present invention uses an algorithm to dynamically calculate and set a threshold to solve the problem of inaccurate threshold calculation and poor user experience in solution 2.

[0013] 3. The present invention is somewhat different from previous high availability implementation ideas. Previous high availability ideas used multiple machines, multiple nodes, and multiple clusters to achieve high availability in deployment methods. The present invention empowers a single node and improves the service capability of a single node in a large traffic environment to achieve high availability under conditions of limited resources.

[0014] The technical solution of the present invention is:

[0015] A method for high availability of financial trading system resources is implemented based on a basic flow calculation formula and through the following request processing method:

[0016] Step 1: When a request is sent through the client, it carries a preset special communication request header, which contains the business module, caller IP, and call weight;

[0017] Step 2: After assigning a special request header to the request to be sent, send it to the server;

[0018] Step 3: During the operation of the server, the resource pool for processing requests is isolated and divided into high-priority and low-priority resource pools to achieve complete isolation at the resource level. Different resource pools contain different numbers of resources, and high-priority resource pools accommodate more processing resources.

[0019] Step 4: After receiving the client request, the server obtains the special request header, records the initial information and business module entering the server, and marks the arrival time;

[0020] Step 5: According to the basic information of the request, the time consumption record of past calls, and the proportion of abnormal situations, the request is divided into different resource pools for processing. The specific operations are as follows:

[0021] Step 5.1: Set impact factors ɑ1, ɑ2, ɑ3…ɑ x … n ;

[0022] Step 5.2: Set abnormal call thresholds β1, β2, β3…β x …β n ;

[0023] Step 5.3: Set the total time consumed in each acquisition cycle θ1, θ2, θ3…θ x …θ n ;

[0024] Step 5.4: Set the total number of calls in each acquisition cycle φ1, φ2, φ3…φ x …φ n;

[0025] Step 5.5:

[0026] Step 5.6:

[0027] Step 5.7: When the previous call duration is greater than the duration threshold, the request is judged as low priority;

[0028] Step 5.8: When the past abnormal calls > the abnormal call threshold, the request is judged as low priority;

[0029] Step 6: Preemptive occupation of resources is allowed. When a request is classified as a high priority, if the low priority resource pool is not busy, the resources of the low priority resource pool are preempted for service.

[0030] Step 7: After the request is fully executed, the result of the request processing is fed back to the calculation module, which performs back-calculation on the data to support subsequent resource allocation:

[0031] Step 7.1: Set the sampling period;

[0032] Step 7.2: Accumulate the number of requests processed during the cycle;

[0033] Step 7.3: Accumulate the number of abnormal requests within the period;

[0034] Step 7.4: Calculate the average request duration and queuing delay;

[0035] Step 7.5: After each sampling period, observe the relationship between the number of processed requests and the queuing delay;

[0036] Step 7.6: When the number of processed requests and the queuing delay keep increasing in the same trend, record the number of processed requests in this sampling period and set it as the service upper threshold;

[0037] Furthermore, a basic flow calculation formula is introduced in this method:

[0038] When the service is stable, concurrency = queuing delay * number of requests processed per second;

[0039] When the service is not overloaded, the concurrency and the number of requests processed per second are linearly correlated;

[0040] When the service is overloaded, the concurrency and queuing delay will increase together, and the number of requests processed per second will stabilize;

[0041] It can be seen that when the service processing is overloaded and on the verge of collapse, queuing delay and concurrency will increase together.

[0042] Furthermore, when a service request is made, adaptive scheduling is performed on the client. The client establishes a resource pool and selects one of the service nodes from multiple optional service nodes for calling. Such a solution relies on the server and the client to maintain communication. It requires redundant communication through heartbeat messages to keep the client aware of the server load situation and perform current limiting on the client.

[0043] A device related to high availability of financial transaction system resources, used to implement the method for high availability of financial transaction system resources as described above, specifically comprising: a client module, a client agent module, a collection module, a backtracking calculation module, a priority identification module, and an execution module, wherein the client and client agent modules belong to the client module, and the collection, backtracking, priority identification, and execution modules belong to the server module; the client and server modules communicate through the network, and the client and the server directly interact using memory data; the operating relationship between the modules of the system is as follows:

[0044] Step 1: When the client module is running, a call request is sent to the client proxy module;

[0045] Step 2: The client proxy module adds a special header to the called request for special processing. Special fields that frequently appear in the financial transaction system are compressed using a special mapping algorithm to reduce the data transmission size.

[0046] Step 3: The client proxy module specifies a special communication request header according to the request of the client module. The request header includes the business module, the caller IP, and the call weight.

[0047] Step 4: After the server receives the client request, the collection module obtains the special request header, records the initial information and business module entering the server, and marks the arrival time;

[0048] Step 5: The priority identification module divides the requests into different resource pools based on the basic information of the requests, the time consumption records of past calls, and the proportion of abnormal situations;

[0049] Step 6: The priority identification module passes the request to the execution module, which is responsible for the specific execution of the request and outputs a return value;

[0050] Step 7: The collection module generates collection data results according to whether an error is reported during the request execution and the total time consumed for the request execution, and passes the results to the calculation backtracking module;

[0051] Step 8: The calculation backtracking module calculates various performance indicators in real time according to step 7 of the high availability method for financial transaction system resources, based on the pre-set influencing factors within the gradient range and the collection results produced by the collection module, and the output is passed to the priority identification module;

[0052] Step 9: The priority identification module identifies subsequent requests based on output indicators and preset indicator thresholds.

[0053] Furthermore, the specific steps of dividing the requests into different resource pools in step 5 are as follows:

[0054] Step 51: determine whether the current processing request exceeds the service upper limit threshold. If it does not exceed the service upper limit threshold, the request is directly delivered to the high priority resource pool for execution. If it exceeds the service upper limit threshold, proceed to step 52;

[0055] Step 52: determine whether the calling time of the current specific request exceeds the time threshold generated by the backtracking calculation module. If so, deliver the request to a low-priority resource pool for execution. Otherwise, proceed to step 53.

[0056] Step 53: determine whether the call exception ratio of the current specific request exceeds the exception threshold. If so, deliver the request to a low-priority resource pool for execution. Otherwise, proceed to step 54.

[0057] Step 54, execution to this point indicates that under high pressure load, the request is still a specific request that needs to be resolved as soon as possible and is in good calling condition, and is delivered to a high priority resource pool for execution.

[0058] The beneficial effects of the present invention are:

[0059] Compared with Solution 1, the present invention has high real-time performance and low cost. Solution 1 relies on multi-node deployment to achieve high-availability design, while the present invention does not rely on redundant machines. Under the condition of limited resources, it provides services to the outside world to the greatest extent while maintaining stable and available services.

[0060] Compared with Solution 2, the present invention has strong performance and high availability. Solution 2 relies on manually setting a static service threshold for each service node. The present invention comprehensively determines the upper limit threshold based on past service data and local load to maximize performance and availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the high availability method for financial transaction system resources and related device architecture of the present invention;

[0062] Figure 2 This is a schematic diagram of dividing requests into different resource pools for processing in the present invention. DETAILED DESCRIPTION

[0063] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0064] The present invention provides a method for high availability of financial transaction system resources. Here, a basic flow calculation formula is introduced:

[0065] When the service is stable, concurrency = queuing delay * number of requests processed per second;

[0066] When the service is not overloaded, the concurrency and the number of requests processed per second are linearly correlated;

[0067] When the service is overloaded, the concurrency and queuing delay will increase together, and the number of requests processed per second will stabilize;

[0068] It can be seen that when the service processing is overloaded and on the verge of collapse, queuing delay and concurrency will increase together.

[0069] See also Figure 1 A method for high availability of financial transaction system resources provided in an embodiment of the present application is applied to high availability of financial transaction system resources composed of a client module, a client proxy module, a collection module, a backtracking calculation module, a priority identification module, and an execution module.

[0070] Based on this theory, the present invention proposes the following method for processing requests: after assigning a special request header to the request to be sent, it is sent to the server; during the operation of the server, the resource pool for processing requests is isolated, and the resource pool is divided into high-priority and low-priority resource pools. After the server receives the client request, it obtains the special request header, records the initial information and business module entering the server, and marks the arrival time; the request is divided into different resource pools for processing, and the resources of the low-priority resource pool are preempted for service; after the request is fully executed, the result of the request processing is fed back to the computing module, and the computing module performs back-calculation on the data to support subsequent resource allocation: improve the service capability of a single node in a high-traffic environment to achieve high availability under limited resource conditions. The details are as follows:

[0071] Step 1: When a request is sent through the client, it carries a preset special communication request header, which contains the business module, caller IP, and call weight;

[0072] Step 2: After assigning a special request header to the request to be sent, send it to the server;

[0073] Step 3: During the operation of the server, the resource pool for processing requests is isolated and divided into high-priority and low-priority resource pools to achieve complete isolation at the resource level. Different resource pools contain different numbers of resources, and high-priority resource pools accommodate more processing resources.

[0074] Step 4: After receiving the client request, the server obtains the special request header, records the initial information and business module entering the server, and marks the arrival time;

[0075] Step 5: According to the basic information of the request, the time consumption record of past calls, and the proportion of abnormal situations, the request is divided into different resource pools for processing. The specific operations are as follows:

[0076] Step 5.1: Set impact factors ɑ1, ɑ2, ɑ3…ɑ x … n ;

[0077] Step 5.2: Set abnormal call thresholds β1, β2, β3…β x …β n ;

[0078] Step 5.3: Set the total time consumed in each acquisition cycle θ1, θ2, θ3…θ x …θ n ;

[0079] Step 5.4: Set the total number of calls in each acquisition cycle φ1, φ2, φ3…φ x …φ n ;

[0080] Step 5.5:

[0081] Step 5.6:

[0082] Step 5.7: When the previous call duration is greater than the duration threshold, the request is judged as low priority;

[0083] Step 5.8: When the past abnormal calls > the abnormal call threshold, the request is judged as low priority;

[0084] Step 6: Preemptive occupation of resources is allowed. When a request is classified as a high priority, if the low priority resource pool is not busy, the resources of the low priority resource pool are preempted for service.

[0085] Step 7: After the request is fully executed, the result of the request processing is fed back to the calculation module, which performs back-calculation on the data to support subsequent resource allocation:

[0086] Step 7.1: Set the sampling period;

[0087] Step 7.2: Accumulate the number of requests processed during the cycle;

[0088] Step 7.3: Accumulate the number of abnormal requests within the period;

[0089] Step 7.4: Calculate the average request duration and queuing delay;

[0090] Step 7.5: After each sampling period, observe the relationship between the number of processed requests and the queuing delay;

[0091] Step 7.6: When the number of processed requests and the queuing delay keep increasing in the same trend, record the number of processed requests in this sampling period and set it as the service upper threshold;

[0092] The key point of the present invention is a complete set of algorithms and processing procedures for the server to process requests. This set of algorithms provides dynamic threshold protection and bottlenecks for the performance upper limit for the server.

[0093] Among them, when a service request is made, adaptive scheduling is performed on the client. The client establishes a resource pool and selects one of the service nodes from multiple optional service nodes for calling. Such a solution relies on the server and the client to maintain communication. It is necessary to maintain the client's awareness of the server's load situation through a variety of redundant communications such as heartbeat messages, and perform current limiting on the client.

[0094] like Figure 1 As shown, attached Figure 1 The following is a diagram of the components and data flow of the resource high availability system, from which we can see the processing method of the entire system. This system consists of a client module, a client proxy module, a collection module, a backtracking calculation module, a priority identification module, and an execution module. The following is the operational relationship between the modules:

[0095] Step 1: When the client module is running, a call request is sent to the client proxy module;

[0096] Step 2: The client proxy module adds a special header to the called request for special processing. Special fields such as StockCode (stock code) and ClientId (user ID) that frequently appear in financial trading systems are compressed using a special mapping algorithm to reduce the data transmission size.

[0097] Step 3: The client proxy module specifies a special communication request header according to the request of the client module. The request header includes the business module, the caller IP, and the call weight.

[0098] Step 4: After the server receives the client request, the collection module obtains the special request header, records the initial information and business module entering the server, and marks the arrival time;

[0099] Step 5: The priority identification module divides the request into different resource pools based on the basic information of the request, the time consumption record of past calls, and the proportion of abnormal situations. For specific identification steps, please refer to Figure 2 ;

[0100] Step 6: The priority identification module passes the request to the execution module, which is responsible for the specific execution of the request and outputs a return value;

[0101] Step 7: The collection module generates collection data results according to whether an error is reported during the request execution and the total time consumed for the request execution, and passes the results to the calculation backtracking module;

[0102] Step 8: The calculation backtracking module calculates various performance indicators in real time according to step 7 of the high availability method for financial transaction system resources, based on the pre-set impact factors within the gradient range of 10 requests, 100 requests, 200 requests, etc., combined with the collection results produced by the collection module, and transmits the output to the priority identification module;

[0103] Step 9: The priority identification module identifies subsequent requests based on output indicators and preset indicator thresholds.

[0104] like Figure 2 As shown, Figure 2 The present invention divides requests into different resource pools for processing, and the specific steps are as follows:

[0105] Step (1), determine whether the current processing request exceeds the service upper limit threshold. If it does not exceed the service upper limit threshold, directly deliver the request to the high priority resource pool for execution. If it exceeds the service upper limit threshold, proceed to step (2);

[0106] Step (2), determine whether the calling time of the current specific request exceeds the time threshold generated by the backtracking calculation module. If so, deliver the request to the low priority resource pool for execution; otherwise, proceed to step (3);

[0107] Step (3), determine whether the call exception ratio of the current specific request exceeds the exception threshold. If so, deliver the request to the low-priority resource pool for execution. Otherwise, proceed to step (4);

[0108] Step (4) is executed here, which means that under high pressure load, the request is still a specific request that needs to be resolved as soon as possible and is in good calling condition, and is delivered to the high priority resource pool for execution.

[0109] The above-mentioned embodiment only expresses one implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.

Claims

1. A method for high availability of financial transaction system resources, characterized in that: Based on the basic traffic calculation formula, it is implemented by the following method of processing requests: Step 1: When a request is sent through the client, it carries a preset special communication request header, which contains the business module, caller IP, and call weight; Step 2: After assigning a special request header to the request to be sent, send it to the server; Step 3: During the operation of the server, the resource pool for processing requests is isolated and divided into high-priority and low-priority resource pools to achieve complete isolation at the resource level. Different resource pools contain different numbers of resources, and high-priority resource pools accommodate more processing resources. Step 4: After receiving the client request, the server obtains the special request header, records the initial information and business module entering the server, and marks the arrival time; Step 5: According to the basic information of the request, the time consumption record of past calls, and the proportion of abnormal situations, the request is divided into different resource pools for processing. The specific operations are as follows: Step 5.1: Set impact factors ɑ1, ɑ2, ɑ3…ɑ x … n ; Step 5.2: Set abnormal call thresholds β1, β2, β3…β x …β n ; Step 5.3: Set the total time consumed in each acquisition cycle θ1, θ2, θ3…θ x …θ n ; Step 5.4: Set the total number of calls in each acquisition cycle φ1, φ2, φ3…φ x …φ n ; Step 5.5: Step 5.6: Step 5.7: When the previous call duration is greater than the duration threshold, the request is judged as low priority; Step 5.8: When the past abnormal calls > the abnormal call threshold, the request is judged as low priority; Step 6: Preemptive occupation of resources is allowed. When a request is classified as a high priority, if the low priority resource pool is not busy, the resources of the low priority resource pool are preempted for service. Step 7: After the request is fully executed, the result of the request processing is fed back to the calculation module, which performs back-calculation on the data to support subsequent resource allocation: Step 7.1: Set the sampling period; Step 7.2: Accumulate the number of requests processed during the cycle; Step 7.3: Accumulate the number of abnormal requests within the period; Step 7.4: Calculate the average request duration and queuing delay; Step 7.5: After each sampling period, observe the relationship between the number of processed requests and the queuing delay; Step 7.6: When the number of processed requests and the queuing delay keep increasing in the same trend, record the number of processed requests in this sampling period and set it as the service upper threshold.

2. The method for high availability of financial transaction system resources according to claim 1, characterized in that: This method introduces a basic flow calculation formula: When the service is stable, concurrency = queuing delay * number of requests processed per second; When the service is not overloaded, the concurrency and the number of requests processed per second are linearly correlated; When the service is overloaded, the concurrency and queuing delay will increase together, and the number of requests processed per second will stabilize; It can be seen that when the service processing is overloaded and on the verge of collapse, queuing delay and concurrency will increase together.

3. The method for high availability of financial transaction system resources according to claim 1, characterized in that: When a service request is made, adaptive scheduling is performed on the client. The client establishes a resource pool and selects one of the service nodes from multiple optional service nodes for calling. Such a solution relies on the server and the client to maintain communication. It requires redundant communication through heartbeat messages to keep the client aware of the server load situation and perform current limiting on the client.

4. A device related to high availability of financial transaction system resources, characterized in that: The method for realizing high availability of financial transaction system resources as claimed in any one of claims 1 to 3 specifically comprises: a client module, a client agent module, a collection module, a backtracking calculation module, a priority identification module, and an execution module, wherein the client and the client agent module belong to the client module, and the collection, backtracking, priority identification, and execution modules belong to the server module; the client and the server modules communicate with each other through a network, and the client and the server directly interact with each other using memory data; the operation relationship between the modules of the system is as follows: Step 1: When the client module is running, a call request is sent to the client proxy module; Step 2: The client proxy module adds a special header to the called request for special processing. Special fields that frequently appear in the financial transaction system are compressed using a special mapping algorithm to reduce the data transmission size. Step 3: The client proxy module specifies a special communication request header according to the request of the client module. The request header includes the business module, the caller IP, and the call weight. Step 4: After the server receives the client request, the collection module obtains the special request header, records the initial information and business module entering the server, and marks the arrival time; Step 5: The priority identification module divides the requests into different resource pools based on the basic information of the requests, the time consumption records of past calls, and the proportion of abnormal situations; Step 6: The priority identification module passes the request to the execution module, which is responsible for the specific execution of the request and outputs a return value; Step 7: The collection module generates collection data results according to whether an error is reported during the request execution and the total time consumed for the request execution, and passes the results to the calculation backtracking module; Step 8: The calculation backtracking module calculates various performance indicators in real time according to step 7 of the high availability method for financial transaction system resources, based on the pre-set influencing factors within the gradient range and the collection results produced by the collection module, and the output is passed to the priority identification module; Step 9: The priority identification module identifies subsequent requests based on output indicators and preset indicator thresholds.

5. The device for high availability of financial transaction system resources according to claim 4, characterized in that: The specific steps for dividing requests into different resource pools in step 5 are as follows: Step 51: determine whether the current processing request exceeds the service upper limit threshold. If it does not exceed the service upper limit threshold, the request is directly delivered to the high priority resource pool for execution. If it exceeds the service upper limit threshold, proceed to step 52; Step 52: determine whether the calling time of the current specific request exceeds the time threshold generated by the backtracking calculation module. If so, deliver the request to a low-priority resource pool for execution. Otherwise, proceed to step 53. Step 53: determine whether the call exception ratio of the current specific request exceeds the exception threshold. If so, deliver the request to a low-priority resource pool for execution. Otherwise, proceed to step 54. Step 54, execution to this point indicates that under high pressure load, the request is still a specific request that needs to be resolved as soon as possible and is in good calling condition, and is delivered to a high priority resource pool for execution.

Images