Method, system and device for realizing market information real-time data calling optimization processing among micro-services, processor and storage medium thereof

Abstract

The invention relates to a method for realizing market information real-time data calling optimization processing among micro-services, which comprises the step that a client performs market information real-time data calling optimization and comprises the following steps: the client initiates a market information data calling request to a server; checking whether the request has cache data of a historical market result locally at the client, obtaining a timestamp of cache result data, and sending the request to the server to wait for data return; updating the local cache data and the timestamp; or the data in the client cache is confirmed to be latest, and the local cache data is read. By adopting the method, the system, the device, the processor and the computer readable storage medium for realizing market information real-time data calling optimization processing among the micro-services, high-concurrency access of hot stocks is avoided by utilizing a method of comparing timestamps of a client and a server and merging requests, and by utilizing the characteristics of market information data calling, the market information real-time data calling optimization processing among the micro-services is realized. And separating processing and operation of user behaviors and market stock data.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to the field of microservices, and specifically refers to a method, system, device, processor and computer-readable storage medium thereof for optimizing the processing of real-time market data calls between microservices. Background Art

[0002] Microservices refer to an architectural style that splits a system into a group of small and autonomous services. Each microservice runs independently and collaborates with each other through lightweight communication mechanisms. More and more large-scale market systems on the market are moving towards microservices. How to efficiently call market data between microservices has become a problem that needs to be solved.

[0003] The securities market is volatile, investors pay attention to different stocks, and there is an imbalance in the market. In the event of a sudden market event, investors pay a sudden increase in attention to certain stock data. The market data call in the microservice-based securities market system often has problems with hot data and high concurrency.

[0004] The current solution is usually to store hot stocks locally on the server or in cache components such as redis and set them to never expire. However, the problem is that during the peak period of the securities market, investors flock in, resulting in a large increase in concurrent client calls, and the covered stock data is large, the market is unpredictable, and hot stocks are constantly changing, resulting in an increasing cache usage. Alternatively, the background monitors the cache usage, and when the cache data is about to expire, the cache is asynchronously updated to extend the expiration time. The disadvantage of this method is that it requires an additional background thread to update the data in the cache. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method, system, device, processor and computer-readable storage medium thereof for optimizing the processing of real-time market data calls between microservices, which are efficient, easy to operate and have a wide range of applications.

[0006] In order to achieve the above objectives, the method, system, device, processor and computer-readable storage medium thereof for optimizing the processing of real-time market data calls between microservices of the present invention are as follows:

[0007] The main feature of the method for optimizing the real-time market data call processing between microservices is that the method includes the steps of optimizing the real-time market data call processing by the client and optimizing the real-time market data call processing by the server.

[0008] The step of optimizing the real-time market data call of the client specifically includes the following steps:

[0009] (1-1) The client sends a request to the server for market data;

[0010] (1-2) Check whether there is cached data of historical market results on the client side, obtain the timestamp of the cached result data, if not, obtain the timestamp as 0, carry it in the request header, send the request to the server, and wait for the server to return data;

[0011] (1-3) If the data result set returned by the server is not empty, the server returns the latest data, updates the local cache data and timestamp; if the data result returned by the server is empty, the server returns without the latest data, confirms that the data in the client cache is the latest, and reads the local cache data;

[0012] The steps of optimizing the real-time market data call on the server include:

[0013] (2-1) The server receives real-time data call requests;

[0014] (2-2) Perform interception preprocessing and compare the timestamp carried in the request header with the timestamp of the local cache data;

[0015] (2-3) If the timestamp carried in the request header is not less than the timestamp of the locally cached data, it is confirmed that the client has the latest data, and the server does not continue to access the data and directly returns an empty data set; if the timestamp carried in the request header is less than the timestamp of the locally cached data, execute step (2-4);

[0016] (2-4) If the timestamp in the request header is less than the timestamp on the server, the server data is the latest and further calls are needed to access the data in the storage component;

[0017] (2-5) Merge the calls for the requests with the same key, call the data for the first request that arrives, and block other concurrent requests for the same key to wait for the result of the first call;

[0018] (2-6) Based on concurrent requests for querying the same key, initiate a data call for the first request that arrives, access the new data in the storage component, and determine whether the process obtains the data within the specified time. If the data is obtained before the timeout, execute step (2-7); if the timeout occurs, execute step (2-8);

[0019] (2-7) Update the latest data obtained to the local cache and return the data results to the client;

[0020] (2-8) Request access to the storage component again and return the data results to the client.

[0021] Preferably, the method further comprises the following steps:

[0022] Monitor and delete the local cache market data of the client microservice and the server microservice based on the preset expiration time and its cache capacity.

[0023] Preferably, the method further comprises the following steps:

[0024] According to the time window sliding mechanism, the market data in the storage component of the server-side microservice is eliminated.

[0025] Preferably, an interceptor is used in steps (1-2), (2-2), (2-3) or (2-4), and the interceptor includes a client interceptor and a server interceptor. The client interceptor is a function called by the gRPC client before calling the actual RPC; the server interceptor is a function called by the gRPC server before reaching the actual RPC method; the client's request obtains a local cache timestamp before being sent out, and when the server receives the request, it preferentially compares the timestamp in the request with the local timestamp of the server.

[0026] Preferably, the steps are specifically as follows:

[0027] By preprocessing the request sent by the client and intercepting it on the server, the timestamp is obtained for comparison. For requests whose client timestamp is not less than the server timestamp, it is returned directly, and the client's local cache data is used as the result; for requests whose client timestamp is less than the server timestamp, the server processes the data call and merges the requests for access with the same key.

[0028] The main feature of the system for optimizing the processing of real-time market data calls between microservices is that the system includes a client, and the client includes:

[0029] Send a request preprocessing module, which is used to preprocess the request and check whether there is a result set in the local cache. If there is a result set, obtain its timestamp; if there is no result set, assign the timestamp to 0 and send a data call request to the server with the timestamp;

[0030] The request return processing module is connected to the sending request preprocessing module and is used to receive the data result set returned by the server and perform different return processing according to whether the data set is empty; if empty data is returned, it is confirmed that the result set in the client's local cache is the latest data; if data is returned, the client's local cache is updated.

[0031] Preferably, the system further comprises a server, and the server comprises:

[0032] The request interception preprocessing module is used to receive data and perform interception preprocessing before the data is actually called. It compares the timestamp carried in the request header with the timestamp of the local cache on the server. If the timestamp carried in the request header is not less than the timestamp of the local cache on the server, it is confirmed that the cached data of the client is the latest, and the server does not need to continue to call and directly returns empty data; if the timestamp carried in the request header is less than the timestamp of the local cache on the server, the request continues to be called;

[0033] A merge call processing module is connected to the request interception preprocessing module and is used to merge the requests for the same key query in the continued call requests, wherein the first request to arrive makes a data call, while other concurrent requests for the same key query are blocked and wait for the result of the first call;

[0034] A call priority processing module, connected to the merge call processing module, is used to initiate a data call for the first request that arrives based on concurrent requests for querying the same key, and process it;

[0035] A data access module, connected to the call priority processing module, for accessing data in the storage component;

[0036] The data return module is connected to the data access module and is used to return data results. If it is returned during the interception preprocessing, empty data is returned; if it is returned after the interception preprocessing, real data results are returned.

[0037] Preferably, the client further comprises a cache elimination module of the client, which is connected to the request return processing module and is used to eliminate cache data.

[0038] Preferably, the server also includes a server cache elimination module, which is connected to the data return module and is used to eliminate cache data.

[0039] The main feature of the device for optimizing the real-time market data call processing between microservices is that the device comprises:

[0040] a processor configured to execute computer-executable instructions;

[0041] The memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned method for optimizing the processing of real-time market data calls between microservices are implemented.

[0042] The processor for realizing optimization processing of real-time market data calls between microservices has the main feature that the processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for realizing optimization processing of real-time market data calls between microservices are realized.

[0043] The main feature of the computer-readable storage medium is that a computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned method for optimizing the processing of real-time market data calls between microservices.

[0044] The method, system, device, processor and computer-readable storage medium thereof for optimizing the processing of real-time market data calls between microservices of the present invention are adopted to solve the problem of hot data and high concurrency in market data calls in market microservice calls, and optimize the current method of storing hot stocks locally on the server or in cache components such as redis, or using cache monitoring to extend the cache time of hot data, and updating cache data through another thread. The present invention utilizes the timestamp comparison between the client and the server, and the method of merging requests to avoid high concurrent access to hot stocks, and utilizes the characteristics of market data calls to separate user behavior from the processing and operation of market stock data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a flow chart of optimizing the real-time market data call of a client according to the method for optimizing the real-time market data call between microservices.

[0046] Figure 2 The present invention is a flow chart of optimizing the real-time market data call processing by the service end of the method for optimizing the real-time market data call processing between microservices.

[0047] Figure 3 This is a schematic diagram of the structure of the client of the system for optimizing the processing of real-time market data calls between microservices of the present invention.

[0048] Figure 4 This is a structural diagram of the server end of the system for optimizing the processing of real-time market data calls between microservices according to the present invention. DETAILED DESCRIPTION

[0049] In order to more clearly describe the technical content of the present invention, further description is given below in conjunction with specific embodiments.

[0050] The method for optimizing the real-time market data call processing between microservices of the present invention includes the steps of optimizing the real-time market data call processing by the client and optimizing the real-time market data call processing by the server.

[0051] The step of optimizing the real-time market data call of the client specifically includes the following steps:

[0052] (1-1) The client sends a request to the server for market data;

[0053] (1-2) Check whether there is cached data of historical market results on the client side, obtain the timestamp of the cached result data, if not, obtain the timestamp as 0, carry it in the request header, send the request to the server, and wait for the server to return data;

[0054] (1-3) If the data result set returned by the server is not empty, the server returns the latest data, updates the local cache data and timestamp; if the data result returned by the server is empty, the server returns without the latest data, confirms that the data in the client cache is the latest, and reads the local cache data;

[0055] The steps of optimizing the real-time market data call on the server include:

[0056] (2-1) The server receives real-time data call requests;

[0057] (2-2) Perform interception preprocessing and compare the timestamp carried in the request header with the timestamp of the local cache data;

[0058] (2-3) If the timestamp carried in the request header is not less than the timestamp of the locally cached data, it is confirmed that the client has the latest data, and the server does not continue to access the data and directly returns an empty data set; if the timestamp carried in the request header is less than the timestamp of the locally cached data, execute step (2-4);

[0059] (2-4) If the timestamp in the request header is less than the timestamp on the server, the server data is the latest and further calls are needed to access the data in the storage component;

[0060] (2-5) Merge the calls for the requests with the same key, call the data for the first request that arrives, and block other concurrent requests for the same key to wait for the result of the first call;

[0061] (2-6) Based on concurrent requests for querying the same key, initiate a data call for the first request that arrives, access the new data in the storage component, and determine whether the process obtains the data within the specified time. If the data is obtained before the timeout, execute step (2-7); if the timeout occurs, execute step (2-8);

[0062] (2-7) Update the latest data obtained to the local cache and return the data results to the client;

[0063] (2-8) Request access to the storage component again and return the data results to the client.

[0064] As a preferred embodiment of the present invention, the method further comprises the following steps:

[0065] Monitor and delete the local cache market data of the client microservice and the server microservice based on the preset expiration time and its cache capacity.

[0066] As a preferred embodiment of the present invention, the method further comprises the following steps:

[0067] According to the time window sliding mechanism, the market data in the storage component of the server-side microservice is eliminated.

[0068] As a preferred implementation of the present invention, an interceptor is used in steps (1-2), (2-2), (2-3) or (2-4), and the interceptor includes a client interceptor and a server interceptor. The client interceptor is a function called by the gRPC client before calling the actual RPC; the server interceptor is a function called by the gRPC server before reaching the actual RPC method; the client's request obtains a local cache timestamp before being sent out, and when the server receives the request, it preferentially compares the timestamp in the request with the local timestamp of the server.

[0069] As a preferred embodiment of the present invention, the steps are specifically as follows:

[0070] By preprocessing the request sent by the client and intercepting it on the server, the timestamp is obtained for comparison. For requests whose client timestamp is not less than the server timestamp, it is returned directly, and the client's local cache data is used as the result; for requests whose client timestamp is less than the server timestamp, the server processes the data call and merges the requests for access with the same key.

[0071] The method for optimizing the processing of real-time market data calls between microservices of the present invention includes a client, and the client includes:

[0072] Send a request preprocessing module, which is used to preprocess the request and check whether there is a result set in the local cache. If there is a result set, obtain its timestamp; if there is no result set, assign the timestamp to 0 and send a data call request to the server with the timestamp;

[0073] The request return processing module is connected to the sending request preprocessing module and is used to receive the data result set returned by the server and perform different return processing according to whether the data set is empty; if empty data is returned, it is confirmed that the result set in the client's local cache is the latest data; if data is returned, the client's local cache is updated.

[0074] As a preferred embodiment of the present invention, the system further includes a server, and the server includes:

[0075] The request interception preprocessing module is used to receive data and perform interception preprocessing before the data is actually called. It compares the timestamp carried in the request header with the timestamp of the local cache on the server. If the timestamp carried in the request header is not less than the timestamp of the local cache on the server, it is confirmed that the cached data of the client is the latest, and the server does not need to continue to call and directly returns empty data; if the timestamp carried in the request header is less than the timestamp of the local cache on the server, the request continues to be called;

[0076] A merge call processing module is connected to the request interception preprocessing module and is used to merge the requests for the same key query in the continued call requests, wherein the first request to arrive makes a data call, while other concurrent requests for the same key query are blocked and wait for the result of the first call;

[0077] A call priority processing module, connected to the merge call processing module, is used to initiate a data call for the first request that arrives based on concurrent requests for querying the same key, and process it;

[0078] A data access module, connected to the call priority processing module, for accessing data in the storage component;

[0079] The data return module is connected to the data access module and is used to return data results. If it is returned during the interception preprocessing, empty data is returned; if it is returned after the interception preprocessing, real data results are returned.

[0080] As a preferred embodiment of the present invention, the client further includes a cache elimination module of the client, which is connected to the request return processing module and is used to eliminate cache data.

[0081] As a preferred embodiment of the present invention, the server also includes a cache elimination module of the server, which is connected to the data return module and is used to eliminate cache data.

[0082] The device for optimizing the real-time market data call processing between microservices of the present invention comprises:

[0083] a processor configured to execute computer-executable instructions;

[0084] The memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned method for optimizing the processing of real-time market data calls between microservices are implemented.

[0085] The processor of the present invention is used to implement the optimization processing of real-time market data calls between microservices, wherein the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for optimizing the processing of real-time market data calls between microservices are implemented.

[0086] The computer-readable storage medium of the present invention stores a computer program thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned method for optimizing the processing of real-time market data calls between microservices.

[0087] In a specific embodiment of the present invention, a method and related device for optimizing the real-time market data call between microservices are involved. The client microservice caches the market data locally. When the microservice is called, the timestamp of the local cached market data is first checked, and the timestamp is carried to request the market data. When the request reaches the server microservice, the request timestamp is intercepted and pre-processed to compare the request timestamp with the local cache timestamp of the server to determine whether the cached data of the client is the latest. When the request timestamp is greater than or equal to the local cache timestamp of the server, the cached market data of the client is the latest, and the request is returned directly; when the request timestamp is less than the local cache timestamp of the server, the cached market data of the client needs to be updated, and the request continues to call the market data of the server, and the market data with the latest timestamp is returned to the client. Thereby reducing the frequent data calls of the server and alleviating its pressure. In addition, the request to continue to call the server data separates the user behavior of the client from the market stock data operation of the server by merging the request, preventing the access of hot data and causing the breakdown of the storage component of the server.

[0088] Reference Figure 1 As shown, an embodiment of the present invention provides a client part in a method for optimizing real-time market data calls between microservices, and the method may include the following steps:

[0089] Step 101: The client initiates a data call request to the server;

[0090] Step 102: Based on the initiated data call request, first check whether there is cached data of historical market results on the client side, obtain the timestamp of the cached result data, if not, obtain the timestamp as 0, carry it in the request header, send the request to the server, and wait for the server to return data. If the data result set returned by the server is not empty, execute step 103; if the data result returned by the server is empty, execute step 104.

[0091] Step 103: After receiving the return of the server data, if the server returns the latest data, update the local cache data and timestamp;

[0092] Step 104: After receiving the return of the server data, if the server return does not contain the latest data, confirm that the data in the client cache is the latest and read the local cache data.

[0093] Reference Figure 2 As shown, an embodiment of the present invention provides a server-side part in a method for optimizing real-time market data calls between microservices. The method may include the following steps:

[0094] Step 201: The server receives a real-time data call request;

[0095] Step 202, perform interception preprocessing, compare the timestamp carried in the request header with the timestamp of the local cache data, if the timestamp carried in the request header is not less than the timestamp of the local cache data, execute step 203; if the timestamp carried in the request header is less than the timestamp of the local cache data, execute step 204.

[0096] Step 203: If the timestamp carried in the request header is not less than the timestamp of the server, it is confirmed that the client has the latest data, and the server does not continue to access the data and directly returns an empty data set.

[0097] Step 204: If the timestamp carried in the request header is less than the timestamp of the server, the server data is the latest and further calls and accesses to the data of the storage component are required.

[0098] Specifically, in step 102, step 202, step 203, and step 204, the technology used is the interceptor in the gRPC technology, which is divided into client and server interceptors; the client interceptor is a function called by the gRPC client before calling the actual RPC; the server interceptor is a function called by the gRPC server before reaching the actual RPC method. The local cache timestamp is obtained before the client's request is sent out. When the server receives the request, it will first compare the timestamp in the request with the local timestamp of the server. By comparing and filtering a part of the server's requests, the number of times the server actually makes service calls and responds is reduced, thereby reducing the pressure on the server.

[0099] Step 205: Based on a series of concurrent requests that continue to be called, requests for the same key query are merged, that is, requests for the same stock data are merged, where the first request that arrives actually makes the data call, while other concurrent requests for the same key query are blocked and wait for the result of the first call.

[0100] Specifically, in step 205, singleflight technology is used. Singleflight is a library in the official Go extension synchronization package, which is mainly used for concurrency control. For multiple requests for the same key, only one needs to be processed, and the remaining requests are blocked waiting for results, thereby suppressing repeated requests under concurrent access, thereby avoiding all requests falling into the database or storage component, reducing the occurrence of frequent queries for hot stocks, and reducing performance impact and downtime risks.

[0101] Step 206: Based on concurrent requests for querying the same key, the first request that arrives initiates a data call to access new data in the storage component and determine whether the process obtains data within the specified time. If the data is obtained before the timeout, execute step 207; if the timeout occurs, execute step 208.

[0102] Step 207: If the data is not acquired within the time limit, the latest acquired data is updated to the local cache, and the data result is returned to the client.

[0103] Step 208: If the data is not obtained within the time limit, a request is made to access the storage component again, and the data result is returned to the client.

[0104] In one embodiment, the method further comprises:

[0105] According to the time window sliding mechanism, the market data in the storage component of the server-side microservice is eliminated.

[0106] In the embodiment, the gRPC interceptor technology is adopted. Through the pre-processing of the request sent by the client and the interception processing of the server, the timestamp is obtained for comparison. For the request whose client timestamp is not less than the server timestamp, it is directly returned, and the client local cache data is used as the result. For the frequent request mode of market data, this process will reduce the concurrency of some requests to the server to access data; for the request whose client timestamp is less than the server timestamp, the server will actually process the data call, and adopt the singleflight technology to merge the requests for access to the same key, and judge whether there is a cache and whether it has timed out, so as to reduce the data access cost of the server to access the storage component and the generation of hot data on the server as much as possible, and prevent the breakdown of the server storage component.

[0107] Based on the same design idea, an embodiment of the present invention also provides a device for optimizing real-time market data calls between microservices. The device can be used to execute each step of the method for optimizing real-time market data calls between microservices described in the above implementation. The device can include two parts: a client and a server.

[0108] Reference Figure 3 As shown, the embodiment of the present invention also provides a client part in a device for optimizing the real-time market data call between microservices, including:

[0109] Send request preprocessing module 301, used to preprocess the request, query the request to see if there is a result set in the local cache. If there is a result set, obtain its timestamp; if there is no result set, assign the timestamp to 0, and send a data call request to the server with the timestamp;

[0110] The request return processing module 302 is used to receive the data result set returned by the server, and perform different return processing according to whether the data set is empty; if empty data is returned, confirm that the result set in the client's local cache is the latest data; if data is returned, update the client's local cache.

[0111] Reference Figure 4 As shown, the embodiment of the present invention also provides a server-side part in a device for optimizing the real-time market data call between microservices, including:

[0112] The request interception preprocessing module 401 is used to receive data and perform interception preprocessing before the data is actually called. It compares the timestamp carried in the request header with the local cache timestamp of the server. If the timestamp carried in the request header is not less than the local cache timestamp of the server, it is confirmed that the cached data of the client is the latest, and the server does not need to continue calling and directly returns empty data; if the timestamp carried in the request header is less than the local cache timestamp of the server, the request continues to be called.

[0113] The merge call processing module 402 merges the calls for the same key queries in the continued call requests, wherein the first request that arrives actually makes the data call, while other concurrent requests for the same key queries are blocked and wait for the result of the first call.

[0114] The call priority processing module 403 is based on concurrent requests for querying the same key, the first request that arrives initiates a data call, whether there is a local cache, whether the call has timed out, etc.

[0115] The data access module 404 is used to access the data in the storage component.

[0116] The data return module 405 is used to return the data result. If it is returned during the interception preprocessing, empty data is returned; if it is returned after the interception preprocessing, real data result is returned.

[0117] In one embodiment, the device further comprises:

[0118] The cache elimination module in the client microservice is used to eliminate cached data based on the time window sliding mechanism, the least recently used mechanism, and the cache capacity limit;

[0119] In one embodiment, the device further comprises:

[0120] The cache elimination module in the server-side microservice is used to eliminate cached data based on the time window sliding mechanism, the least recently used mechanism, and the cache capacity limit.

[0121] The present invention obtains the timestamps of the requests sent by the client through preprocessing and interception processing by the server, and compares them. For requests whose timestamps on the client are not less than the timestamps on the server, the requests are directly returned, and the client's local cached data is used as the result. For the frequent requests for market data, this process will reduce the concurrency of some requests to the server to access data. For requests whose timestamps on the client are less than the timestamps on the server, the server will actually process the data call, and merge the calls for the requests with the same key access, and judge whether there is a cache and whether it has timed out, so as to reduce the market stock data access cost of the server accessing the storage component and the generation of hot stock data on the server as much as possible, and prevent the breakdown of the storage component on the server.

[0122] The specific implementation scheme of this embodiment can refer to the relevant description in the above embodiment, which will not be repeated here.

[0123] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0124] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0125] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0126] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0127] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0128] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0129] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0131] The method, system, device, processor and computer-readable storage medium thereof for optimizing the processing of real-time market data calls between microservices of the present invention are adopted to solve the problem of hot data and high concurrency in market data calls in market microservice calls, and optimize the current method of storing hot stocks locally on the server or in cache components such as redis, or using cache monitoring to extend the cache time of hot data, and updating cache data through another thread. In the technical solution of the present invention, the timestamp comparison between the client and the server and the method of merging requests are used to avoid high concurrent access to hot stocks, and the characteristics of market data calls are used to separate user behavior from the processing and operation of market stock data.

[0132] In this specification, the present invention has been described with reference to specific embodiments thereof. However, it is apparent that various modifications and variations may be made without departing from the spirit and scope of the present invention. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method for optimizing the processing of real-time market data calls between microservices, characterized in that: The method comprises the steps of optimizing the real-time market data call by the client and optimizing the real-time market data call by the server. The step of optimizing the real-time market data call of the client specifically includes the following steps: (1-1) The client sends a request to the server for market data; (1-2) Check whether there is cached data of historical market results on the client side, obtain the timestamp of the cached result data, if not, obtain the timestamp as 0, carry it in the request header, send the request to the server, and wait for the server to return data; (1-3) If the data result set returned by the server is not empty, the server returns the latest data, updates the local cache data and timestamp; if the data result returned by the server is empty, the server returns without the latest data, confirms that the data in the client cache is the latest, and reads the local cache data; The steps of optimizing the real-time market data call on the server include: (2-1) The server receives real-time data call requests; (2-2) Perform interception preprocessing and compare the timestamp carried in the request header with the timestamp of the local cache data; (2-3) If the timestamp carried in the request header is not less than the timestamp of the locally cached data, it is confirmed that the client has the latest data, and the server does not continue to access the data and directly returns an empty data set; if the timestamp carried in the request header is less than the timestamp of the locally cached data, execute step (2-4); (2-4) If the timestamp in the request header is less than the timestamp on the server, the server data is the latest and further calls are needed to access the data in the storage component; (2-5) Merge the calls for the requests with the same key, call the data for the first request that arrives, and block other concurrent requests for the same key to wait for the result of the first call; (2-6) Based on concurrent requests for querying the same key, initiate a data call for the first request that arrives, access the new data in the storage component, and determine whether the process obtains the data within the specified time. If the data is obtained before the timeout, execute step (2-7); if the timeout occurs, execute step (2-8); (2-7) Update the latest data obtained to the local cache and return the data results to the client; (2-8) Request access to the storage component again and return the data results to the client.

2. The method for optimizing the processing of real-time market data calls between microservices according to claim 1 is characterized in that: The method further comprises the following steps: Monitor and delete the local cache market data of the client microservice and the server microservice based on the preset expiration time and its cache capacity.

3. The method for optimizing the processing of real-time market data calls between microservices according to claim 1 is characterized in that: The method further comprises the following steps: According to the time window sliding mechanism, the market data in the storage component of the server-side microservice is eliminated.

4. The method for optimizing the processing of real-time market data calls between microservices according to claim 1 is characterized in that: In the steps (1-2), (2-2), (2-3) or (2-4), an interceptor is used, and the interceptor includes a client interceptor and a server interceptor. The client interceptor is a function called by the gRPC client before calling the actual RPC; the server interceptor is a function called by the gRPC server before reaching the actual RPC method; the client's request obtains a local cache timestamp before being sent out, and when the server receives the request, it preferentially compares the timestamp in the request with the local timestamp of the server.

5. The method for optimizing the processing of real-time market data calls between microservices according to claim 3 is characterized in that: The steps are specifically as follows: By preprocessing the request sent by the client and intercepting it on the server, the timestamp is obtained for comparison. If the client timestamp is not less than the server timestamp, the request is returned directly, and the client's local cache data is used as the result. For requests whose client timestamp is smaller than the server timestamp, the server processes the data call and merges the requests for access to the same key.

6. A system for optimizing the processing of real-time market data calls between microservices for implementing the method of claim 1, characterized in that: The system includes a client, and the client includes: Send a request preprocessing module, which is used to preprocess the request and check whether there is a result set in the local cache. If there is a result set, obtain its timestamp; if there is no result set, assign the timestamp to 0 and send a data call request to the server with the timestamp; The request return processing module is connected to the sending request preprocessing module and is used to receive the data result set returned by the server and perform different return processing according to whether the data set is empty; if empty data is returned, it is confirmed that the result set in the client's local cache is the latest data; if data is returned, the client's local cache is updated.

7. The system for optimizing the real-time market data call processing between microservices according to claim 6 is characterized in that: The system further includes a server, which includes: The request interception preprocessing module is used to receive data and perform interception preprocessing before the data is actually called. It compares the timestamp carried in the request header with the timestamp of the local cache on the server. If the timestamp carried in the request header is not less than the timestamp of the local cache on the server, it is confirmed that the cached data of the client is the latest, and the server does not need to continue to call and directly returns empty data; if the timestamp carried in the request header is less than the timestamp of the local cache on the server, the request continues to be called; A merge call processing module is connected to the request interception preprocessing module and is used to merge the requests for the same key query in the continued call requests, wherein the first request to arrive makes a data call, while other concurrent requests for the same key query are blocked and wait for the result of the first call; A call priority processing module, connected to the merge call processing module, is used to initiate a data call for the first request that arrives based on concurrent requests for querying the same key, and process it; A data access module, connected to the call priority processing module, for accessing data in the storage component; The data return module is connected to the data access module and is used to return data results. If it is returned during the interception preprocessing, empty data is returned; if it is returned after the interception preprocessing, real data results are returned.

8. The system for optimizing the processing of real-time market data calls between microservices according to claim 6, characterized in that: The client also includes a cache elimination module of the client, which is connected to the request return processing module and is used to eliminate cache data.

9. The system for optimizing the real-time market data call processing between microservices according to claim 6, characterized in that: The server also includes a cache elimination module of the server, which is connected to the data return module and is used to eliminate cache data.

10. A device for optimizing the processing of real-time market data calls between microservices, characterized in that: The device comprises: a processor configured to execute computer-executable instructions; A memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for optimizing the processing of real-time market data calls between microservices described in any one of claims 1 to 5 are implemented.

11. A processor for optimizing the real-time market data call processing between microservices, characterized in that: The processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for optimizing the processing of real-time market data calls between microservices described in any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the method for optimizing the processing of real-time market data calls between microservices as described in any one of claims 1 to 5.

Images