A data query method and a computer device

Abstract

The present application discloses a data query method and a computer device, relating to the field of computer technology. The data query method includes: configuring a first-level cache, a second-level cache, and a third-level cache respectively according to a preset cache data classification method; when a user initiates a query request, accessing cache data according to the read routing processing rule of the cache data and in accordance with the preset cache data classification method.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data query method and a computer device. Background Art

[0002] In traditional computer systems, data storage is mostly done in a real-time manner. For example, when the system receives a piece of data information, it directly processes and stores it in the background database.

[0003] A database is a warehouse that organizes, stores, and manages data according to a data structure. The database can receive query requests and return data corresponding to the query requests. When there are too many or too frequent query requests, the response time of the database will be longer or even crash. Therefore, when performing data queries, how to reduce the response time of the request and improve the efficiency of data queries has become an urgent problem to be solved.

[0004] Accordingly, the art needs a data query method and a computer device to solve the above problems. Summary of the invention

[0005] In order to overcome the above defects, the present invention is proposed to provide a data query method and computer device that solves or at least partially solves the problem that a single cache causes poor service degradation and long data query response time.

[0006] In a first aspect, the present invention provides a data query method, characterized by comprising:

[0007] Configure the first-level cache, second-level cache, and third-level cache respectively according to the preset cache data classification method;

[0008] When a user initiates a query request, the cached data is accessed according to the read routing processing rules of the cached data and the preset cached data classification method.

[0009] In a specific embodiment, configuring the first-level cache, the second-level cache, and the third-level cache respectively according to the preset cache data classification method includes:

[0010] Initialize cache.

[0011] According to the number of times the cache data is called and the preset cache data classification method, all cache data are divided into first-level cache, second-level cache and third-level cache. The first-level cache stores dictionary data, the second-level cache stores semi-finished data processed by business rules, and the third-level cache stores hot finished data processed by business rules.

[0012] In a specific embodiment, when a user initiates a query request, accessing cache data is performed according to a read routing processing rule of cache data and a preset cache data classification method, including:

[0013] Obtaining the input parameters of the query request, and generating a cache key from the input parameters of the query request according to a preset cache key generation rule;

[0014] According to the cache key, query the hit quantity matching the cache key in the third-level cache;

[0015] If the query is successful, the number of hits matching the cache key is output; and / or

[0016] If the query fails, the first-level cache is queried to determine whether there is a hit quantity that matches the cache key.

[0017] In a specific embodiment, querying in the first-level cache and determining whether there is a hit quantity matching the cache key includes:

[0018] If the query is successful, the number of hits matching the cache key is output; and / or

[0019] If the query fails, the hit count matching the cache key is obtained from the database and placed in the first-level cache accordingly.

[0020] In a specific embodiment, the method further comprises:

[0021] After obtaining the hit count that matches the cache key in the database, query and determine whether there is a hit count that matches the cache key in the secondary cache;

[0022] If the query is successful, the number of hits matching the cache key is output; and / or

[0023] If the query fails, the basic data corresponding to the cache key is obtained from the database, and the matching hit quantity is processed by the business rules and placed in the secondary cache accordingly.

[0024] In a specific embodiment, the method further comprises:

[0025] The number of calls of the cache key is monitored, and when the number of calls is greater than a first preset number, the hit quantity matched by the cache key is put into the third-level cache.

[0026] In a specific embodiment,

[0027] Monitor the consistency of the cache data in the first-level cache and the database, the changes of the cache data in the second-level cache, and the call times of the cache keys in the third-level cache in real time through the monitoring software.

[0028] In a specific embodiment,

[0029] According to the cache key, if the query fails in any one of the first-level cache, the second-level cache, or the third-level cache, an asynchronous warning notification is sent for manual processing.

[0030] In a specific embodiment, the method further includes:

[0031] In response to the changed cache data, update the cache data in the third-level cache. If the update fails, retry. When the number of retries is greater than the second preset number, an asynchronous warning notification is sent for manual processing.

[0032] In a second aspect, the present invention provides a computer device, characterized in that the computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the data query method described in any item of the first aspect is implemented.

[0033] One or more of the above technical solutions of the present invention have at least one or more of the following beneficial effects:

[0034] By implementing the technical solution of the present invention, the problem that a single cache causes poor service degradation and a long response time for data query can be solved or at least partially solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings here are incorporated into the specification and constitute a part of this specification, showing the embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0036] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other accompanying drawings can also be obtained based on these drawings without creative efforts.

[0037] Figure 1 is a schematic main step flow diagram of the data query method in an embodiment of the present invention;

[0038] Figure 2 is a schematic step flow diagram of the data query method in another embodiment of the present invention;

[0039] Figure 3Schematic diagram of a computer device according to an embodiment of the present invention. Detailed implementation manners

[0040] In order to enable those skilled in the art to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0042] In order to solve the above technical problems, especially to solve the problems of poor service degradation and long response time for data query caused by a single cache, for this purpose, a data query method and a computer device of the present invention are proposed.

[0043] In an embodiment of the present invention, refer to the attached Figure 1 , Figure 1 is a schematic diagram of the main step flow of a data query method according to an embodiment of the present invention, as Figure 1 shown, including the following steps S1-step S2:

[0044] S1. Configure a first-level cache, a second-level cache, and a third-level cache respectively according to a preset cache data grading method.

[0045] In a specific embodiment, the step of configuring a first-level cache, a second-level cache, and a third-level cache respectively according to a preset cache data grading method includes:

[0046] Initialize and establish a cache;

[0047] According to the number of times the cache data is called and the preset cache data classification method, all cache data are divided into first-level cache, second-level cache and third-level cache. The first-level cache stores dictionary data, the second-level cache stores semi-finished data processed by business rules, and the third-level cache stores hot finished data processed by business rules.

[0048] For example, the first-level cache uses the local cache caffine to store dictionary data that rarely changes, such as the institution number-related data of the transaction application; the second-level cache uses the redis cache to store semi-finished data processed by certain business rules, such as data filtered by rules such as institutions and marketing activities; the third-level cache uses the redis cache to store hot finished data processed by business rules, such as the result set returned by the routing system.

[0049] S2. When a user initiates a query request, the cache data is accessed according to the read routing processing rules of the cache data and the preset cache data classification method.

[0050] In a specific embodiment, when a user initiates a query request, accessing cache data is performed according to a read routing processing rule of cache data and a preset cache data classification method, including:

[0051] Obtaining the input parameters of the query request, and generating a cache key from the input parameters of the query request according to a preset cache key generation rule;

[0052] According to the cache key, query the hit quantity matching the cache key in the third-level cache;

[0053] If the query is successful, the number of hits matching the cache key is output; and / or

[0054] If the query fails, the first-level cache is queried to determine whether there is a hit quantity that matches the cache key.

[0055] In one example, according to a preset key value generation rule, the query request input parameters are used to generate a cache key, and corresponding data is obtained from caches at all levels based on the generated cache key. Furthermore, one or more cache keys can be generated based on one or more single-value parameter values ​​and / or multi-value parameter values ​​corresponding to one or more single-value parameters and / or multi-value parameters in the input parameters.

[0056] It should be noted that a single-valued parameter is a parameter that includes only one element, and the value of this element is the single-valued parameter value. A multi-valued parameter is an array that includes multiple elements. If the input parameter includes a multi-valued parameter (including the cases of 0, one, or more single-valued parameters), then based on each element in the multi-valued parameter, cache keys corresponding to each element are generated respectively. Further, if the input parameter includes one or more single-valued parameters and a multi-valued parameter, then based on all the single-valued parameter values and each element in the multi-valued parameter, cache keys corresponding to each element are generated respectively. Here, what is finally generated are multiple cache keys corresponding to the multiple elements in the multi-valued parameter.

[0057] Further, the hit count corresponding to the cache key is queried in the three-level cache, where the cache key is matched with the corresponding value.

[0058] In the embodiments of the present invention, if the three-level cache is a traditional key-value (KV) non-relational database, then for the KV database, if the key and value contain multiple fields, it is inconvenient to query some of the fields. In addition, the KV data can only obtain a unique value through the key value, and it is inconvenient to query data through the value. During the actual development process, it is necessary to understand the attributes of each field in the key and value in the DB, such as: the number of primary keys, the type of each primary key, how many fields the value corresponding to the primary key has, and the type of each field, etc. In addition, in business, it often does not satisfy the mapping of obtaining a unique key value. Usually, it is necessary to pull all the data that meets a certain condition. In addition, when a value contains multiple fields and the development only focuses on one or several of them, there is often a need to search for some fields.

[0059] In the embodiments of the present invention, the routing system caches the hot finished product data after being processed by business rules into the three-level cache using the merchant code: time: region: institution number: payment method as the key. When the input parameter of the query request is obtained, the input parameter is generated into a cache key according to the preset cache key generation rule, and then the routing system queries the hit count matching the cache key in the three-level cache according to the cache key.

[0060] In the embodiments of the present invention, if the query is successful, the hit count matching the cache key is output in the three-level cache, and the hit count can be one or multiple.

[0061] In this embodiment, when querying data through a cache key, the query is performed according to key-value pairs. The three-level cache is a key-value database, and the data query request carries a query condition for data query in the key-value database. The data in the key-value database is stored in the form of key-value pairs, and the key-value pairs are used to represent the correspondence between the query condition and the cache database. In response to the data query request, the hit count matching the query condition is determined in the key-value database.

[0062] In an embodiment of the present invention, if the query fails, then query in the first-level cache and determine whether there is a hit count matching the cache key.

[0063] In a specific embodiment, the querying in the first-level cache and determining whether there is a hit count matching the cache key includes:

[0064] If the query is successful, then output the hit count matching the cache key; and / or

[0065] If the query fails, then obtain the hit count matching the cache key in the database and put it into the first-level cache accordingly.

[0066] For example, the routing system obtains the incoming parameters of the query request, generates a cache key according to the generation rule of the preset cache key for the incoming parameters of the query request, and if the query fails in the three-level cache according to the cache key, then query in the first-level cache and determine whether there is a hit count matching the cache key. Among them, if the query is successful in the first-level cache, then output the hit count matching the cache key; if the query fails in the first-level cache, then obtain the hit count matching the cache key in the DB database and put it into the first-level cache accordingly.

[0067] In a specific embodiment, referring to the appendix Figure 2 , the method further includes:

[0068] After obtaining the hit count matching the cache key in the database, query in the second-level cache and determine whether there is a hit count matching the cache key;

[0069] If the query is successful, then output the hit count matching the cache key; and / or

[0070] If the query fails, then obtain the hit count matching the cache key corresponding to the basic data in the database after being processed by the business rules and put it into the second-level cache accordingly.

[0071] In an embodiment of the present invention, the routing system obtains information such as marketing activities, calculates based on complex information such as payment method, payment time, payment location, merchant qualification, and merchant level in the transaction request sent by the user, and caches the result set in the secondary cache using merchant code: time: region: institution number as the key.

[0072] In a specific embodiment, the method further includes:

[0073] Monitoring the call count of the cache key, and when the call count is greater than a first preset count, putting the hit count matched by the cache key into the tertiary cache.

[0074] For example, monitoring the call count of the cache key in each level of cache. When the call count is greater than a first preset count (e.g., 20 times), it means that the hit count matched by the cache key is hot data with a relatively large number of usage times. Then, putting the hit count matched by the cache key into the tertiary cache, which facilitates the next call and shortens the response time of the query request.

[0075] In a specific embodiment, the consistency of the cached data in the primary cache and the database, the changes in the cached data in the secondary cache, and the call count of the cache key in the tertiary cache are monitored in real time through monitoring software.

[0076] For example, data is instrumented in each level of cache in a non-intrusive programming manner, the cached data is monitored in real time and integrated into Prometheus, the consistency of the cached data in the primary cache and the DB database is compared, the changes in the cached data in the secondary cache (calculated based on complex information such as payment method, payment time, payment location, merchant qualification, and merchant level in the transaction request sent by the user) and the call count of the cache key in the tertiary cache are monitored.

[0077] In a specific embodiment, according to the cache key, if the query fails in any one of the primary cache, secondary cache, or tertiary cache, a warning notification is sent asynchronously for manual processing.

[0078] In an example, if the query fails in any one of the primary cache, secondary cache, or tertiary cache, the routing system uses multi-threading to asynchronously send an exception warning notification for manual processing, with relatively high processing efficiency.

[0079] In a specific embodiment, the method further includes:

[0080] In response to the changed cached data, updating the cached data in the tertiary cache. If the update fails, retry. When the retry count is greater than a second preset count, a warning notification is sent asynchronously for manual processing.

[0081] In one example, when the cached data changes, cache refresh is implemented based on spring-retry, springCloud-bus, and MQKAFKA. The three-level cache related to the changed cached data is updated. If the update fails, it will be automatically retried. When the number of retries is greater than the second preset number (for example, 5 times), the routing system will asynchronously send an exception warning for manual handling.

[0082] Based on the above steps S1 - S2, a hierarchical caching system for cached data is established according to different business scenario requirements and different call times, significantly shortening the response time of query requests.

[0083] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiment of the present invention can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0084] Furthermore, the present invention also provides a computer device. Refer to the appendix Figure 3 , characterized in that the computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it executes the above-mentioned Figure 1 data query method.

[0085] Furthermore, it should be understood that since the setting of each module is only to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules can be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only illustrative.

[0086] Those skilled in the art can understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solution to deviate from the principle of the present invention. Therefore, the technical solutions after splitting or combining will all fall within the protection scope of the present invention.

[0087] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the drawings. However, those skilled in the art can easily understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims

1. A data query method, characterized in that, include: Initialize cache. All cache data are divided into first-level cache, second-level cache and third-level cache according to the number of cache data calls and the preset cache data classification method. The first-level cache stores dictionary data, the second-level cache stores semi-finished data processed by business rules, and the third-level cache stores hotspot finished data processed by business rules. When a user initiates a query request, the input parameters of the query request are obtained, and a cache key is generated from the input parameters of the query request according to a preset cache key generation rule; According to the cache key, query the hit quantity matching the cache key in the third-level cache; If the query is successful, the number of hits matching the cache key is output; If the query fails, query the first-level cache and determine whether there is a hit quantity matching the cache key; if the query succeeds, output the hit quantity matching the cache key; if the query fails, obtain the hit quantity matching the cache key in the database and put it into the first-level cache accordingly; After obtaining the hit count that matches the cache key in the database, query and determine whether there is a hit count that matches the cache key in the secondary cache; If the query is successful, the number of hits matching the cache key is output; If the query fails, the basic data corresponding to the cache key is obtained from the database, and the matching hit quantity after being processed by the business rules is put into the secondary cache accordingly; The number of calls of the cache key is monitored, and when the number of calls is greater than a first preset number, the hit quantity matched by the cache key is put into the third-level cache.

2. The method according to claim 1, characterized in that The monitoring software is used to monitor in real time the consistency of the cache data in the first-level cache and the database, the changes in the cache data in the second-level cache, and the number of calls to the cache key in the third-level cache.

3. The method according to claim 1, characterized in that According to the cache key, if the query fails in any of the first-level cache, the second-level cache or the third-level cache, a warning notification is sent asynchronously for manual processing.

4. The method according to claim 1, wherein The method further comprises: In response to the changed cache data, the cache data in the third-level cache is updated, and if the update fails, a retry is performed. When the number of retries is greater than a second preset number, a warning notification is asynchronously sent for manual processing.

5. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the data query method according to any one of claims 1 to 4 is implemented.

Images