Data access method, device, equipment, medium and product
Through the double-cache database architecture, the request code preloading mechanism is adopted to solve the problem of falling hit rate caused by the decrease in the proportion of cached database data, and improve data access efficiency and system performance.
Patent Information
- Application Number
- CN202211618074.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-15
- Publication Date
- 2025-08-26
- Estimated Expiration
- 2042-12-15
AI Technical Summary
In the prior art, as the amount of data in the database increases, the proportion of data in the cached database decreases, resulting in a decrease in the hit rate of cached data. Frequent access to the disk physical database reduces system performance, especially when massive access requests are made.
The double-cache database architecture is adopted, the first-level cache database caches user data, and the second-level cache database caches business data. By generating a request code, the data in the physical database in the disk is preloaded to the second-level cache to improve the cached data hit rate.
Improve data access efficiency, avoid system performance degradation caused by frequent access to disk databases during massive access requests, and improve cached data hit rate and system performance.
Smart Images

Figure CN115905321B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of data access technology, and in particular relates to a data access method, device, equipment, medium and product. Background Art
[0002] With the development of network services, more and more data needs to be stored in the database, and more and more data access requests are received.
[0003] In related technologies, data is generally stored through a cache database and a disk physical database. The disk physical database stores all data, while the cache database has a limited capacity. When the amount of data is small, the cache database can cache all the data stored in the disk physical database. However, when the amount of data is large, the cache database can only cache part of the data stored in the disk physical database. When an access request for target data is received, the target data can be searched from the cache database first. If the target data cannot be found in the cache database, it is necessary to search from the disk physical database.
[0004] Therefore, as the amount of data stored in the physical disk database increases, the proportion of data cached in the cache database becomes smaller and smaller, which will lead to a decrease in the hit rate of the cache data. In other words, it will often happen that the target data cannot be found in the cache database and needs to be searched in the physical disk database, thus reducing data access efficiency. Especially when receiving massive access requests, frequent access to the physical disk database will reduce system performance. Summary of the Invention
[0005] The embodiments of the present application provide a data access method, apparatus, device, medium and product, which can improve the hit rate of cached data, thereby improving data access efficiency, and also avoid the problem of frequent access to disk databases degrading system performance when receiving massive access requests.
[0006] In a first aspect, an embodiment of the present application provides a data access method, the method comprising:
[0007] Receiving a request from a target system to access target data;
[0008] In a case where the access request is a first access request for target user data, determining the target user data from the first-level cache database in response to the first access request;
[0009] Allow the target system to access the target user data;
[0010] In a case where the access request is a second access request for the target business data, in response to the second access request, generating a target request code based on the target user data cached in the first-level cache database, and sending the target request code to the target system and the disk physical database;
[0011] According to the target request code, the target business data in the disk physical database is loaded into the secondary cache database, and the target request code is loaded into the request code validation queue of the secondary cache database to make the target request code valid;
[0012] Upon receiving the target request code sent by the target system, allowing the target system to access the target business data;
[0013] Among them, the first-level cache database is used to cache user data, the second-level cache database is used to cache business data, and the disk physical database is used to store user data and business data.
[0014] In a second aspect, an embodiment of the present application provides a data access device, the device comprising:
[0015] A receiving module, configured to receive a request from a target system to access target data;
[0016] a first determining module configured to, when the access request is a first access request for target user data, determine the target user data from the first-level cache database in response to the first access request;
[0017] A first processing module, configured to allow a target system to access target user data;
[0018] a second determining module configured to, if the access request is a second access request for target business data, generate a target request code based on the target user data cached in the first-level cache database in response to the second access request, and send the target request code to the target system and the disk physical database;
[0019] The first loading module is used to load the target business data in the disk physical database into the secondary cache database according to the target request code, and load the target request code into the request code validation queue of the secondary cache database to validate the target request code;
[0020] The second processing module is configured to allow the target system to access the target business data upon receiving the target request code sent by the target system;
[0021] Among them, the first-level cache database is used to cache user data, the second-level cache database is used to cache business data, and the disk physical database is used to store user data and business data.
[0022] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions;
[0023] When the processor executes the computer program instructions, the data access method as shown in any one of the embodiments of the first aspect is implemented.
[0024] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, which, when executed by a processor, implements the data access method shown in any one of the embodiments of the first aspect.
[0025] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data access method shown in any one of the embodiments of the first aspect.
[0026] The data access method, apparatus, device, medium and product of the embodiments of the present application can receive an access request from a target system to target data. When the access request is a first access request to target user data, in response to the first access request, the target user data can be determined from the first-level cache database, and the target system can be allowed to access the target user data. When the access request is a second access request to target business data, in response to the second access request, a target request code can be generated based on the target user data cached in the first-level cache database, and the target request code can be sent to the target system and the disk physical database. Then, based on the target request code, the target business data in the disk physical database can be loaded into the second-level cache database, and the target request code can be loaded into the request code validity queue of the second-level cache database to make the target request code valid. Then, upon receiving the target request code sent by the target system, the target system can be allowed to access the target business data. That is to say, the first-level cache database can cache user data, and the second-level cache database can load the data that the target system needs to access from the disk physical database according to the access request. When only user data is accessed, it can be searched only from the user data cached in the first-level cache database. When user data is accessed, it can be searched from the business data cached in the second-level cache database. Therefore, the hit rate of cached data can be improved, thereby improving data access efficiency, and avoiding the problem of frequent access to the disk database reducing system performance when receiving massive access requests. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0028] Figure 1 This is a workflow diagram of an existing database provided by an embodiment of the present application;
[0029] Figure 2 This is a flow chart of a data access method provided by one embodiment of the present application;
[0030] Figure 3 This is a schematic diagram of a first-level cache database caching user data provided by an embodiment of the present application;
[0031] Figure 4 A schematic diagram of an application scenario of a request code provided by an embodiment of the present application;
[0032] Figure 5 This is a schematic diagram of a data access scenario provided by an embodiment of the present application;
[0033] Figure 6 This is another data access scenario diagram provided by an embodiment of the present application;
[0034] Figure 7 This is a schematic diagram of another data access scenario provided by an embodiment of the present application;
[0035] Figure 8 This is a schematic diagram of another data access scenario provided by an embodiment of the present application;
[0036] Figure 9 This is an architectural diagram of a data access system provided by one embodiment of the present application;
[0037] Figure 10 This is a structural diagram of a data access device provided by an embodiment of the present application;
[0038] Figure 11 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0039] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.
[0040] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.
[0041] As mentioned in the background technology, with the development of the real-name query and verification business on the existing network, the real-name query and verification business of the customer center of the entire network needs to support more and more peripheral system access, and the customer information content queried is also increasing. The existing technical architecture can no longer meet the huge request volume, and the memory cache capacity is limited. It cannot meet high-frequency and massive requests during business peak periods. It is very easy for the memory cache data hit rate to decrease, and access to the disk physical database will affect system performance.
[0042] like Figure 1 As shown, the existing database includes a cache database and a disk physical database.
[0043] For read operations, when a request to read data is received from an application, it is first read from the cache database. If the read is successful, the corresponding data is returned. If the read fails, it is read from the physical disk database.
[0044] For write operations (or update operations), when a request is received from an application to write (or update) data, the data in the cache database must be invalidated first, then the data is written (or updated) in the physical disk database, and then the data in the physical disk database is synchronized to the cache database.
[0045] In the existing cache database + disk-based physical database architecture, data is checked through interfaces. As data increases and cache database storage space is limited, the amount of data in the cache database gradually decreases. This increases the probability of cache data misses and accessing the disk-based physical database, thus reducing data access speed. Frequent access to the disk-based physical database can also degrade system performance when receiving massive access requests.
[0046] Embodiments of the present application provide a data access method, apparatus, device, medium, and product, which can receive an access request from a target system to target data. When the access request is a first access request to target user data, in response to the first access request, the target user data can be determined from a first-level cache database, allowing the target system to access the target user data. When the access request is a second access request to target business data, in response to the second access request, a target request code can be generated based on the target user data cached in the first-level cache database, and the target request code can be sent to the target system and the disk physical database. Then, based on the target request code, the target business data in the disk physical database can be loaded into the second-level cache database, and the target request code can be loaded into the request code validation queue of the second-level cache database to validate the target request code. Then, upon receiving the target request code sent by the target system, the target system can be allowed to access the target business data. That is to say, the first-level cache database can cache user data, and the second-level cache database can load the data that the target system needs to access from the disk physical database according to the access request. When only user data is accessed, it can be searched only from the user data cached in the first-level cache database. When user data is accessed, it can be searched from the business data cached in the second-level cache database. Therefore, the hit rate of cached data can be improved, thereby improving data access efficiency, and avoiding the problem of frequent access to the disk database reducing system performance when receiving massive access requests.
[0047] Figure 2 A flow chart of a data access method provided by an embodiment of the present application is shown. It should be noted that the execution subject of the data access method may be a data access system. It should be noted that the execution subject does not constitute a limitation of the present application.
[0048] like Figure 2 As shown, the data access method may include the following steps:
[0049] S210, receiving a request from a target system to access target data;
[0050] S220 , in a case where the access request is a first access request for target user data, determining the target user data from the first-level cache database in response to the first access request;
[0051] S230, allowing the target system to access the target user data;
[0052] S240 , in a case where the access request is a second access request for the target business data, in response to the second access request, generating a target request code based on the target user data cached in the first-level cache database, and sending the target request code to the target system and the disk physical database;
[0053] S250, according to the target request code, the target business data in the disk physical database is loaded into the secondary cache database, and the target request code is loaded into the request code validation queue of the secondary cache database to validate the target request code;
[0054] S260: Upon receiving the target request code sent by the target system, allow the target system to access the target business data.
[0055] In this way, an access request from the target system to the target data can be received. In the case where the access request is a first access request to the target user data, in response to the first access request, the target user data can be determined from the first-level cache database, and the target system can be allowed to access the target user data; in the case where the access request is a second access request to the target business data, in response to the second access request, a target request code can be generated based on the target user data cached in the first-level cache database, and the target request code can be sent to the target system and the disk physical database, and then, based on the target request code, the target business data in the disk physical database can be loaded into the second-level cache database, and the target request code can be loaded into the request code validity queue of the second-level cache database to make the target request code valid, and then, upon receiving the target request code sent by the target system, the target system can be allowed to access the target business data. That is to say, the first-level cache database can cache user data, and the second-level cache database can load the data that the target system needs to access from the disk physical database according to the access request. When only user data is accessed, it can be searched only from the user data cached in the first-level cache database. When user data is accessed, it can be searched from the business data cached in the second-level cache database. Therefore, the hit rate of cached data can be improved, thereby improving data access efficiency, and avoiding the problem of frequent access to the disk database reducing system performance when receiving massive access requests.
[0056] Regarding S210 , the target system may be a system capable of initiating an access request, a peripheral system, or an internal system, which is not limited here.
[0057] The target data may include target user data and / or target service data. The target user data may be basic information of the user, and the target service data may be service data of the user.
[0058] In the data access method provided in the embodiment of the present application, the first-level cache database can be used to cache the full amount of user data, the second-level cache database can be used to cache part of the business data, and the disk physical database can be used to store the full amount of user data and the full amount of business data.
[0059] That is, the target user data is cached in the first-level cache database, the target user data and target business data are stored in the disk physical database, and the target business data may or may not be cached in the second-level cache database.
[0060] For example, the first-level cache database can store all user data, which may include mobile phone numbers, user names, and ID codes. Each piece of user data can be stored in the first-level cache database in the form of an encrypted string row, such as Figure 3 As shown, the mobile phone number is encrypted with MD5 (32 bits), and the "user name + ID code" is encrypted with MD5 (32 bits). The two sets of encrypted strings are combined to form a unique index "MD5 (mobile phone number) + MD5 (user name and ID code)" in the first-level cache database.
[0061] Exemplarily, business data may be stored in the secondary cache database, and the business data may include but is not limited to user subscription data, user rights information, package balance information, and user account information.
[0062] For example, both the first-level and second-level cache databases can be Redis, and the physical disk database can be BC-PostgreSQL. Redis can support hundreds of thousands of transactions per second (TPS) with only a few milliseconds of latency, meeting the requirements of high front-end concurrency and fast response. The physical disk database can prevent data loss.
[0063] The access request may be a request to add, delete, modify, or query the target data.
[0064] In step S220 , the first access request is for target user data. The first-level cache database can store all user data. If the target system initiates a first access request for target user data, the target user data can be determined from the first-level cache database.
[0065] Regarding S230 , after determining the target user data from the first-level cache database, the data access system may allow the target system to access the target user data.
[0066] Exemplarily, if the first access request is a query request for target user data, the target user data may be directly queried from the first-level cache database, and the query result may be returned.
[0067] In some implementations, when the first access request is a request to modify the target user data, in order to modify the target user data more securely and efficiently, the above S230 may include:
[0068] In response to the first access request, if the target system identity authentication is passed, clear the target user data in the first-level cache database, and modify the target user data in the disk physical database according to the modification data carried in the first access request to obtain the modified target user data;
[0069] Load the modified target user data from the physical disk database into the first-level cache database.
[0070] Here, the first access request may be a request to modify the target user data. The authentication method may be password authentication, verification code authentication, or face authentication. Of course, other authentication methods are also possible and are not limited here.
[0071] Specifically, if the first access request is a request to modify the target user data, the data access system can respond to the modification request and prompt the user to authenticate through the target system. If the authentication is successful, it means that the user has the authority to modify the target user data. Therefore, the target user data in the first-level cache database can be cleared first, and the target user data in the disk physical database can be modified according to the modification data carried in the first access request to obtain the modified target user data, and then the modified target user data can be loaded from the disk physical database to the first-level cache database.
[0072] In this way, the security of the target user data can be guaranteed through identity authentication. When modifying the target user data, there is no need to clear all the data in the first-level cache database first, thereby improving the efficiency of data modification.
[0073] Regarding S240 , the second access request may be a request for access to target business data.
[0074] The target request code may be generated based on the target user data. For example, the target request code may be the unique index "MD5 (mobile phone number) + MD5 (user name and ID code)".
[0075] In addition to the string encrypted from the target user data, the target request code can also include a string encrypted from a variable to avoid the risk of data leakage caused by the same target request code being generated multiple times based on the target user data. For example, the variable can be the time the request code was generated.
[0076] In some implementations, in order to perform the access task more reasonably and orderly, after S240, the method may further include:
[0077] Insert the target request code into the request code queue in the first-level cache database in the order of the target request code generation time from earliest to latest;
[0078] Determine the queue number of the target request code according to the order of the target request code in the request code queue;
[0079] According to the queue sequence number of the target request code, the target business data corresponding to the target request code is determined from the disk physical database.
[0080] Here, the first-level cache database may include a request code queue, into which request codes may be inserted in descending order of their generation time. The order of the request codes in the request code queue may then be used to determine their queue numbers, and the business data corresponding to the request codes may then be determined from the physical disk database in the order of the queue numbers.
[0081] In this way, by inserting the request code into the request code queue according to the generation time and determining the queue number, and then according to the order of the queue number, the business data corresponding to the request code can be determined from the disk physical database more reasonably and orderly, so that the access task can be executed more reasonably and orderly.
[0082] Regarding S250, sending the target request code to the disk physical database enables the disk physical database to determine the target business data corresponding to the target request code based on the target request code, and load the target business data and the target request code into the secondary cache database. Sending the target request code to the target system enables the target system to access the target business data preloaded into the secondary cache database based on the target request code.
[0083] In some embodiments, when a first access request for target user data is received from a target system, a target request code can also be generated based on the target user data cached in the first-level cache database, and the target request code can be sent to the target system and the disk physical database, so that the disk database can pre-load the target business data into the second-level cache database, and when a second access request for the target business data is received from the target system, the access result can be returned more quickly.
[0084] The secondary cache database may include a request code validation queue. The request codes in the request code validation queue are valid, and the business data corresponding to the request codes can be accessed according to the request codes.
[0085] In some implementations, if the target business data has been previously loaded into the secondary cache database and has not been cleared, it is not necessary to load the target business data from the physical disk database again after receiving the second access request.
[0086] Regarding S260, if the user wants to access the target business data, the target system may send a target request code to the data access system, and the data access system may allow the target system to access the target business data corresponding to the target request code.
[0087] In some implementations, in order to modify the target service data more securely and efficiently, when the second access request is a request to modify the target service data, the above S260 may include:
[0088] In response to the second access request, if the target system identity authentication is passed, clear the target business data in the secondary cache database, and modify the target business data in the disk physical database to obtain the modified target business data;
[0089] The modified target business data is loaded from the physical disk database to the secondary cache database.
[0090] Here, the second access request may be a request to modify the target business data. The authentication method may be password authentication, verification code authentication, or face authentication. Of course, other authentication methods are also possible and are not limited here.
[0091] Specifically, if the second access request is a request to modify the target business data, the data access system can respond to the modification request and prompt the user to authenticate through the target system. If the authentication is successful, it means that the user has the authority to modify the target business data. Therefore, the target business data in the secondary cache database can be cleared, and the target business data in the disk physical database can be modified to obtain the modified target business data, and then the modified target business data can be loaded from the disk physical database into the secondary cache database.
[0092] It should be noted that if there is an unexpired request code corresponding to the target business data when the second access request is received, it is necessary to first clear the target business data in the secondary cache database, and then modify the target business data in the disk physical database to obtain the modified target business data, so as to avoid accessing the target business data before modification through the unexpired request code.
[0093] In this way, the security of the target business data can be guaranteed through identity authentication. When modifying the target business data, there is no need to clear all the data in the secondary cache database first, thereby improving the efficiency of data modification.
[0094] In some embodiments, to facilitate storing the target data in a database or deleting the target data from a database, when the target data includes target user data and target service data, after S210, the method may further include:
[0095] In the case where the access request is a request to add target data, in response to the access request, adding target user data and target business data in the disk physical database;
[0096] Load the target user data from the physical disk database into the first-level cache database;
[0097] In the case where the access request is a request to delete the target data, in response to the access request, the target user data in the first-level cache database is deleted, and the target user data and target business data in the disk physical database are deleted.
[0098] Here, if a request to add target data is received from the target system, the target data can be added to the disk physical database in response to the addition request, and then the target user data included in the target data can be loaded from the disk physical database to the first-level cache database.
[0099] If a deletion request for target data is received from the target system, the target user data in the first-level cache database can be deleted in response to the deletion request, and the target user data and target business data in the disk physical database, that is, the target data, can also be deleted. If the target business data exists in the second-level cache database, the target business data in the second-level cache database also needs to be deleted.
[0100] For example, if a new user handles a business, the new user's user data a and business data A need to be stored in the database. At this time, the staff can initiate an add request for user data a and business data A through the target system, and then the data access system can respond to the add request and add user data a and business data A to the disk physical database, and then load user data a from the disk physical database into the first-level cache database.
[0101] In this way, the target data can be stored in the database or deleted from the database more conveniently and quickly without clearing the cache database first.
[0102] In some implementations, to improve data security, after S260, the method may further include:
[0103] The target request code is cleared from the request code validity queue and inserted into the request code invalidation queue in the secondary cache database to invalidate the target request code;
[0104] Periodically send the request codes in the request code invalidation queue to the first-level cache database;
[0105] The request code queue is cleared of request codes that are identical to the request code in the request code invalidation queue.
[0106] That is, after the target system accesses the target service data based on the target request code, the target request code can be removed from the request code validation queue and inserted into the request code invalidation queue, invalidating the target request code. Once the target request code has expired, the target service data cannot be accessed using that request code. This prevents multiple attempts to access the target service data using that request code, improving the security of the target service data.
[0107] Furthermore, the request codes in the request code expiration queue can be periodically sent to the primary cache database so that, based on the request codes in the request code expiration queue, the request codes in the request code expiration queue that are identical to the request codes in the request code expiration queue can be cleared from the request code queue. In other words, expired request codes can be cleared from the request code queue to avoid occupying storage space. Furthermore, after the request codes in the request code expiration queue are sent to the primary cache database, the request code expiration queue can be cleared to avoid occupying storage space.
[0108] In this way, after the target system accesses the target business data based on the target request code, the target request code is removed from the request code validation queue and inserted into the request code invalidation queue, rendering the target request code invalid after use. This prevents the target business data from being accessed again using the target request code, thereby improving data security. Furthermore, by periodically sending the request codes in the request code invalidation queue to the first-level cache database to clear the request code queue of request codes that are identical to those in the request code invalidation queue, excessive storage space usage can be avoided.
[0109] In some implementations, to further improve data security, the target request code may carry an expiration time of the target request code. After S240, the method may further include:
[0110] When the expiration time of the target request code is reached, the target request code in the request code queue and the request code validity queue is cleared to make the target request code invalid.
[0111] Here, when generating the target request code, an expiration time of the target request code can also be generated. When the expiration time of the target request code is reached, even if the target request code has not been used, the target request code needs to be removed from the request code queue and the request code validation queue to render the target request code invalid.
[0112] In this way, through the above process, the target request code can be made to expire, thereby further improving the security of the data.
[0113] In some implementations, to improve the accuracy of data access, before S210, the method may further include:
[0114] Upon receiving a third access request for the target service data, generating a first request code based on the target user data cached in the first-level cache database in response to the third access request;
[0115] After the above S210, the method may further include:
[0116] When the access request is a second access request for the target business data and the first request code is not invalid, in response to the second access request, the first request code in the request code queue and the request code validity queue is cleared to invalidate the first request code.
[0117] Here, before receiving the second access request from the target system for the target business data, another access request for the target business data, i.e., a third access request, may have been received, and a first request code may have been generated based on the third access request. The first request code may not have been used and may not have reached its expiration time, and therefore has not expired.
[0118] When a second access request for target business data is received from the target system, if the first request code exists in the request code queue and the request code validity queue, the first request code can be cleared to invalidate the first request code. The target request code is then generated and inserted into the request code queue.
[0119] In this way, by invalidating the request code generated earlier for the same data, it can be ensured that there is only one request code for the same data at the same time, avoiding data changes that lead to inconsistent data accessed by multiple request codes for the same data, thereby affecting the accuracy of the data.
[0120] In some implementations, to avoid insufficient storage space in the secondary cache database, the above S250 may include:
[0121] When the number of request codes in the request code validity queue reaches the quantity threshold, the second request code inserted earliest in the request code validity queue is deleted, and the service data corresponding to the second request code is released;
[0122] The target business data in the disk physical database is loaded into the secondary cache database, and the target request code is loaded into the request code validation queue to make the target request code valid.
[0123] Here, since the amount of business data is usually large, the amount of business data cached in the secondary cache database is limited, and there is a one-to-one correspondence between valid request codes and business data. Therefore, the capacity of the request code validity queue can be limited to avoid the secondary cache database from storing too much business data.
[0124] The quantity threshold can be set according to actual needs and is not limited here.
[0125] When the number of request codes in the request code validity queue reaches the quantity threshold, the second request code inserted earliest in the request code validity queue can be deleted, and the business data corresponding to the second request code can be released to provide sufficient storage space to load the target business data in the disk physical database into the secondary cache database and load the target request code into the request code validity queue.
[0126] In this way, by limiting the capacity of the request code validity queue, when the number of request codes reaches the quantity threshold, the earliest inserted request code is deleted and the corresponding business data is released, which can provide storage space for the business data subsequently loaded into the secondary cache database, avoiding insufficient storage space in the secondary cache database.
[0127] In existing solutions that use a single cache database and a physical disk database, cache data expiration strategies primarily include timed expiration (key expiration upon key expiration), lazy expiration (accessing a key determines if it has expired and then clears it), and periodic expiration (regularly scanning for expired key data and clearing any expired keys). Lazy expiration often results in a large number of expired keys not being accessed again and therefore unable to be cleared, consuming a large amount of memory. Timed expiration, on the other hand, results in frequent scans in milliseconds, consuming CPU resources and reducing system data processing performance.
[0128] According to statistics, based on the existing solution using a single cache database + disk physical database data, the cache hit rate is shown in Table 1. Based on the solution using a dual cache database + disk physical database data provided by the embodiment of the application, the cache hit rate is shown in Table 2.
[0129] Table 1 - Existing cache hit ratios
[0130]
[0131] Table 2 - Cache hit rate of this application
[0132] Y month 2021 User data verification Business data query Total Total number of requests 178923526 17023415 195946941 Average processing time 16.915ms 18.638ms 35.553ms Hit rate 99.95% 99.98% 99.96%
[0133] From the user data verification and business data query requests in Month X of 2021 in Table 1, it can be seen that the number of user data verification requests is huge, reaching over 100 million, which is more than ten times that of business data queries. The existing architecture of a single cache database + a disk physical database cannot meet business needs. A large amount of invalid business data will exist in the single cache database, occupying cache space, reducing cache data capacity, and resulting in a low cache hit rate. This can easily lead to the situation where the accessed data is not in the cache database and a request to the disk physical database is required.
[0134] To achieve a cache hit rate of over 99% for external application requests and speed up data access, in an embodiment of the present application, user data is fully loaded into the first-level cache database in the form of indexed string encoding, meeting millisecond-level verification request responses for user data verification requests. At the same time, business data queries are preloaded into the second-level cache database in a phased request manner, meeting a cache hit rate of over 99% for tens of millions of business data. This solves the problem of frequent disk physical database access in existing solutions, improves the cache hit rate, avoids cache penetration and cache avalanche, and enhances the security of external application data queries.
[0135] Because business data in the secondary cache database is constantly refreshed with business requests, a dual-cache data loading and elimination mechanism is introduced to ensure that the secondary cache database contains the latest business data. The main process is that when an application initiates a business query request, a request code is generated and fed back to the application. Simultaneously, the physical database on disk is notified to preload business data into the secondary cache database according to the request code sequence. When the application initiates a query request with the request code, the secondary cache database returns the business data corresponding to the request code.
[0136] From the system performance data in Table 2, which shows the dual-caching strategy implemented in this application after upgrading in January 2021, we can see that, with the exception of a few exceptional misses, the vast majority of requests hit both the L1 and L2 caches, achieving the design effect of dual caching to handle access requests separately based on business access categories. In terms of data access performance, the total access time increased by 4-5 times; in terms of cache hit rate, the total hit rate increased by 19%.
[0137] The preloading of business data in the secondary cache database mainly depends on the generation and invalidation of the request code, and maintains consistency with the status of the request code. The dual cache data loading elimination mechanism can include four types: invalidation after use, invalidation at expiration, early invalidation and forced invalidation.
[0138] Based on the above embodiments, the application scenarios of the request codes provided in the embodiments of the present application are introduced in detail below.
[0139] The application scenarios of the request code provided in the embodiment of the present application can be as follows: Figure 4 shown.
[0140] 1. Common scenarios for business data query:
[0141] S401 receives a query request for target business data and feeds back a target request code and expiration time to the application;
[0142] S402: A request code generator in the first-level cache database generates target request code information, which may include a target request code, expiration time, and a request code queue sequence number, and loads the target request code information into the request code queue.
[0143] S403: Initiate a request to preload target business data to the disk physical database according to the queue sequence number;
[0144] S404, pre-load the target business data from the disk physical database to the secondary cache database, and insert the target request code into the request code validation queue according to the queue sequence number;
[0145] S405, receiving a query request carrying a target request code, and the secondary cache database returns target business data;
[0146] S418, after the target request code is used, it is moved from the request code valid queue to the request code invalid queue;
[0147] S406: The secondary cache database may notify the primary cache database of all request code information in the request code invalidation queue every 500ms.
[0148] S407, at the same time, the secondary cache database eliminates all request code information in the request code invalidation queue and releases the corresponding business data resources;
[0149] S408 , the first-level cache database receives the invalid request code information notified by the second-level cache database, and eliminates the invalid request code information in the request code queue.
[0150] 2. Scenario where the request is not queried before the request code expires:
[0151] S401, receiving a query request for target business data;
[0152] S402, generating target request code information;
[0153] S409: The target request code information is added to the request code queue, and the request code information that has been added to the request code queue and has not expired is moved back;
[0154] S403, while generating target request code information, initiate a request to preload target business data to the disk physical database according to the request code queue sequence number;
[0155] S404: The disk physical database preloads the target business data into the secondary cache database, and simultaneously inserts the target request code information into the request code validation queue in the secondary cache database according to the request code queue sequence number;
[0156] S410, the non-expired request code information added to the request code validation queue is moved backward;
[0157] S411: In the request code queue of the first-level cache database, if no service is initiated when the request code expires, the request code information can be directly eliminated;
[0158] S412, in the request code validity queue of the secondary cache database, if no business is initiated before the request code expires, the request code information will be directly eliminated after the request code expires. The secondary cache database can also release the business data corresponding to the request code information to avoid invalid data occupying cache space for a long time.
[0159] 3. Scenario where the same user frequently initiates business data queries in a short period of time:
[0160] S401, receiving a query request for target business data and feeding back a target request code and expiration time to the application;
[0161] S402: The request code generator in the first-level cache database generates target request code information and loads the request code information into the request code queue;
[0162] S403, while generating target request code information, initiate a request to preload target business data to the disk physical database according to the request code queue sequence number;
[0163] S404, pre-load the target business data into the secondary cache database, and insert the target request code into the request code validation queue according to the request code queue sequence number;
[0164] S413: The same user frequently initiates business data query requests within a short period of time. At this time, the request code information already in the first-level cache database and the second-level cache database has not expired. The request code generator in the first-level cache database generates request code information again. When adding the new request code information to the request code queue, it finds that an earlier request code information corresponding to the target business data exists.
[0165] S414, the first-level cache database notifies the second-level cache database to invalidate the earlier request code information;
[0166] S415, simultaneously eliminating the earlier request code in the request code queue of the first-level cache database;
[0167] S416, the secondary cache database receives the invalid request code notified by the primary cache database, directly eliminates the request code information in the request code validity queue of the secondary cache database, and releases the target business data at the same time to avoid inconsistencies caused by data changes affecting business accuracy.
[0168] 4. Scenario where a large number of users initiate business data queries simultaneously:
[0169] S401, receiving a query request for target business data and feeding back a target request code and expiration time to the application;
[0170] S402: The request code generator in the first-level cache database generates target request code information and loads the request code information into the request code queue;
[0171] S403, while generating the target request code, initiate a request to preload the target business data to the disk physical database according to the request code queue sequence number;
[0172] S404, pre-load the target business data into the secondary cache database, and insert the target request code into the request code validation queue according to the request code queue sequence number;
[0173] S409: The application program initiates a large number of other business data query requests again, generating a large number of request code information. A large number of request code information is added to the request code queue, and the request code information that has been added to the request code queue and has not expired is moved back;
[0174] S410, in the request code validation queue of the secondary cache database, the non-expired request code information that has been added to the request code validation queue is moved backward;
[0175] S417: Since the capacity of the secondary cache database is limited and generally cannot store all business data, the capacity of the request code validation queue can be limited. When the number of request code information reaches a threshold, if a new request code information is needed, the earlier non-expired request code information in the request code validation queue can be forcibly invalidated and moved to the request code invalidation queue, giving priority to releasing the business data corresponding to the forcibly invalidated request code information in the secondary cache database.
[0176] S406: The secondary cache database may notify the primary cache database of all request code information in the request code invalidation queue every 500 ms. The secondary cache database will also eliminate the business data corresponding to the request code information to release resources.
[0177] S408: The first-level cache database receives the notification from the second-level cache database and eliminates the invalid request code information in the request code queue.
[0178] Based on the above embodiments, the various data access scenarios involved in the embodiments of the present application are introduced in detail below.
[0179] Scenario 1: Receive a verification request for target user data, route it to the first-level cache database according to the verification request, and the first-level cache database returns the verification result according to the verification request.
[0180] Scenario 2: Receive a query request for the target business data and route it to the first-level cache database based on the query request. The first-level cache database returns the verification result based on the verification request information of the user data. If the verification is successful, it returns the target request code and the expiration time of the target request code. If there is a previously generated request code that has not expired, it can be immediately invalidated, and a request to invalidate the target business data is initiated to the second-level cache database. The first-level cache database initiates a query request for the target business data to GoldenDB based on the target user information. GoldenDB preloads the query result into the second-level cache database and adds the target request code to the request code validation queue. Then, it can receive a query request for the target business data carrying the target request code and route it to the second-level cache database based on the query request. After the second-level cache database verifies the target request code and passes it, it feedbacks the target business data, and the target business data is invalidated at the same time.
[0181] Scenario 3: Figure 5 As shown, S501, receives a modification request for the target user data, routes it to the first-level cache database according to the modification request, and after the first-level cache database is successfully verified, invalidates the target user data in the first-level cache database; S502, simultaneously initiates a modification request to the disk physical database; S503, after the modification operation is completed, loads it into the first-level cache database according to the first-level cache index string requirements.
[0182] Scenario 4: Figure 6 As shown, S601, receives a modification request for the target business data, and routes it to the first-level cache database according to the modification request. After the first-level cache database successfully verifies, it can return the target request code and the expiration time of the target request code; S602, simultaneously initiates a modification request for the target business data to the disk physical database; S603, if there is a previously generated request code for the target business data that has not expired, then the previously generated request code is immediately invalidated, and the target business data in the second-level cache database is invalidated first; S604, after the modification operation is completed, the modified target business data is loaded into the second-level cache database; S605, then, if an external application initiates a query request carrying the target request code, it can be routed to the second-level cache database according to the query request. After the second-level cache database verifies the request code and passes, it feedbacks the latest business data.
[0183] Scene 5, such as Figure 7As shown, S701, receives a request to add target data, routes it to the first-level cache database according to the add request, and if the target data does not exist in the first-level cache database, generates a target request code and an expiration time of the target request code, and returns the target request code and the expiration time to the external application (target system); S702, simultaneously initiates a request to add target data to the disk physical database; S703, after the disk physical database adds the target data, loads the target user data to the first-level cache database; S704, loads the target business data to the second-level cache database; S705, then, if the external application initiates a query request carrying the target request code, it can be routed to the second-level cache database according to the query request, and after the second-level cache database verifies that the target request code is passed, it feeds back the latest business data.
[0184] Scene 6, such as Figure 8 As shown, at S801, the data access system can receive a delete request for target data and route the request to the first-level cache database, invalidating the target user data in the first-level cache database. At S802, a delete request for the target data is simultaneously initiated to the physical disk database, deleting the target data in the physical disk database. At S803, it should be noted that if the request code generated before receiving the delete request has not expired, the request code must also be immediately invalidated, and the target business data in the second-level cache database must be deleted first, followed by the target data in the physical disk database.
[0185] Figure 9 The figure shows an architecture diagram of a data access system provided by one embodiment of the present application.
[0186] like Figure 9 As shown, the data access system may include: a service access module 901 , a scheduling management module 902 , a cache management module 903 , a data cache module 904 and a disk physical database 905 .
[0187] The service access module 901 can be used to provide unified service access capabilities and cluster management for business applications. External applications can access cache data by calling an application program interface (API).
[0188] The scheduling management module 902 can be used to manage the internal service status of the system and schedule tasks. It can perceive the status of the entire system, task information, reception of external task requests, and task scheduling and issuance.
[0189] The cache management module 903 may include a cache loading unit 9031 , a request code generating unit 9032 , a request code applying unit 9033 and an elimination and refreshing unit 9034 . Among them, the cache loading unit 9031 can be used to load data from the disk physical database 905 to the data cache module 904 at the beginning of the system, and can also refresh the user data in the first-level cache database 9041 in real time according to the business situation to ensure that the user data in the first-level cache database 9041 is consistent with the user data in the disk physical database 905; the request code generation unit 9032 can be used to cooperate with the first-level cache access, generate a request code during business authentication, add it to the request code queue management, and trigger the second-level cache database 9042 to preload; the request code application unit 9033 can be used to cooperate with the second-level cache access, and after preloading the second-level cache according to the request code, insert the request code into the request code validity queue, insert the request code into the request code invalidation queue after the request code is used or reaches the expiration time, and clear the request code in the request code queue after reaching the expiration time; the elimination refresh unit 9034 can be used to continuously load and refresh the business data in the second-level cache database 9042 based on the access request for the business data. The elimination refresh mechanism is mainly aimed at the business data in the second-level cache database, eliminates and cleans up the invalid data according to the request code, and preloads and refreshes the business data.
[0190] The data cache module 904 may include a first-level cache database 9041 and a second-level cache database 9042. The first-level cache database 9041 can be used to cache all user data, satisfying user data authentication, query, and verification capabilities. The second-level cache database 9042 can be used to dynamically load business data based on access requests for business data. After user authentication is passed, business data is preloaded from the disk physical database 905, improving cache data hit rate and data access speed.
[0191] The disk physical database 905 can be used to store all user data and all business data.
[0192] The data access system provided by the embodiment of the present application improves the cache strategy of data access. Its principle is as follows: the first-level cache database of the data cache module stores the full amount of user data, and the second-level cache database stores the business data. After the cache management module loads the business data cache, it manages the request code according to the business access characteristics, schedules and manages the access to the business data, and eliminates and refreshes the data cache. When a user needs to query business data, when checking the user data in the first-level cache database, it carries a request to query the business data. The first-level cache database checks the request and returns a request code at the same time. When a business data query request is made, it carries the request code and routes the request to the second-level cache database, directly feedbacks the business data corresponding to the request code, and at the same time, the request code in the second-level cache database is invalidated. By introducing a dual-cache data loading and elimination mechanism, the latest business data is guaranteed in the second-level cache database. The preloading of business data in the second-level cache database mainly depends on the generation and invalidation of the request code. Specifically, when an external application initiates a business query request, a request code is generated and fed back to the external application, and the physical disk database is notified to preload business data into the second-level cache database according to the request code queue. When the external application carries the request code to query, the second-level cache database feeds back the business data.
[0193] In this way, an access request from the target system to the target data can be received. In the case where the access request is a first access request to the target user data, in response to the first access request, the target user data can be determined from the first-level cache database, and the target system can be allowed to access the target user data; in the case where the access request is a second access request to the target business data, in response to the second access request, a target request code can be generated based on the target user data cached in the first-level cache database, and the target request code can be sent to the target system and the disk physical database, and then, based on the target request code, the target business data in the disk physical database can be loaded into the second-level cache database, and the target request code can be loaded into the request code validity queue of the second-level cache database to make the target request code valid, and then, upon receiving the target request code sent by the target system, the target system can be allowed to access the target business data. That is to say, the first-level cache database can cache user data, and the second-level cache database can load the data that the target system needs to access from the disk physical database according to the access request. When only user data is accessed, it can be searched only from the user data cached in the first-level cache database. When user data is accessed, it can be searched from the business data cached in the second-level cache database. Therefore, the hit rate of cached data can be improved, thereby improving data access efficiency, and avoiding the problem of frequent access to the disk database reducing system performance when receiving massive access requests.
[0194] Based on the same inventive concept, the embodiment of the present application also provides a data access device. The data access method provided in the embodiment of the present application can be executed by a data access device. Figure 10 The data access device provided in the embodiment of the present application is described in detail.
[0195] Figure 10 A structural diagram of a data access device provided by an embodiment of the present application is shown.
[0196] like Figure 10 As shown, the data access device may include:
[0197] Receiving module 1001, used to receive a request from a target system to access target data;
[0198] A first determining module 1002 is configured to, when the access request is a first access request for target user data, determine the target user data from the first-level cache database in response to the first access request;
[0199] A first processing module 1003 is configured to allow a target system to access target user data;
[0200] The second determining module 1004 is configured to, if the access request is a second access request for target business data, generate a target request code based on the target user data cached in the first-level cache database in response to the second access request, and send the target request code to the target system and the disk physical database;
[0201] The first loading module 1005 is used to load the target business data in the disk physical database into the secondary cache database according to the target request code, and load the target request code into the request code validation queue of the secondary cache database to validate the target request code;
[0202] The second processing module 1006 is configured to allow the target system to access the target business data upon receiving the target request code sent by the target system;
[0203] Among them, the first-level cache database is used to cache user data, the second-level cache database is used to cache business data, and the disk physical database is used to store user data and business data.
[0204] In this way, an access request from the target system to the target data can be received. In the case where the access request is a first access request to the target user data, in response to the first access request, the target user data can be determined from the first-level cache database, and the target system can be allowed to access the target user data; in the case where the access request is a second access request to the target business data, in response to the second access request, a target request code can be generated based on the target user data cached in the first-level cache database, and the target request code can be sent to the target system and the disk physical database, and then, based on the target request code, the target business data in the disk physical database can be loaded into the second-level cache database, and the target request code can be loaded into the request code validity queue of the second-level cache database to make the target request code valid, and then, upon receiving the target request code sent by the target system, the target system can be allowed to access the target business data. That is to say, the first-level cache database can cache user data, and the second-level cache database can load the data that the target system needs to access from the disk physical database according to the access request. When only user data is accessed, it can be searched only from the user data cached in the first-level cache database. When user data is accessed, it can be searched from the business data cached in the second-level cache database. Therefore, the hit rate of cached data can be improved, thereby improving data access efficiency, and avoiding the problem of frequent access to the disk database reducing system performance when receiving massive access requests.
[0205] In some implementations, in order to modify the target user data more securely and efficiently, the first processing module 1003 may include:
[0206] a first modification submodule for, in response to a first access request for modifying target user data and if the target system identity authentication is passed, clearing the target user data in the first-level cache database and modifying the target user data in the disk physical database according to the modification data carried in the first access request to obtain modified target user data;
[0207] The first loading submodule is used to load the modified target user data from the disk physical database into the first-level cache database.
[0208] In some implementations, in order to modify the target service data more securely and efficiently, the second processing module 1006 may include:
[0209] a second modification submodule for, in response to the second access request, when the second access request is a request to modify the target business data and when the target system identity authentication is passed, clearing the target business data in the secondary cache database and modifying the target business data in the disk physical database to obtain the modified target business data;
[0210] The second loading submodule is used to load the modified target business data from the disk physical database into the secondary cache database.
[0211] In some embodiments, in order to facilitate storing the target data in the database or deleting the target data from the database, the data access device may further include:
[0212] an adding module for, when the target data includes target user data and target business data, after receiving an access request for the target data from the target system, and when the access request is a request to add the target data, adding the target user data and the target business data to the disk physical database in response to the access request;
[0213] The second loading module is used to load the target user data from the disk physical database into the first-level cache database;
[0214] The deletion module is used to delete the target user data in the first-level cache database and delete the target user data and target business data in the disk physical database in response to the access request when the access request is a request to delete the target data.
[0215] In some implementations, in order to perform access tasks more reasonably and orderly, the data access device may further include:
[0216] a first insertion module configured to, after generating a target request code based on target user data cached in the first-level cache database, insert the target request code into a request code queue in the first-level cache database in descending order of generation time of the target request code;
[0217] A third determining module is used to determine the queue sequence number of the target request code according to the order of the target request code in the request code queue;
[0218] The fourth determining module is used to determine the target business data corresponding to the target request code from the disk physical database according to the queue sequence number of the target request code.
[0219] In some embodiments, in order to improve data security, the data access device may further include:
[0220] a second insertion module configured to, upon receiving a target request code sent by a target system and allowing the target system to access target business data, clear the target request code from a request code validation queue and insert the target request code into a request code invalidation queue in a secondary cache database to invalidate the target request code;
[0221] A sending module, used for periodically sending the request codes in the request code invalidation queue to the first-level cache database;
[0222] The first clearing module is configured to clear the request code in the request code queue that is the same as the request code in the request code invalidation queue.
[0223] In some embodiments, in order to further improve data security, the data access device may further include:
[0224] The second clearing module is used for the target request code to carry the expiration time of the target request code. After generating the target request code based on the target user data cached in the first-level cache database, when the expiration time of the target request code is reached, the target request code in the request code queue and the request code validity queue is cleared to make the target request code invalid.
[0225] In some embodiments, in order to improve the accuracy of data access, the data access device may further include:
[0226] a generating module configured to generate a first request code based on the target user data cached in the first-level cache database in response to a third access request for target business data before receiving an access request for target data from the target system;
[0227] The third clearing module is used to, after receiving an access request to the target data from the target system, clear the first request code in the request code queue and the request code validity queue in response to the second access request, thereby invalidating the first request code, if the access request is a second access request to the target business data and the first request code has not expired.
[0228] In some implementations, to avoid insufficient storage space in the secondary cache database, the first loading module 1005 may include:
[0229] A deletion submodule is used to delete the second request code inserted earliest in the request code validity queue when the number of request codes in the request code validity queue reaches a quantity threshold, and release the business data corresponding to the second request code;
[0230] The third loading submodule is used to load the target business data in the disk physical database into the secondary cache database, and load the target request code into the request code validation queue to make the target request code valid.
[0231] Figure 11 A schematic structural diagram of an electronic device provided by an embodiment of the present application is shown.
[0232] like Figure 11 As shown in FIG, the electronic device 11 is a block diagram of an exemplary hardware architecture of an electronic device capable of implementing the data access method and data access device according to the embodiments of the present application. The electronic device may refer to the electronic device in the embodiments of the present application.
[0233] The electronic device 11 may include a processor 1101 and a memory 1102 storing computer program instructions.
[0234] Specifically, the processor 1101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0235] The memory 1102 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 1102 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 1102 may include removable or non-removable (or fixed) media. Where appropriate, the memory 1102 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1102 is a non-volatile solid-state memory. In a specific embodiment, the memory 1102 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, typically, memory 1102 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software comprising computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to a method according to an aspect of the present application.
[0236] The processor 1101 implements any one of the data access methods in the above embodiments by reading and executing computer program instructions stored in the memory 1102 .
[0237] In one example, the electronic device may further include a communication interface 1103 and a bus 1104. Figure 11 As shown, the processor 1101 , the memory 1102 , and the communication interface 1103 are connected via a bus 1104 and communicate with each other.
[0238] The communication interface 1103 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0239] Bus 1104 comprises hardware, software or both, couples the parts of electronic equipment to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus 1104 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.
[0240] The electronic device can execute the data access method in the embodiment of the present application, thereby realizing the combination Figures 1 to 10 Described is a data access method and apparatus.
[0241] In addition, in combination with the data access method in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data access methods in the above embodiments is implemented.
[0242] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.
[0243] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0244] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0245] Aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed via the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or the flowchart and the combination of the boxes in the block diagram and / or the flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.
[0246] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.
Claims
1. A data access method, characterized in that: include: Receiving a request from a target system to access target data; In a case where the access request is a first access request for target user data, determining the target user data from a first-level cache database in response to the first access request; allowing the target system to access the target user data; In a case where the access request is a second access request for target business data, in response to the second access request, generating a target request code based on the target user data cached in the first-level cache database, and sending the target request code to the target system and the disk physical database; According to the target request code, the target business data in the disk physical database is loaded into the secondary cache database, and the target request code is loaded into the request code validation queue of the secondary cache database to validate the target request code; Upon receiving the target request code sent by the target system, allowing the target system to access the target service data; The first-level cache database is used to cache user data, the second-level cache database is used to cache business data, and the disk physical database is used to store user data and business data.
2. The method according to claim 1, characterized in that In a case where the first access request is a request to modify the target user data, allowing the target system to access the target user data includes: In response to the first access request, if the target system identity authentication passes, clearing the target user data in the first-level cache database, and modifying the target user data in the disk physical database according to the modification data carried in the first access request to obtain modified target user data; The modified target user data is loaded from the disk physical database into the first-level cache database.
3. The method according to claim 1, characterized in that In a case where the second access request is a request to modify the target business data, allowing the target system to access the target business data includes: In response to the second access request, if the target system identity authentication is passed, clearing the target business data in the secondary cache database and modifying the target business data in the disk physical database to obtain modified target business data; The modified target business data is loaded from the disk physical database into the secondary cache database.
4. The method according to claim 1, wherein In a case where the target data includes the target user data and the target service data, after receiving the access request to the target data from the target system, the method further includes: In a case where the access request is a request to add target data, in response to the access request, adding the target user data and the target business data to the disk physical database; Loading the target user data from the disk physical database into the first-level cache database; In the case where the access request is a request to delete the target data, in response to the access request, the target user data in the first-level cache database is deleted, and the target user data and the target business data in the disk physical database are deleted.
5. The method according to claim 1, characterized in that After generating a target request code according to the target user data cached in the first-level cache database, the method further includes: Inserting the target request codes into the request code queue in the first-level cache database in descending order of the generation time of the target request codes; Determine the queue sequence number of the target request code according to the order of the target request code in the request code queue; According to the queue sequence number of the target request code, the target service data corresponding to the target request code is determined from the disk physical database.
6. The method according to claim 5, characterized in that Upon receiving the target request code sent by the target system, after allowing the target system to access the target service data, the method further includes: Clearing the target request code from the request code validation queue and inserting the target request code into the request code invalidation queue in the secondary cache database to invalidate the target request code; Periodically sending the request codes in the request code invalidation queue to the first-level cache database; Request codes included in the request code queue and identical to the request codes in the request code invalidation queue are cleared.
7. The method according to claim 5, characterized in that The target request code carries an expiration time of the target request code. After generating the target request code according to the target user data cached in the first-level cache database, the method further includes: When the expiration time of the target request code is reached, the target request code in the request code queue and the request code validity queue is cleared to make the target request code invalid.
8. The method according to claim 1, characterized in that Before receiving the access request to the target data from the target system, the method further includes: upon receiving a third access request for target service data, generating a first request code according to the target user data cached in the first-level cache database in response to the third access request; After receiving the access request to the target data from the target system, the method further includes: When the access request is a second access request for target business data and the first request code has not expired, in response to the second access request, the first request code in the request code queue and the request code validity queue is cleared to invalidate the first request code.
9. The method according to claim 1, characterized in that The step of loading the target business data in the disk physical database into the secondary cache database according to the target request code, and loading the target request code into a request code validation queue of the secondary cache database to validate the target request code includes: When the number of request codes in the request code validity queue reaches a quantity threshold, deleting the second request code inserted earliest in the request code validity queue and releasing the service data corresponding to the second request code; The target business data in the disk physical database is loaded into the secondary cache database, and the target request code is loaded into the request code validation queue to validate the target request code.
10. A data access device, characterized in that: The device comprises: A receiving module, configured to receive a request from a target system to access target data; a first determining module configured to, when the access request is a first access request for target user data, determine the target user data from a first-level cache database in response to the first access request; A first processing module, configured to allow the target system to access the target user data; a second determining module configured to, if the access request is a second access request for target business data, generate a target request code based on the target user data cached in the first-level cache database in response to the second access request, and send the target request code to the target system and the disk physical database; a first loading module, configured to load the target business data in the disk physical database into a secondary cache database according to the target request code, and load the target request code into a request code validation queue of the secondary cache database to validate the target request code; a second processing module, configured to allow the target system to access the target service data upon receiving the target request code sent by the target system; The first-level cache database is used to cache user data, the second-level cache database is used to cache business data, and the disk physical database is used to store user data and business data.
11. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the data access method according to any one of claims 1 to 9 is implemented.
12. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, which, when executed by a processor, implement the data access method according to any one of claims 1 to 9.
13. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data access method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Cache management method and device and computer readable storage medium
CN108509586A
Software architecture for capability and quality-of-service negotiations and session establishment for distributed multimedia applications
CN1729672A