Data storage method and apparatus

Abstract

The application discloses a data storage method and device, and relates to the technical field of information security; wherein the method comprises the following steps: receiving data results sent by a database server according to a data acquisition request, and a first access time when a data identifier in the data acquisition request is accessed last time; calculating a first time length difference value between a first sending time when the data acquisition request is sent to the database server and the first access time; when the first time length difference value is less than a first preset time length, generating a data record corresponding to the data identifier according to the data identifier, the data result corresponding to the data identifier, an initial data heat degree of the data identifier pre-configured, and the first sending time; storing the data record into a Redis server, so that when the data result corresponding to the data identifier is acquired again, the data result is acquired from the Redis server, and the data record corresponding to the data identifier is updated. The application can meet the storage of large data volume, and improve the processing efficiency of data.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to data storage methods and apparatus. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] In banking systems, transactions are processed on an account-by-account basis, and large banks often have a huge number of existing accounts and corresponding massive amounts of data. This data is uniformly stored in the bank's database, using identical storage devices and conditions. However, the usage frequency of different accounts varies greatly; some accounts are used frequently, while others haven't been used for years. If banks adopt a uniform management approach and low-performance data storage methods, they cannot meet the usage needs of high-frequency accounts, leading to transaction timeouts or delays. On the other hand, using high-performance data storage methods, given the massive amounts of data, would result in costs increasing exponentially. Current banking data storage methods cannot simultaneously satisfy both data processing efficiency and large-volume data storage requirements. Summary of the Invention

[0004] This invention provides a data storage method applied to an application server to improve data processing efficiency while meeting the requirements for storing large amounts of data. The method includes:

[0005] If the data result corresponding to the data identifier in the data retrieval request is not found in the local memory of the application server or the Redis server, a data retrieval request is sent to the database server.

[0006] Receive the data result sent by the database server based on the data acquisition request, and the first access time of the most recent access of the data identifier;

[0007] Calculate the first duration difference between the first time the data retrieval request is sent to the database server and the first time the data is accessed.

[0008] When the first duration difference is less than the first preset duration, a data record corresponding to the data identifier is generated based on the data identifier, the data result corresponding to the data identifier, the initial data popularity of the pre-configured data identifier, and the first sending time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time;

[0009] Data records are stored in a Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server.

[0010] This invention also provides a data storage device for use in an application server, which improves data processing efficiency while meeting the requirements for storing large amounts of data. The device includes:

[0011] The first processing module is used to send a data retrieval request to the database server when the data result corresponding to the data identifier in the data retrieval request is not found in the local memory of the application server or the Redis server; wherein the data retrieval request carries the data identifier to be retrieved;

[0012] The second processing module is used to receive the data result sent by the database server according to the data acquisition request, and the first access time of the most recent access of the data identifier;

[0013] The third processing module is used to calculate the first duration difference between the first sending time and the first access time of sending a data retrieval request to the database server.

[0014] The fourth processing module is used to generate a data record corresponding to the data identifier based on the data identifier, the data result corresponding to the data identifier, the initial data popularity of the pre-configured data identifier, and the first sending time when the first time difference is less than the first preset time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time;

[0015] The fifth processing module is used to store data records in the Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server.

[0016] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described data storage method.

[0017] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data storage method.

[0018] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described data storage method.

[0019] In this embodiment of the invention, when no data result corresponding to the data identifier in the data retrieval request is found in the local memory of the application server or in the Redis server, a data retrieval request is sent to the database server; the data result sent by the database server according to the data retrieval request and the first access time of the most recent access of the data identifier are received; a first duration difference between the first sending time of sending the data retrieval request to the database server and the first access time are calculated; when the first duration difference is less than a first preset duration, a data record corresponding to the data identifier is generated according to the data identifier, the data result corresponding to the data identifier, the pre-configured initial data popularity of the data identifier, and the first sending time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time; the data record is stored in the Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server. In this way, data results can be stored in different storage locations based on their access frequency. Since the data read and write efficiency varies in different storage locations, data processing efficiency can be improved while meeting the storage requirements for large amounts of data. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0021] Figure 1 This is a flowchart of a data storage method provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the server architecture used in the storage process of a data storage method provided in an embodiment of the present invention;

[0023] Figure 3 This is an example diagram illustrating the logical features of an application server, a Redis server, and a database server used for storage during the storage process of a data storage method provided in an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of a data storage device provided in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0027] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0028] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] In the description of this specification, the terms "comprising," "including," "having," and "containing" are open-ended terms, meaning that they include but are not limited to. The terms "an embodiment," "a specific embodiment," "some embodiments," and "for example," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. The order of steps involved in the various embodiments is used to illustrate the implementation of this application, and the order of steps is not limited and can be adjusted appropriately as needed.

[0030] Research has revealed that in banking systems, transactions are processed on an account-by-account basis, and large banks often have a vast number of existing accounts and corresponding massive amounts of data. This data is uniformly stored in the bank's database, using identical storage devices and conditions. However, the usage frequency of different accounts varies greatly; some accounts are used frequently, while others haven't been used for years. If banks adopt a uniform management approach and low-performance data storage methods, they cannot meet the needs of high-frequency accounts, leading to transaction timeouts or delays. Conversely, using high-performance data storage methods, given the sheer volume of data, would result in exponentially increased costs. For example:

[0031] Currently, some systems logically use a traditional database, but physically shard the database using servers with different configurations. For example, high-performance servers are used for high-speed sharding, and low-performance servers for low-speed sharding. When account usage frequency changes significantly, there is a mechanism to adjust the data across different shards. However, migrating data between different shards within the same database can easily lead to new problems. For instance, partial data backup or recovery may become impossible (because the entire database changes, requiring a full backup / recovery, rather than partial backup / recovery by region, which would result in data corruption). In distributed systems, using fast and slow database partitioning can also lead to high-frequency accounts being aggregated on the same server for processing, which, while improving data storage performance, reduces subsequent data processing performance.

[0032] Some systems employ a two-tier storage mechanism: a Redis server + a database server, or an application server's local memory + a database server. The performance improvement offered by this two-tier storage mechanism is limited, and the optimization methods are rather rigid.

[0033] Therefore, current bank data storage methods cannot simultaneously meet the requirements of data processing efficiency and large-scale data storage.

[0034] In response to the above research, embodiments of the present invention provide a data storage method applied to an application server, such as... Figure 1 As shown, it includes:

[0035] S101: When the data result corresponding to the data identifier in the data retrieval request is not found in the local memory of the application server or the Redis server, a data retrieval request is sent to the database server.

[0036] S102: Receive the data result sent by the database server according to the data acquisition request, and the first access time of the most recent access of the data identifier;

[0037] S103: Calculate the first duration difference between the first sending time and the first access time of sending a data retrieval request to the database server;

[0038] S104: When the first duration difference is less than the first preset duration, generate a data record corresponding to the data identifier based on the data identifier, the data result corresponding to the data identifier, the initial data popularity of the pre-configured data identifier, and the first transmission time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first transmission time;

[0039] S105: Store the data record in the Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server.

[0040] In this embodiment of the invention, when no data result corresponding to the data identifier in the data retrieval request is found in the local memory of the application server or in the Redis server, a data retrieval request is sent to the database server; the data result sent by the database server according to the data retrieval request and the first access time of the most recent access of the data identifier are received; a first duration difference between the first sending time of sending the data retrieval request to the database server and the first access time are calculated; when the first duration difference is less than a first preset duration, a data record corresponding to the data identifier is generated according to the data identifier, the data result corresponding to the data identifier, the pre-configured initial data popularity of the data identifier, and the first sending time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time; the data record is stored in the Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server. In this way, data results can be stored in different storage locations based on their access frequency. Since the data read and write efficiency varies in different storage locations, data processing efficiency can be improved while meeting the storage requirements for large amounts of data.

[0041] The above data storage method will be explained in detail below.

[0042] like Figure 2 The diagram shown is a schematic of the server architecture used in the storage process of a data storage method provided in an embodiment of the present invention. The data storage method of the present invention may use multiple distributed application servers, at least one remote dictionary Redis server, and at least one database (DB) server when storing data.

[0043] Traditional databases (such as MySQL, Oracle, and DB2) offer large storage capacities, typically reaching tens or even billions of records, but their access speeds are slow, leading to even lower overall system efficiency under frequent read / write operations. Redis servers, on the other hand, offer substantial storage capacity, typically reaching hundreds of thousands to tens of millions of records. While more expensive than traditional databases, their efficiency is far superior under frequent read / write operations. Application server local memory has very limited storage capacity and is difficult to expand. It is primarily used for application execution, with only a few hundred to a few thousand records available for data storage. However, its execution and modification speeds are high.

[0044] Specifically, such as Figure 3 The diagram illustrates the logical features of a data storage method according to an embodiment of the present invention, which utilizes an application server, a Redis server, and a database server for storage. The database (DB) server holds all data, and its records include the last access time. If active data is found in the database server, it is temporarily stored in the Redis server. The Redis server uses a key-value storage method, where the key is the access key (e.g., a data identifier), and the value includes, for example, the data result corresponding to the data identifier in the data record, the data popularity of the data identifier, and the first sending time. This can be encoded as "data result corresponding to the data identifier - first sending time (i.e., updated to the last processing time each time) - data popularity - sampling time (i.e., the last processing time)" before storage. Batch processing data and more active hot data within the active data need to be locked and read into the server's local memory for storage. Since there are multiple distributed application servers, and each can only access its own local memory, the application server's local memory is divided into two areas: a shared data area and a dedicated data area. The shared data area allows other application servers to retrieve the same data from Redis (generally used for read-only operations such as fast lookup of customer information), while the exclusive data area stores data that cannot be accessed by other application servers at the same time (generally used for batch processing or peak transactions involving account activity).

[0045] Specifically, when the application server receives a data retrieval request, it searches for the corresponding data result in its local memory. If the data result is not found in the application server's local memory, it accesses the Redis server to retrieve the data result corresponding to the data retrieval request. If the data result corresponding to the data identifier in the data retrieval request is not found in either the application server's local memory or the Redis server, a data retrieval request is sent to the database server.

[0046] Here, data identifiers may include, for example, the user's bank account number, other user identifiers at the bank, etc.

[0047] In one embodiment of the present invention, the application server local memory access time is approximately 100ns, the Redis server access time is approximately 70μs, and the database access time is approximately 0.2-2ms. The difference in magnitude means that even in the worst case scenario of performing three data lookups, the overall efficiency will not be much lower than that of only looking up the database.

[0048] The system receives the data result sent by the database server based on the data retrieval request, and the first access time of the most recent access to the data identifier. It then calculates a first duration difference between the first sending time of the data retrieval request and the first access time. Based on the first duration difference and a first preset duration, it determines whether the data identifier is frequently accessed. Therefore, when the first duration difference is less than the first preset duration, it generates a data record corresponding to the data identifier based on the data identifier, the corresponding data result, the pre-configured initial data popularity of the data identifier, and the first sending time. This data record is stored in a Redis server for future retrieval of the data result corresponding to the data identifier. When the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and the data popularity in the updated data record is greater than a preset popularity, the data identifier and the corresponding data result are stored in the application server.

[0049] Here, the initial data popularity can be set according to the actual application scenario. The initial data popularity of each data identifier can be set to the same value, or it can be set to different values ​​according to the importance of the data results corresponding to the data identifier.

[0050] In addition, the first preset duration is used to evaluate whether the time interval between accesses of the data identifier meets the conditions for hot data identifier. The first preset duration can be set according to the actual application scenario.

[0051] For example, the login timeout for online banking and mobile banking is generally 15 minutes. Therefore, any operation performed within 15 minutes is considered active data, so the first preset timeout can be set to 15 minutes.

[0052] In this embodiment, the data results corresponding to frequently accessed data identifiers are stored in a Redis server. When the data results corresponding to the data identifiers are retrieved again, the data results can be retrieved from the Redis server, thus improving the efficiency of finding the data results corresponding to the data identifiers.

[0053] Furthermore, the data records are updated each time a data identifier is accessed. For example, the data popularity and the first sending time (updated to the access time of this data identifier, i.e., the time when the application server sent the data retrieval request for that data identifier to the Redis server) are updated. Therefore, based on the data records corresponding to the data identifiers, the access status of the data identifiers can be determined in real time. This allows for a decision on whether to continue storing the data results corresponding to the data identifiers in the Redis server, whether to further store the data results corresponding to the data identifiers in the application server's local memory, or whether to remove them from the application server's local memory. This avoids long-term invalid occupation of the Redis server or the application server's local memory, preventing resource waste, and ensures that the data results of frequently accessed hot data identifiers are stored in the application server's local storage, improving the retrieval efficiency of the data results corresponding to the data identifiers.

[0054] In one embodiment of the present invention, when the data result corresponding to the data identifier is obtained again, updating the data record corresponding to the data identifier includes: when the data result corresponding to the data identifier is obtained again, sending a data acquisition request to the Redis server; determining the updated data popularity based on the first sending time, the second sending time of sending the data acquisition request to the Redis server, and the first preset duration; obtaining the updated data record based on the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time; wherein the updated data record includes the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time.

[0055] In one embodiment of the present invention, the updated data popularity is determined based on a first sending time, a second sending time for sending a data retrieval request to the Redis server, and a first preset duration. This includes, for example, calculating a second duration difference between the second sending time and the first sending time; when the second duration difference is greater than the first preset duration, determining the updated access popularity using the following formula: Updated access popularity = Access popularity before update × (Second sending time - First preset duration) ÷ (Second sending time - First sending time) + 1; When the second duration difference is not greater than the first preset duration, determining the updated access popularity using the following formula: Updated access popularity = Access popularity before update + 1.

[0056] In one embodiment of the present invention, when the data popularity in the updated data record is greater than a preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server. For example, this includes storing the data identifier and the data result corresponding to the data identifier in the shared data area of ​​the application server when the data popularity in the updated data record is greater than the preset popularity.

[0057] Furthermore, to further improve data storage efficiency and facilitate batch data processing or peak transaction processing, in another embodiment of the present invention, the data record further includes: a distribution processing identifier field and a lock identifier field; it also includes: when a data acquisition request occurs during batch data processing or peak transaction, setting the distribution processing identifier field and the lock identifier field in the data record to lock identifiers; storing the data record with all distribution processing identifier fields and lock identifier fields set to lock identifiers in the application server's exclusive data area; and when the data results stored in the data record in the exclusive data area are used up, setting the distribution processing identifier field and the lock identifier field in the data record in the exclusive data area to unlock identifiers, and then storing the data record in the exclusive data area in the Redis server.

[0058] For example, the content of the data record in the Value field of the Redis server is "data result corresponding to data identifier - lock identifier field - distribution processing identifier field - last access time of data identifier - data popularity - sampling time (i.e., the last access time of data identifier)". The content of the data record of data identifier A in the Value field is "[Data]|N|N|20210716101803589|2|20210716101803589". When the data acquisition request occurs during batch data processing or transaction peak, |N|N| is set to |Y|Y|.

[0059] In another embodiment of the present invention, when a data modification instruction corresponding to a data identifier is received, the lock identifier field in the data record corresponding to the data identifier in the Redis server is set to the lock identifier. For example, "[Data]|N|N|20210716101803589|2|20210716101803589" is set to "[Data]|Y|N|20210716101803589|2|20210716101803 589”; After modifying the data result corresponding to the data identifier, set the lock flag field in the data record corresponding to the data identifier in the Redis server to the unlock flag. For example, set “[Data]|Y|N|20210716101803589|2|20210716101803589” to “[Data]|N|N|20210716101803589|2|20210716101803589”.

[0060] Furthermore, in one embodiment of the present invention, when data processing is completed and synchronization is finished, the following priority is followed, for example:

[0061] I. When the Redis server is locked, synchronization is based on the data at the Redis server endpoint;

[0062] II. When the Redis server is not locked, synchronization is based on the data from the database server;

[0063] III. After the application server cache processing is complete, update the Redis server results and unlock it;

[0064] IV. If the application server cache processing fails, the transaction must be rolled back (all changes to the transaction must be undone) and the Redis server must be unlocked.

[0065] In addition, to further avoid wasting storage space, one embodiment of the present invention further includes: at a preset time, calculating a third duration difference between the second access time of the most recent access of each data record and the preset time; and clearing data records in the local memory of the Redis server and the application server whose third duration difference is greater than the second preset duration.

[0066] In another embodiment of the present invention, the method further includes: when the number of data acquisition requests exceeds a preset threshold, sorting the data records in the application server and the Redis server according to the second access time of the most recent access to the data record, and deleting data records that are not within the preset sorting range.

[0067] This invention also provides a data storage device, as described in the following embodiments. Since the principle by which this device solves the problem is similar to that of the data storage method, the implementation of this device can be referred to the implementation of the data storage method, and repeated details will not be elaborated further.

[0068] like Figure 4 The diagram shown is a schematic representation of a data storage device provided in an embodiment of the present invention, applied to an application server, and includes:

[0069] The first processing module 401 is used to send a data retrieval request to the database server when the data result corresponding to the data identifier in the data retrieval request is not found in the local memory of the application server or the Redis server; wherein the data retrieval request carries the data identifier to be retrieved;

[0070] The second processing module 402 is used to receive the data result sent by the database server according to the data acquisition request, and the first access time of the most recent access of the data identifier;

[0071] The third processing module 403 is used to calculate the first duration difference between the first sending time and the first access time of sending a data acquisition request to the database server.

[0072] The fourth processing module 404 is used to generate a data record corresponding to the data identifier based on the data identifier, the data result corresponding to the data identifier, the initial data popularity of the pre-configured data identifier, and the first sending time when the first time difference is less than the first preset time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time;

[0073] The fifth processing module 405 is used to store data records in a Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server.

[0074] In one possible implementation, it further includes: a lookup module, configured to look up the data result corresponding to the data request from the local memory of the application server according to the data acquisition request; and if the data result is not found in the local memory of the application server, access the Redis server to look up the data result corresponding to the data acquisition request.

[0075] In one possible implementation, the fifth processing module is specifically used to send a data retrieval request to the Redis server when retrieving the data result corresponding to the data identifier; determine the updated data popularity based on the first sending time, the second sending time of sending the data retrieval request to the Redis server, and the first preset duration; and obtain the updated data record based on the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time; wherein the updated data record includes the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time.

[0076] In one possible implementation, the fifth processing module is specifically used to calculate a second duration difference between the second sending time and the first sending time; when the second duration difference is greater than the first preset duration, the updated access popularity is determined using the following formula: Updated access popularity = Access popularity before update × (Second sending time - First preset duration) ÷ (Second sending time - First sending time) + 1; when the second duration difference is not greater than the first preset duration, the updated access popularity is determined using the following formula: Updated access popularity = Access popularity before update + 1.

[0077] In one possible implementation, the fifth processing module is specifically used to store the data identifier and the corresponding data result in the shared data area of ​​the application server when the data popularity in the updated data record is greater than the preset popularity.

[0078] In one possible implementation, the data record further includes a distribution processing identifier field and a lock identifier field; the device further includes a sixth processing module, configured to set the distribution processing identifier field and the lock identifier field in the data record to lock identifiers when a data acquisition request occurs during batch data processing or transaction peaks; store the data record in which both the distribution processing identifier field and the lock identifier field are lock identifiers in the exclusive data area of ​​the application server; and when the data results stored in the data record in the exclusive data area are used up, set the distribution processing identifier field and the lock identifier field in the data record in the exclusive data area to unlock identifiers, and then store the data record in the exclusive data area in the Redis server.

[0079] In one possible implementation, the sixth processing module is further configured to, upon receiving a data modification instruction corresponding to the data identifier, set the lock identifier field in the data record corresponding to the data identifier in the Redis server to a locked identifier; and after modifying the data result corresponding to the data identifier, set the lock identifier field in the data record corresponding to the data identifier in the Redis server to an unlocked identifier.

[0080] In one possible implementation, it further includes: a seventh processing module, used to calculate, at a preset time, a third duration difference between the second access time of the most recent access of each data record and the preset time; and to clear data records in the local memory of the Redis server and the application server whose third duration difference is greater than the second preset duration.

[0081] In one possible implementation, it further includes: an eighth processing module, used to calculate, at a preset time, a third duration difference between the second access time of the most recent access of each data record and the preset time; and to clear data records in the local memory of the Redis server and the application server whose third duration difference is greater than the second preset duration.

[0082] Based on the aforementioned inventive concept, such as Figure 5 As shown, the present invention also proposes a computer device 500, including a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 530, it implements the aforementioned data storage method.

[0083] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data storage method.

[0084] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described data storage method.

[0085] In this embodiment of the invention, when no data result corresponding to the data identifier in the data retrieval request is found in the local memory of the application server or in the Redis server, a data retrieval request is sent to the database server; the data result sent by the database server according to the data retrieval request and the first access time of the most recent access of the data identifier are received; a first duration difference between the first sending time of sending the data retrieval request to the database server and the first access time are calculated; when the first duration difference is less than a first preset duration, a data record corresponding to the data identifier is generated according to the data identifier, the data result corresponding to the data identifier, the pre-configured initial data popularity of the data identifier, and the first sending time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time; the data record is stored in the Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server. In this way, data results can be stored in different storage locations based on their access frequency. Since the data read and write efficiency varies in different storage locations, data processing efficiency can be improved while meeting the storage requirements for large amounts of data.

[0086] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data storage method, characterized in that, Used in application servers, including: If the data result corresponding to the data identifier in the data retrieval request is not found in the local memory of the application server or the Redis server, a data retrieval request is sent to the database server. Receive the data result sent by the database server based on the data acquisition request, and the first access time of the most recent access of the data identifier; Calculate the first duration difference between the first time the data retrieval request is sent to the database server and the first time the data is accessed. When the first duration difference is less than the first preset duration, a data record corresponding to the data identifier is generated based on the data identifier, the data result corresponding to the data identifier, the initial data popularity of the pre-configured data identifier, and the first sending time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time; Data records are stored in a Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server. When retrieving the data result corresponding to the data identifier again, updating the data record corresponding to the data identifier includes: sending a data retrieval request to the Redis server; determining the updated data popularity based on the first sending time, the second sending time of sending the data retrieval request to the Redis server, and the first preset duration; obtaining the updated data record based on the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time; wherein the updated data record contains the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time; Based on the first sending time, the second sending time of the data retrieval request sent to the Redis server, and the first preset duration, the updated data popularity is determined, including: calculating the second duration difference between the second sending time and the first sending time; when the second duration difference is greater than the first preset duration, the updated data popularity is determined using the following formula: Updated data popularity = Data popularity before update. (Second transmission time - first preset duration) (Second sending time - First sending time) + 1; When the second duration difference is not greater than the first preset duration, the updated data popularity is determined by the following formula: Updated data popularity = Data popularity before update + 1.

2. The data storage method as described in claim 1, characterized in that, Before sending the data retrieval request to the database server, if the data result corresponding to the data identifier in the data retrieval request is not found in the application server's local memory or the Redis server, the following steps are also included: Based on the data acquisition request, retrieve the data result corresponding to the data request from the local memory of the application server; If the data result is not found in the local memory of the application server, access the Redis server to retrieve the data corresponding to the request.

3. The data storage method as described in claim 1, characterized in that, When the data popularity in the updated data record exceeds the preset popularity, the data identifier and the corresponding data result are stored in the application server, including: When the data popularity in the updated data record exceeds the preset popularity, the data identifier and the corresponding data result are stored in the shared data area of ​​the application server.

4. The data storage method as described in claim 1, characterized in that, The data record also includes: a distribution processing identifier field and a lock identifier field; Also includes: When a data acquisition request occurs during batch data processing or peak transaction times, set the distribution processing identifier field and the lock identifier field in the data record to the lock identifier. Data records whose distribution processing identifier field and lock identifier field are both set to lock identifier are stored in the application server's dedicated data area; When the data results stored in the exclusive data area are no longer needed, the distribution processing identifier field and the lock identifier field in the exclusive data area are set to unlocked, and then the data record in the exclusive data area is stored in the Redis server.

5. The data storage method as described in claim 4, characterized in that, Also includes: When a data modification instruction corresponding to a data identifier is received, the lock identifier field in the data record corresponding to the data identifier in the Redis server is set to the lock identifier; After modifying the data result corresponding to the data identifier, set the lock flag field in the data record corresponding to the data identifier in the Redis server to the unlock flag.

6. The data storage method as described in claim 1, characterized in that, Also includes: At a preset time, calculate the third duration difference between the second access time of the most recent access to each data record and the preset time. Clear data records in the local memory of the Redis server and application server whose third duration difference is greater than the second preset duration.

7. The data storage method as described in claim 1, characterized in that, Also includes: When the number of data retrieval requests exceeds a preset threshold, the data records in the application server and Redis server are sorted according to the second access time of the most recent access to the data record, and data records that are not within the preset sorting range are deleted.

8. A data storage device, characterized in that, Used in application servers, including: The first processing module is used to send a data retrieval request to the database server when the data result corresponding to the data identifier in the data retrieval request is not found in the local memory of the application server or the Redis server; wherein the data retrieval request carries the data identifier to be retrieved; The second processing module is used to receive the data result sent by the database server according to the data acquisition request, and the first access time of the most recent access of the data identifier; The third processing module is used to calculate the first duration difference between the first sending time and the first access time of sending a data retrieval request to the database server. The fourth processing module is used to generate a data record corresponding to the data identifier based on the data identifier, the data result corresponding to the data identifier, the initial data popularity of the pre-configured data identifier, and the first sending time when the first time difference is less than the first preset time; wherein, the data record contains the data identifier, the data result corresponding to the data identifier, the data popularity of the data identifier, and the first sending time; The fifth processing module is used to store data records in the Redis server so that when the data result corresponding to the data identifier is retrieved again, the data result is retrieved from the Redis server, the data record corresponding to the data identifier is updated, and when the data popularity in the updated data record is greater than the preset popularity, the data identifier and the data result corresponding to the data identifier are stored in the application server. The fifth processing module is specifically used to send a data retrieval request to the Redis server when retrieving the data result corresponding to the data identifier; determine the updated data popularity based on the first sending time, the second sending time of sending the data retrieval request to the Redis server, and the first preset duration; and obtain the updated data record based on the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time; wherein the updated data record includes the data identifier, the data result corresponding to the data identifier, the updated data popularity, and the second sending time. The fifth processing module is specifically used to calculate the second duration difference between the second transmission time and the first transmission time; when the second duration difference is greater than the first preset duration, the updated data popularity is determined using the following formula: Updated data popularity = Data popularity before update. (Second transmission time - first preset duration) (Second sending time - First sending time) + 1; When the second duration difference is not greater than the first preset duration, the updated data popularity is determined by the following formula: Updated data popularity = Data popularity before update + 1.

9. The data storage device as claimed in claim 8, characterized in that, Also includes: The lookup module is used to look up the data result corresponding to the data request from the local memory of the application server based on the data acquisition request; If the data result is not found in the local memory of the application server, access the Redis server to retrieve the data corresponding to the request.

10. The data storage device as claimed in claim 8, characterized in that, The fifth processing module is specifically used to store the data identifier and the corresponding data result in the shared data area of ​​the application server when the data popularity in the updated data record is greater than the preset popularity.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

Images