Data reading method and device, data writing method and device and computer equipment

Abstract

The invention relates to a data reading method and device, a data writing method and device and computer equipment. The reading method comprises the following steps: under the condition that a read request cache of target data is not hit, reading the target data from an original data source; and when it is determined that the concurrent write mark of the target data does not exist in the concurrent read-write table, updating the read target data to a cache. By adopting the method, the cache consistency in a read-write concurrent scene can be ensured.

Description

Technical Field

[0001] The present application relates to the field of computer storage technology, and in particular to a data reading method, a data writing method, a device and a computer device. Background Art

[0002] In modern computer systems, cache is used as a storage acceleration technology to cache copies of data frequently requested by users in high-speed storage media. When reading data, the cache is checked first. If the cache hits, it can be responded directly. If it misses, it is obtained from the original data source and read and updated to the cache. In combination with the cache replacement algorithm, the system performance can be improved. At the same time, after modifying the original data source, there are two cache write refresh methods: write update and write invalidation. Write update means updating the cache immediately after modification, and write invalidation means invalidating the cache and updating it when reading. Write invalidation is more commonly used because of idempotence, no cache disturbance, and no cache inconsistency problems during concurrent writes.

[0003] There are cache consistency issues in concurrent read and write scenarios. For example, if a read request is initiated before a write operation is completed and the read operation returns slowly, the old data will be updated back to the cache, resulting in data inconsistency. Summary of the invention

[0004] Based on this, it is necessary to provide a data reading method, data writing method, device and computer equipment that can ensure cache consistency in read and write concurrent scenarios in response to the above technical problems.

[0005] In a first aspect, the present application provides a data reading method, comprising:

[0006] In the case where the read request cache of the target data does not hit, reading the target data from the original data source;

[0007] When it is determined that the concurrent write mark of the target data does not exist in the concurrent read-write table, the read target data is updated to the cache.

[0008] In a second aspect, the present application also provides a method for writing data, comprising:

[0009] Write the target data into the original data source;

[0010] When it is determined that the concurrent read count of the target data in the concurrent read-write table is greater than 0, a concurrent write mark is recorded in the concurrent read-write table;

[0011] Invalidate the old target data in the cache.

[0012] In a third aspect, the present application further provides a data reading device, comprising:

[0013] A reading module, configured to read the target data from an original data source when a read request cache of the target data does not hit the target data;

[0014] The update module is used to update the read target data to the cache when it is determined that the concurrent write mark of the target data does not exist in the concurrent read-write table.

[0015] In a fourth aspect, the present application further provides a data writing device, the device comprising:

[0016] A writing module, used to write the target data into the original data source;

[0017] A recording module, configured to record a concurrent write mark in the concurrent read-write table when determining that the concurrent read count of the target data in the concurrent read-write table is greater than 0;

[0018] The invalidation module is used to invalidate the old target data in the cache.

[0019] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect and / or the second aspect are implemented.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect and / or the second aspect are implemented.

[0021] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect and / or the second aspect.

[0022] The above-mentioned data reading method, when the read request cache of the target data does not hit, reads the target data from the original data source; when it is determined that the concurrent write mark of the target data does not exist in the concurrent read-write table, the read target data is updated to the cache. This scheme, by checking whether the concurrent write mark of the target data exists in the concurrent read-write table to decide whether to update the cache, can effectively avoid erroneous updates of the cache when concurrent write operations are in progress. Concurrent write operations may cause data inconsistency. If the cache is directly updated without making such a judgment, subsequent requests to read the cache may obtain partially updated or erroneous data. Updating the cache only when it is determined that there is no concurrent write mark, that is, the write operation is stably completed or there is no write operation, helps to ensure the accuracy of the data in the cache and avoid updating old data back to the cache.

[0023] The above-mentioned data writing method writes the target data into the original data source; when it is determined that the read concurrency count of the target data in the concurrent read-write table is greater than 0, the concurrent write mark is recorded in the concurrent read-write table; and the old target data in the cache is invalidated. This solution records the concurrent write mark when the read concurrency count of the target data in the concurrent read-write table is greater than 0, which effectively copes with concurrent read scenarios and allows the system to know the read-write crossover status and handle it reasonably. Furthermore, invalidating the old target data in the cache can maintain the validity of the cached data and prevent read requests from obtaining outdated information. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] FIG1 is a diagram of an application environment in one embodiment;

[0026] FIG2 is a schematic diagram of a flow chart of a data reading method in an embodiment;

[0027] FIG3 is a schematic diagram of a flow chart of a method for reading data in a storage system according to an embodiment;

[0028] FIG4 is a schematic diagram of a flow chart of a method for reading data in a database system according to an embodiment;

[0029] FIG5 is a schematic diagram of a flow chart of a method for writing data in an embodiment;

[0030] FIG6 is a schematic diagram of a flow chart of a method for writing data in a storage system according to an embodiment;

[0031] FIG7 is a schematic diagram of a flow chart of a method for writing data in a database system according to an embodiment;

[0032] FIG8 is a block diagram of a data reading device according to an embodiment;

[0033] FIG9 is a block diagram of a data writing device in one embodiment;

[0034] FIG. 10 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] In modern computer systems, cache is a widely used storage acceleration technology. For data copies that users frequently request, they can be cached in high-speed storage media. In this way, when users access data, they do not need to recalculate or obtain data from the slow original data source, thereby speeding up access. When users read data, they will first query the cache: if the requested data is stored in the cache, this is a cache hit, which can respond immediately and no longer needs to make a request to the slower original data source; if the corresponding data is not found in the cache, that is, a cache miss, then it is necessary to continue to request data from the original data source, and after obtaining the data, update it back to the cache, so that the user's subsequent identical requests can be accelerated. This process is called "read update". Cache read update cooperates with cache replacement algorithms such as LRU, which can gradually accumulate hot data that users frequently access in the cache, thereby improving the overall performance of the system.

[0037] In order to keep the cache consistent with the original data source, after the user modifies the original data source, the cache needs to be refreshed synchronously. There are two cache write refresh methods: "write update" and "write invalidation". First, the cache is updated immediately after the original data is modified, which is "write update". The subsequent read requests will hit the cache, and the performance is high. However, if the written data is not frequently accessed by the user, frequent write update operations will cause cache churn, resulting in reduced cache benefits. Second, after the original data is modified, the cache is not updated immediately, but the cache invalidation operation is performed, which is called "write invalidation". When the data is read again, the cache will miss, which triggers the cache read update after reading the original data. Write invalidation is an idempotent operation. In the case of concurrent writes, there will be no cache inconsistency problems and no cache disturbance, so this method is more commonly used.

[0038] In concurrent read and write scenarios, there is a cache consistency problem. Before the write operation succeeds, if a read request for the same data is initiated at the same time, and the read operation occasionally returns more slowly, the old data will be updated back to the cache, resulting in data inconsistency. In addition, the read-update-write-update mode also has cache consistency issues during concurrent writes.

[0039] Cache consistency issues in concurrent reads and writes mainly occur in two situations: read-write concurrency and write concurrency. Given that write invalidation has the idempotent property, when concurrent writes and cache write invalidation work together, cache inconsistency will not occur.

[0040] Under existing technical conditions, in order to ensure that the cache read-update-write invalidation mode can maintain cache consistency, it is often necessary to use locking or queuing to implement concurrent read and write operations of the same data in serial order, thereby ensuring the atomicity and transactionality of write data and write cache operations, and effectively preventing cache inconsistencies that may be caused by concurrent reads and writes. Some systems also use a cache delayed double deletion mechanism. Specifically, before writing the original data, the cache invalidation operation is first performed. The purpose of this is to avoid the sudden crash of the system service after the write operation is successful, resulting in the failure to update the cache in time; and after the write operation is successful, the cache invalidation operation will be performed again after a short delay to clear the old data that may be inserted during the read update process, further improving the accuracy and consistency of the cached data.

[0041] In the existing technical solutions, the read and write operations for the same data are serialized by locking or queuing. In this way, although it can effectively avoid the situation where old data is written back to the cache when reading and updating, at the same time, the read and write concurrency performance is greatly reduced. As for the delayed double deletion mechanism, the setting of its delay time mainly relies on experience and judgment, which is difficult to accurately estimate. And in actual application scenarios, if there is no concurrent reading of the corresponding data, the second cache invalidation operation is redundant. In addition, before the second cache invalidation operation is executed, the phenomenon of reading old data will still occur, so this mechanism can only guarantee the eventual consistency of the cache, and cannot achieve a stronger consistency standard.

[0042] In order to solve the above problems, the embodiments of the present application provide a data reading method, a data writing method, an apparatus and a computer device, which can ensure cache consistency in a concurrent reading and writing scenario.

[0043] It should be noted that the data reading method and data writing method provided in the embodiments of the present application can be applied to a computer device, and the computer device can be a server. The data reading method and the data writing method can be applied to a storage system in a server, and can also be applied to a database system in a server. The data reading method can also be applied to a data reading device, and the data reading device can be a functional module or functional entity in a server for implementing the data reading method. The data writing method can also be applied to a data writing device, and the data writing device can be a functional module or functional entity in a server for implementing the data writing method.

[0044] Exemplarily, as shown in FIG1 , is an application environment diagram of a method for reading data and a method for writing data. As shown in FIG1 , this is a case where the method for reading data and the method for writing data are applied in a storage system. The read request for reading data and the write request for writing data can usually be obtained through an upper-layer application. The specific method for reading data and the method for writing data are mainly implemented through a storage service. Specifically, they can be implemented through a cache manager in the storage service. During the implementation process, a concurrent read-write table is used to record whether there are concurrent reads and writes, so as to achieve consistency between the cache and the persistence layer. Among them, the upper-layer application can be an application that makes a read request or a write request, and the upper-layer application can be a client in a terminal, or the upper-layer application can be an application in another server. Among them, the above-mentioned persistence layer is the original data source, which can actually be a low-speed mechanical hard disk.

[0045] In an exemplary embodiment, as shown in FIG2 , a flow chart of a method for reading data is provided, and the method includes but is not limited to the following steps:

[0046] 201. When a read request for target data does not hit the cache, read the target data from the original data source.

[0047] When a computer system processes a data read request, it first searches for the target data in the cache (a cache is a storage area that temporarily stores some commonly used data to speed up data access). When the target data cannot be found in the cache, that is, a cache miss occurs, the system needs to turn to the original data source to obtain the target data. The original data source usually refers to the basic location where data is stored, such as a database, which has a relatively slow read and write speed but can store complete data permanently.

[0048] For example, on an e-commerce website, if a user queries for detailed information about a particular product and the product information is not available in the cache (cache miss), the system will read the detailed information about the product from the product database (original data source) to meet the user's query needs.

[0049] 202. Determine whether there is a concurrent write mark for the target data in the concurrent read-write table.

[0050] Among them, the concurrent read-write table is a mechanism for recording the relevant status of data during concurrent operations (multiple threads or processes read and write data at the same time).

[0051] The concurrent write flag is used to indicate that there is a write operation concurrent with the read operation.

[0052] After reading the target data from the original data source, you need to check whether there is a concurrent write mark for the target data in the concurrent read-write table. The concurrent write mark is used to indicate whether other threads or processes are currently writing to the target data. If this mark exists, it means that concurrent writing is occurring and the data may be unstable and being modified; if this mark does not exist, it means that there is no concurrent write operation for the target data and the data is relatively stable.

[0053] For example, in an online document collaboration system, multiple users operate a document at the same time. When a certain part of the document data (target data) needs to be processed later, by checking the corresponding concurrent write mark in the concurrent read-write table, you can know whether anyone else is modifying this part of the data at this moment.

[0054] 203. Update the read target data to the cache.

[0055] When it is determined through step 202 that there is no concurrent write mark of the target data in the concurrent read-write table, that is, there is no concurrent write operation in progress for the target data and its state is relatively stable, the target data previously read from the original data source can be updated to the cache. The advantage of this is that if there is a subsequent read request for the target data, the data can be quickly obtained directly from the cache without having to read from the original data source again, thereby improving the efficiency of data reading.

[0056] For example, in a financial trading system, when querying the real-time price of a certain stock, if the cache does not hit during the first read, after reading from the original data source (such as a market data server), it is found that there is no concurrent write mark, and the price data is updated to the cache. When other users query the stock price again, they can quickly get the result from the cache.

[0057] 204. Do not update the read target data to the cache.

[0058] In contrast to step 203, when it is determined that there is a concurrent write mark for the target data in the concurrent read-write table, it means that the target data is being written by other threads or processes, and the data may change at any time and is in an unstable state. At this time, the target data read from the original data source cannot be updated to the cache, otherwise the data in the cache may conflict with the data being modified, resulting in errors or inconsistencies in the data subsequently read from the cache.

[0059] For example, in an e-commerce system where multiple users update inventory information simultaneously, if old inventory data read from the original data source is hastily updated to the cache while the inventory data is being written (modified), then subsequent users will obtain inaccurate old data from the cache when querying the inventory. Therefore, in this case, it is necessary to give up updating the cache, wait for the concurrent write operation to complete and the data to stabilize, and then determine whether to update the cache based on the corresponding rules.

[0060] The above-mentioned data reading method reads the target data from the original data source when the read request cache of the target data does not hit; when it is determined that the concurrent write mark of the target data does not exist in the concurrent read-write table, the read target data is updated to the cache. This solution can effectively avoid erroneous updates of the cache when concurrent write operations are in progress by checking whether there is a concurrent write mark of the target data in the concurrent read-write table. Concurrent write operations may cause data inconsistency. If the cache is directly updated without making such a judgment, subsequent requests to read the cache may obtain partially updated or erroneous data. Updating the cache only when it is determined that there is no concurrent write mark, that is, the write operation is stably completed or there is no write operation, helps to ensure the accuracy of the data in the cache and avoid updating old data back to the cache.

[0061] In some embodiments, in the above-mentioned data reading method, before reading the target data from the original data source, if the read request cache of the target data does not hit, the read concurrency count of the target data can be increased by 1 in the concurrent read-write table; after reading the target data from the original data source, the read concurrency count of the target data can be reduced by 1 in the concurrent read-write table.

[0062] In the above embodiment, when the target data read request cache misses, the read concurrency count of the target data is added by 1 in the concurrent read-write table, and subtracted by 1 after reading the data. This can accurately track the concurrent read situation, such as accurately recording the changes in the number of read operations in multiple systems, which can be used as a basis for adjusting the cache strategy and ensuring the coordination of concurrent operations. By combining information such as the read concurrency count and the concurrent write mark, the data status can be comprehensively judged, and reasonable strategies can be formulated to maintain the accuracy and stability of the system data, thereby improving the overall system performance and ensuring cache consistency.

[0063] In some embodiments, in the above-mentioned data reading method, in the concurrent read-write table, after the read concurrency count of the target data is reduced by 1, if the read concurrency count is 0, the record of the read concurrency count is deleted.

[0064] In the above embodiment, when the target data read concurrency count is reduced by 1 and becomes 0, its record is deleted, which can optimize storage resource utilization, improve concurrent read and write table query efficiency, simplify system data management complexity, and enhance the system's ability to respond to data status changes, providing multi-faceted support for efficient and stable system operation.

[0065] Exemplarily, as shown in FIG3 , a flowchart of a method for reading data in a storage system is provided. In this method, upper-layer applications can read and write data through a storage service (server), and the data of the storage service is stored in a persistence layer (original data source). To improve the read performance of the storage service, the cache is used to accelerate read requests. The storage service can use the cache manager to schedule cache queries and refreshes in the read and write process. The cache manager maintains a concurrent read and write table to detect concurrent read and write behaviors, avoid reading and updating the cache when reading and writing are concurrent, and ensure cache consistency. The method includes but is not limited to the following steps:

[0066] 301. Cache miss, the storage service notifies the cache manager to prepare to read the persistence layer.

[0067] When the storage service receives a data read request from an upper-layer application, it first searches for the target data in the cache. If it cannot find the target data in the cache (i.e., a cache miss), the storage service sends a notification to the cache manager, informing it that it needs to obtain the data from the persistence layer (such as the original data source such as the database). This is because the cache does not have the required data, so it can only turn to the persistence layer, which stores complete data and has a relatively slow read and write speed, to satisfy the read request.

[0068] 302. The cache manager queries the concurrent records of the current data in the concurrent read-write table, and adds new records if there are no records.

[0069] After receiving the notification from the storage service, the cache manager will first check the concurrent records of the current target data in the concurrent read-write table. The concurrent read-write table is used to record the relevant status information of the data during multi-threaded or multi-process concurrent operations. If no concurrent record of the current data is found in the table, it means that there may be no concurrent operations on the data before. At this time, a new concurrent record of this data needs to be added to accurately track and manage its concurrent read and write status in the future.

[0070] 303. The cache manager modifies the concurrent record of the current data, and the read concurrent count increases by 1.

[0071] After determining the concurrent record status of the current data in the concurrent read-write table, whether it is a new record or an existing record, the read concurrency count needs to be increased by 1. The purpose of this step is to record that a read operation is being initiated for the current target data at this moment. Through the dynamic change of the read concurrency count, it is possible to accurately count how many read operations are currently being performed on the data at the same time, thereby providing a basis for the system to fully understand the concurrent reading status of the data, and also helping to coordinate the relationship between different operations in multiple concurrent operation scenarios and formulate reasonable cache strategies.

[0072] 304. The storage service reads the persistence layer.

[0073] After the cache manager has operated on the concurrent read and write table, the storage service officially reads the target data from the persistence layer. As the original storage location of the data, the persistence layer has a slower read and write speed than the cache, but it can ensure the integrity and persistence of the data. In the case of a cache miss, reading data from the persistence layer is a necessary step to satisfy the read request. Only after obtaining the original data can subsequent cache updates and other operations be performed to improve the processing speed of future read requests for the same data.

[0074] 305. The storage service completes reading the persistence layer and notifies the cache manager to update the cache.

[0075] When the storage service successfully reads the target data from the persistence layer, it sends a notification to the cache manager, indicating that the data reading is complete and the cache update operation can be performed (if the update conditions are met). This notification mechanism ensures that the cache manager can know the data reading status in a timely manner, so as to determine whether to update the read data to the cache according to the established rules, so as to achieve the purpose of using the cache to accelerate subsequent read requests.

[0076] 306. The cache manager checks the concurrent write mark of the current data in the concurrent read-write table. If there is no mark, the cache is updated with the read data.

[0077] After receiving the cache update notification from the storage service, the cache manager will first check whether there is a concurrent write mark for the current data in the concurrent read-write table. The concurrent write mark is used to indicate whether other threads or processes are writing to the current target data. If there is no concurrent write mark, it means that the current data is in a relatively stable state and there is no interference from the ongoing write operation. At this time, the data previously read from the persistence layer can be updated to the cache. Doing so can improve the response speed of subsequent read requests for the same data, because the cache hit speed is usually much faster than the speed of reading data from the persistence layer.

[0078] 307. The cache manager reduces the concurrent read count of the current data in the concurrent read-write table, and deletes the record if the count is reduced to 0.

[0079] After completing the cache update operation (or determining not to update the cache), the cache manager will reduce the read concurrency count of the current data in the concurrent read-write table by 1 to indicate that a previously initiated read operation has ended. If the read concurrency count becomes 0 after the decrement operation, it means that there is no ongoing read operation for the data. At this time, deleting the record of the data in the concurrent read-write table can optimize the utilization of storage resources, reduce redundant data in the concurrent read-write table, improve query efficiency, simplify the complexity of system data management, and enable the system to be more sensitive to changes in data status, providing multi-faceted support for efficient and stable operation of the system.

[0080] Exemplarily, as shown in FIG4 , a flowchart of a method for reading data in a database system is provided, in which the upper-layer application accesses data through an application server, and the data of the application server is stored in a database. Use cache to accelerate database hotspot queries. The application server uses a cache manager to schedule cache queries and updates in the read and write process. The cache manager maintains a concurrent read and write table in memory to check concurrent queries and insertion behaviors, avoid updating the cache during database queries, insertions / modifications, and ensure cache consistency. The method includes but is not limited to the following steps:

[0081] 401, cache miss, the application server notifies the cache manager to prepare to query the database.

[0082] When an upper-layer application initiates a data read request through an application server, the application server first attempts to obtain the target data in the cache. Since a cache miss means that the required data does not exist in the cache, the application server sends a notification to the cache manager, informing it that it needs to perform a data query operation from the database (as the original data source). Because the database stores complete data information, although its query speed is slower than the cache, it can provide comprehensive and accurate data to meet the read request of the upper-layer application.

[0083] 402. The cache manager checks the concurrent records of the current query in the concurrent read-write table, and adds new records if there are no records.

[0084] After receiving the notification from the application server, the cache manager will search for concurrent records of the target data related to the current query in the concurrent read-write table in memory. The concurrent read-write table is used to record the status information of data during concurrent query, insertion, or modification operations. If no concurrent record of the data is found in the table, it means that there may not be any concurrent operations on this data before. At this time, a new concurrent record needs to be created so that the concurrent operation status of the data can be accurately tracked and managed in the future, providing an effective status monitoring basis for the entire data processing process.

[0085] 403. The cache manager modifies the concurrent record of the current data and increases the read concurrency count by 1.

[0086] After determining the concurrent records of the target data in the concurrent read-write table, the read concurrency count must be incremented by 1, regardless of whether it is an existing record or a newly created record. The core significance of this operation is to accurately record the changes in the number of read operations initiated for the target data at the current moment. Through this dynamic count change, the system can fully grasp the concurrent reading of the data, for example, it can understand the popularity of a certain data in a specific time period (the frequency of being read simultaneously), thereby providing a key basis for the subsequent formulation of a reasonable cache strategy. It also helps to coordinate the relationship between different operations in multi-threaded or multi-process concurrent operation scenarios to ensure the overall stability of the system and the consistency of the data.

[0087] 404. The application server queries the database.

[0088] After the cache manager has pre-processed the concurrent read and write tables, the application server officially starts the query operation on the database. As a persistent data repository, the database is not as efficient as the cache in terms of query efficiency, but it can ensure the integrity and persistence of the data and is the ultimate source of data. In the case of a cache miss, querying data from the database is a key step in obtaining the required information. Only when the original data is successfully obtained can it be updated to the cache according to certain rules in subsequent operations, thereby improving the processing efficiency of subsequent read requests for the same data and reducing the pressure of repeated queries on the database.

[0089] 405. The application server completes the database query and notifies the cache manager to update the cache.

[0090] When the application server successfully retrieves the target data from the database, it will promptly send a notification to the cache manager to inform it that the data query task has been completed. At this time, it can consider whether to perform a cache update operation based on the situation. This notification mechanism ensures that the cache manager can obtain data query result information in a timely manner so that it can determine whether to write the newly retrieved data into the cache based on established cache management rules, such as checking concurrent write marks and other conditions, thereby achieving the purpose of using the cache to speed up the processing speed of subsequent read requests for the same data and improving the read performance of the entire database system.

[0091] 406. The cache manager checks the concurrent write mark of the current data in the concurrent read-write table. If there is no mark, the cache is updated with the queried data.

[0092] After receiving the cache update notification from the application server, the cache manager first checks whether there is a concurrent write mark in the concurrent read-write table for the current target data. The concurrent write mark is used to indicate whether other processes or threads are inserting or modifying the data. If there is no concurrent write mark, it means that the current data is in a relatively stable state and there is no interference from ongoing write operations. At this time, it is safe and beneficial to update the data queried from the database to the cache. Because the speed of cache hitting data is much faster than querying the database again, this can significantly improve the response speed of subsequent read requests for the same data, improve user experience, and also help optimize system resource utilization and reduce excessive occupation of database resources.

[0093] 407. The cache manager reduces the read concurrency count of the current data in the concurrent read-write table, and deletes the record if the count is reduced to 0.

[0094] After completing the cache update (or determining not to update the cache), the cache manager will perform a subtraction operation on the read concurrency count of the current data in the concurrent read-write table, which means that a read operation initiated previously has been successfully completed. If the read concurrency count becomes 0 after the subtraction operation, it means that all read operations for the data have been completed. At this time, deleting the record of the data in the concurrent read-write table has many important significances. On the one hand, it can optimize the utilization of memory resources and prevent the concurrent read-write table from occupying too much memory space due to storing a large number of useless read concurrency count records; on the other hand, it can improve the query efficiency of the concurrent read-write table and reduce unnecessary record traversal time; at the same time, it can also simplify the complexity of system data management, so that the system can perceive changes in data status more quickly, so as to adjust relevant strategies in time, and provide all-round and powerful support for the efficient and stable operation of the database system.

[0095] In an exemplary embodiment, as shown in FIG5 , a flow chart of a method for writing data is provided, and the method includes but is not limited to the following steps:

[0096] 501. Write the target data into the original data source.

[0097] When data needs to be updated or inserted, the system will first write the target data to the original data source. The original data source is usually a storage medium such as a database that can store complete data persistently and ensure data integrity and reliability. Although its writing speed may be relatively slow, it is the foundation of the data. All data changes must eventually be reflected in the original data source to ensure the authority and consistency of the data, providing a stable data foundation for subsequent cache operations and data flow of the entire system.

[0098] 502. Determine whether the read concurrency count of the target data in the concurrent read-write table is greater than 0.

[0099] After writing to the original data source, you need to check the concurrent read count of the target data in the concurrent read-write table. The concurrent read-write table is used to record the relevant status information of the data during multi-threaded or multi-process concurrent operations, where the read concurrency count reflects how many read operations are currently being performed on the target data. By determining whether the read concurrency count is greater than 0, you can find out whether other threads or processes are reading the data when the write operation occurs. This judgment is of great guiding significance for subsequent decisions on whether to record concurrent write marks in the concurrent read-write table and how to handle cache, because ongoing read operations may interact with write operations, thereby affecting cache consistency and other issues.

[0100] 503. Record the concurrent write mark in the concurrent read and write table.

[0101] If it is determined in step 502 that the read concurrency count of the target data in the concurrent read-write table is greater than 0, it means that a read operation is performed concurrently with the current write operation. At this time, a concurrent write mark is recorded in the concurrent read-write table, which is used to indicate that there is a write operation concurrent with the read operation. In this way, when other threads or processes perform read operations and query the concurrent read-write table, they will know that the data is being modified by the write operation, so as to take corresponding measures, such as avoiding updating potentially outdated data to the cache, so as to ensure cache consistency, prevent data errors or inconsistencies caused by concurrent reads and writes, and maintain the accuracy and stability of the data state of the entire system.

[0102] 504. Do not record the concurrent write mark in the concurrent read and write table.

[0103] When step 502 determines that the concurrent read count of the target data in the concurrent read-write table is not greater than 0, that is, there is no concurrent read operation with the current write operation, there is no need to record the concurrent write mark in the concurrent read-write table. This is because in the absence of concurrent read operations, write operations are relatively independent and will not cause data conflicts with read operations or affect cache consistency. At this time, the system can continue to perform subsequent processing according to the normal write operation process without additional marking of concurrent write status, reducing unnecessary status recording and processing overhead, and improving system processing efficiency.

[0104] 505. Invalidate the old target data in the cache.

[0105] Regardless of whether the concurrent write mark is recorded in the concurrent read-write table, the old target data in the cache needs to be invalidated. The purpose of this step is to ensure data consistency between the cache and the original data source. When the data in the original data source is updated, the old data in the cache is no longer valid. If it continues to be retained, subsequent read requests may obtain incorrect data. By invalidating the old target data in the cache, subsequent read requests will re-obtain the latest data from the original data source when the cache misses, and then decide whether to update the new data to the cache based on the relevant marks and status in the concurrent read-write table, thereby ensuring the accuracy and timeliness of the cached data in the entire system during the data update process, and avoiding various data consistency issues caused by outdated cached data.

[0106] The above-mentioned data writing method writes the target data into the original data source; when it is determined that the read concurrency count of the target data in the concurrent read-write table is greater than 0, the concurrent write mark is recorded in the concurrent read-write table; and the old target data in the cache is invalidated. This solution records the concurrent write mark when the read concurrency count of the target data in the concurrent read-write table is greater than 0, which effectively copes with concurrent read scenarios and allows the system to know the read-write crossover status and handle it reasonably. Furthermore, invalidating the old target data in the cache can maintain the validity of the cached data and prevent read requests from obtaining outdated information.

[0107] In some embodiments, after the old target data in the cache is invalidated, the concurrent write mark recorded in the concurrent read-write table may be deleted.

[0108] In the above embodiment, after the old data in the cache is invalidated, the concurrent write mark in the concurrent read-write table is deleted, which can effectively release the storage space of the concurrent read-write table, reduce redundant data records, and improve the query efficiency of the table. At the same time, this helps to simplify the system's judgment logic on the concurrent state of data, so that the system can respond to subsequent data operations more quickly, enhance the fluency and efficiency of the data processing process, and further ensure the stability and data consistency of the system in multi-threaded or multi-process concurrent read-write scenarios.

[0109] As shown in FIG6 , a schematic diagram of a method for writing data in a storage system is provided. In the method for writing data, a cache invalidation process is performed after the storage service successfully writes data. The method may include but is not limited to:

[0110] 601. The storage service completes writing data to the persistence layer.

[0111] This step indicates that the data has been successfully written to the persistence layer by the storage service. As the core storage area of ​​system data, such as databases, the persistence layer has a relatively slow write operation, but it ensures the persistence and integrity of the data. This step is a key link in the entire data writing process. Subsequent operations such as cache processing rely on the correct writing of data in the persistence layer, laying the foundation for data consistency and reliability, so that data can flow stably between different modules and operations of the system and maintain its authority.

[0112] 602. The storage service notifies the cache manager to perform cache invalidation.

[0113] When the storage service completes writing data to the persistence layer, it sends a notification to the cache manager, asking it to perform a cache invalidation operation. This is because the data in the persistence layer has been updated, and if the old data continues to exist in the cache, it may cause data inconsistency. By notifying the cache manager to invalidate the corresponding cache, it can ensure that subsequent read requests obtain the latest data from the persistence layer when the cache misses, thereby maintaining the consistency of the cache and persistence layer data.

[0114] 603. The cache manager checks the concurrent read count of the current data in the concurrent read-write table, and if it is greater than 0, a concurrent write mark is added.

[0115] After receiving the notification from the storage service, the cache manager will first check the read concurrency count of the current data in the concurrent read-write table. The concurrent read-write table is used to record the status information of the data during multi-threaded or multi-process concurrent operations. The read concurrency count reflects the current number of read operations on the data. If the read concurrency count is greater than 0, it means that there is a concurrent read operation at the same time as the current write operation. At this time, the concurrent write mark is marked so that other threads or processes can know that the data is in the process of writing by querying the concurrent read-write table when performing a read operation, and then take corresponding measures, such as avoiding updating potentially outdated data to the cache, effectively ensuring the consistency of cached data in concurrent read-write scenarios, and preventing data errors or inconsistencies caused by read-write crossover.

[0116] 604. The cache manager invalidates the data in the cache.

[0117] Regardless of the read concurrency count, the cache manager needs to invalidate the old data in the cache that corresponds to the target data currently written to the persistence layer. This is a necessary step to ensure the consistency of cache and persistence layer data. Once the persistence layer data is updated, the old data in the cache loses its timeliness and validity. After invalidating the data in the cache, if the subsequent read request does not find the data in the cache (cache miss), it will re-obtain the latest data from the persistence layer, and then decide whether to update the new data to the cache based on the mark and status in the concurrent read and write table, thereby ensuring the accuracy and timeliness of the cached data during the data update process of the entire system.

[0118] As shown in FIG7 , a schematic diagram of a method for writing data in a database system is provided. The method for writing data is a cache invalidation process after the application server successfully inserts / updates data in the database insert / update process. The method may include but is not limited to:

[0119] 701. The application server completes inserting / updating data into the database.

[0120] This step indicates that the application server has successfully inserted or updated the data into the database. As the core data repository of the system, the database bears the heavy responsibility of persistently storing data, and its data integrity and reliability are crucial. When this step is completed, the state of the data in the database has changed, and this change will trigger a series of subsequent operations to ensure the consistency of the data in the entire system, especially the data synchronization between the cache and the database.

[0121] 702. The application server notifies the cache manager to execute cache invalidation.

[0122] After the application server completes the data operation on the database, it will send a notification to the cache manager, asking it to perform a cache invalidation operation. Since the data in the database has been updated, if the old data is still retained in the cache, data inconsistency will occur. Through this notification mechanism, the cache manager can know the changes in the database data in a timely manner and handle the cache accordingly. In this way, when subsequent read requests access the cache, if the cache is found to be invalid, the latest data will be obtained from the database, ensuring the consistency of the cache and database data.

[0123] 703. The cache manager checks the concurrent read count of the current data in the concurrent read-write table, and if it is greater than 0, a concurrent write mark is added.

[0124] After receiving the notification from the application server, the cache manager will first check the read concurrency count of the current data in the concurrent read-write table. The concurrent read-write table is used to record the status information of data during multi-threaded or multi-process concurrent operations, where the read concurrency count reflects the number of read operations currently being performed on the data. If the read concurrency count is greater than 0, it means that there is a concurrent read operation occurring at the same time as this database write operation. The purpose of marking the concurrent write at this time is to allow other threads or processes to clearly understand that the data is in the process of a write operation when performing a read operation and querying the concurrent read-write table. This helps them take appropriate measures, such as avoiding updating data that may become outdated due to write operations into the cache, thereby effectively ensuring the consistency of cached data in concurrent read-write scenarios, preventing data errors or inconsistencies caused by read-write conflicts, and maintaining the accuracy and stability of the data status of the entire system.

[0125] 704. The cache manager invalidates the data in the cache.

[0126] Regardless of the read concurrency count, the cache manager must set the old data in the cache corresponding to the target data currently being inserted or updated in the database to an invalid state. This is a key step to ensure the consistency of cache and database data. Once the data in the database changes, the old data in the cache loses its timeliness and validity. When the data in the cache is invalidated, if the subsequent read request does not find valid data in the cache, it will re-obtain the latest data from the database, and then decide whether to update the new data to the cache based on the mark and status in the concurrent read and write table. Through this mechanism, the accuracy and timeliness of the cached data in the entire system during the data update process are guaranteed, avoiding various data consistency issues caused by stale cached data.

[0127] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0128] Based on the same inventive concept, the embodiment of the present application also provides a data reading device for implementing the above-mentioned data reading method, and a data writing device for implementing the above-mentioned data writing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more device embodiments provided below can refer to the limitations in the above-mentioned method embodiments, and will not be repeated here.

[0129] In an exemplary embodiment, as shown in FIG8 , a data reading device is provided, including:

[0130] The reading module 801 is used to read the target data from the original data source when the read request cache of the target data does not hit;

[0131] The updating module 802 is configured to update the read target data to the cache when it is determined that there is no concurrent write mark of the target data in the concurrent read-write table.

[0132] In some embodiments, the update module 802 is further configured to:

[0133] When it is determined that there is a concurrent write mark of the target data in the concurrent read-write table, the read target data is not updated to the cache.

[0134] In some embodiments, the update module 802 is also used to: before the read module 801 reads the target data from the original data source, if the read request cache of the target data does not hit, add 1 to the read concurrency count of the target data in the concurrent read-write table; after the read module 801 reads the target data from the original data source, subtract 1 from the read concurrency count of the target data in the concurrent read-write table.

[0135] In some embodiments, the update module 802 is further used to, after reducing the read concurrency count of the target data by 1, the method further includes: if the read concurrency count is 0, deleting the record of the read concurrency count.

[0136] In an exemplary embodiment, as shown in FIG9 , a data writing device is provided, including:

[0137] Writing module 901, used to write target data into the original data source;

[0138] A recording module 902 is used to record a concurrent write mark in the concurrent read-write table when it is determined that the concurrent read count of the target data in the concurrent read-write table is greater than 0;

[0139] The invalidation module 903 is used to invalidate old target data in the cache.

[0140] Each module in the above device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0141] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG10. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned data reading method and / or data writing method are implemented.

[0142] Those skilled in the art will understand that the structure shown in FIG. 10 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0143] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the data reading method and / or data writing method in the above embodiment is implemented.

[0144] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data reading method and / or data writing method in the above embodiments is implemented.

[0145] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the data reading method and / or data writing method in the above embodiments is implemented.

[0146] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0147] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for reading data, characterized in that: The method comprises: In the case where the read request cache of the target data does not hit, reading the target data from the original data source; When it is determined that the concurrent write mark of the target data does not exist in the concurrent read-write table, the read target data is updated to the cache.

2. The method according to claim 1, characterized in that The method further comprises: When it is determined that the concurrent write mark of the target data exists in the concurrent read-write table, the read target data is not updated to the cache.

3. The method according to claim 1, characterized in that Before reading the target data from the original data source, the method further includes: In the case where the read request cache of the target data does not hit, the read concurrency count of the target data is increased by 1 in the concurrent read-write table; After reading the target data from the original data source, the method further includes: In the concurrent read-write table, the read concurrency count of the target data is reduced by 1.

4. The method according to claim 3, characterized in that After the concurrent read and write table reduces the read concurrency count of the target data by 1, the method further includes: If the read concurrency count is 0, the record of the read concurrency count is deleted.

5. A method for writing data, characterized in that: The method comprises: Write the target data into the original data source; When it is determined that the concurrent read count of the target data in the concurrent read-write table is greater than 0, a concurrent write mark is recorded in the concurrent read-write table; Invalidate the old target data in the cache.

6. A data reading device, characterized in that: The device comprises: A reading module, configured to read the target data from an original data source when a read request cache of the target data does not hit the target data; The update module is used to update the read target data to the cache when it is determined that the concurrent write mark of the target data does not exist in the concurrent read-write table.

7. A data writing device, characterized in that: The device comprises: A writing module, used to write the target data into the original data source; A recording module, configured to record a concurrent write mark in the concurrent read-write table when determining that the concurrent read count of the target data in the concurrent read-write table is greater than 0; The invalidation module is used to invalidate the old target data in the cache.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images