Data reading method, device, computer storage medium and system

Abstract

The embodiments of the present application provide a data reading method, device, computer storage medium, and system. The method includes: receiving a data query request for data to be queried; based on the data query request, starting a first query thread and a second query thread in parallel; the first query thread is used to query the data to be queried in a database component, and the second query thread is used to query the data to be queried in a cache component; when the first query thread queries the data to be queried, controlling the first query thread to send a first stop signal to the second query thread to read the data to be queried from the database component; when the second query thread queries the data to be queried, controlling the second query thread to send a second stop signal to the first query thread to read the data to be queried from the cache component. In this way, by starting the first query thread and the second query thread in parallel and performing queries in the database component and the cache component at the same time, data reading performance and resource utilization are improved.

Description

Technical Field

[0001] The present application relates to the field of data management technology, and in particular to a data reading method, device, computer storage medium and system. Background Art

[0002] With the evolution of Internet technology, the overall architecture of software applications has become increasingly complex, and higher requirements have been placed on user experience and system performance. For large systems, data reading is a relatively important link, which will directly affect the system's response time to users. At present, the acceleration method for data reading mainly relies on caching technology, which refers to the process of reading data from the cache component. Although the cache component reads and writes faster and more flexibly, due to the small capacity of the cache component and the need to maintain the life cycle of the data, the system needs to provide data reading services through a dedicated caching mechanism. However, the caching mechanism in the relevant technology still has shortcomings in resource utilization, performance improvement and system stability. Summary of the Invention

[0003] The present application provides a data reading method, device, computer storage medium and system, which can query data in the database component and cache component at the same time by starting the first query thread and the second query thread in parallel, thereby improving data reading performance and resource utilization.

[0004] The technical solution of this application is achieved as follows:

[0005] In a first aspect, an embodiment of the present application provides a data reading method, the method comprising:

[0006] Receiving a data query request for data to be queried;

[0007] Based on the data query request, a first query thread and a second query thread are started in parallel; wherein the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component;

[0008] When the first query thread finds the data to be queried, controlling the first query thread to send a first stop signal to the second query thread, and reading the data to be queried from the database component; wherein the first stop signal is used to stop the second query thread;

[0009] When the second query thread finds the data to be queried, the second query thread is controlled to send a second stop signal to the first query thread, and reads the data to be queried from the cache component; wherein the second stop signal is used to stop the first query thread.

[0010] In a second aspect, an embodiment of the present application provides a data reading device, which includes a receiving unit, a starting unit, a first reading unit, and a second reading unit, wherein:

[0011] A receiving unit configured to receive a data query request for data to be queried;

[0012] A starting unit is configured to start a first query thread and a second query thread in parallel based on the data query request; wherein the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component;

[0013] a first reading unit configured to, when the first query thread finds the data to be queried, control the first query thread to send a first stop signal to the second query thread and read the data to be queried from the database component; wherein the first stop signal is used to stop the second query thread;

[0014] The second reading unit is configured to control the second query thread to send a second stop signal to the first query thread and read the data to be queried from the cache component when the second query thread queries the data to be queried; wherein the second stop signal is used to stop the first query thread.

[0015] In a third aspect, an embodiment of the present application provides a data reading device, which includes a memory and a processor; wherein,

[0016] The memory is used to store a computer program that can be run on the processor;

[0017] The processor is configured to execute the steps of the method described in the first aspect when running the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a data reading program. When the data reading program is executed by at least one processor, it implements the steps of the method described in the first aspect.

[0019] In a fifth aspect, the present application provides a data reading system, which at least includes the data reading device as described in the second aspect or the third aspect.

[0020] The embodiment of the present application provides a data reading method, device, computer storage medium and system, which receive a data query request for data to be queried; based on the data query request, start a first query thread and a second query thread in parallel; wherein the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component; when the first query thread queries the data to be queried, the first query thread is controlled to send a first stop signal to the second query thread, and read the data to be queried from the database component; wherein the first stop signal is used to stop the second query thread; when the second query thread queries the data to be queried, the second query thread is controlled to send a second stop signal to the first query thread, and read the data to be queried from the cache component; wherein the second stop signal is used to stop the first query thread. In this way, by starting the first query thread and the second query thread in parallel, the data to be queried can be queried in the database component and the cache component at the same time, thereby avoiding the latency problem of cache reading and improving data reading performance and resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of a data reading method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of the structure of a data reading system provided in an embodiment of the present application;

[0023] Figure 3 A flowchart of another data reading method provided in an embodiment of the present application;

[0024] Figure 4 A flowchart of another data reading method provided in an embodiment of the present application;

[0025] Figure 5 A flowchart of another data reading method provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of a data reading device provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of another data reading device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the hardware structure of a data reading device provided in an embodiment of the present application;

[0029] Figure 9A schematic diagram of the composition structure of another data reading system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0031] With the evolution of internet systems, the overall architecture of software systems has become increasingly complex, placing higher demands on user experience and system performance. Data access is a critical component of any internet system, directly impacting system response time. Currently, data access acceleration relies primarily on caching technology. While caches offer faster and more flexible read and write speeds, they have smaller capacity and require data lifecycle (TTL) maintenance, increasing system complexity. Therefore, specialized caching mechanisms are often required in business domains to provide data access services.

[0032] In related technologies, caching mechanisms mainly include the following:

[0033] (1) Database caching mechanism

[0034] By enabling the query cache mechanism (query cache) of the database component (or database), the database component itself maintains the query result cache. At this time, the loading, reading, and deletion of the cache are all completed by the database component. When the application queries the same data, the database component skips parsing, optimization, and execution, directly accesses the cache and returns the query results.

[0035] The database caching mechanism is relatively simple and easy to use, and the application does not need to worry about whether the data is obtained from the cache or by executing a query. It is relatively transparent and does not require the introduction of additional technical components.

[0036] (2) Distributed caching mechanism

[0037] The system introduces an independent distributed cache component. When an application accesses data, it first queries the cache component. If a cache hit is found, the data is retrieved directly. If a cache miss is found, the database component is queried and the data is loaded into the cache component. This process is a two-stage serial query, requiring the application to implement a series of logical code to control and maintain the lifecycle of cache entries. Multiple interactions are required to complete the entire data retrieval and cache loading process.

[0038] The distributed caching mechanism is currently a relatively mainstream technical solution. By appropriately introducing professional high-performance cache components, cache maintenance can be handed over to professional components, and the application only needs to initiate relevant commands.

[0039] (3) System caching mechanism

[0040] A cache component is designed and opened up within the system, and the memory space is shared with the business system program. When the application accesses data, it first accesses the cache component. If the cache is hit, it is directly obtained. If the cache is not hit, the database component is queried to return the data and the data is loaded into the cache component.

[0041] The implementation process of the system cache mechanism is similar to that of the distributed cache mechanism, but the system cache mechanism does not require interaction with an external distributed cache component. Instead, it uses a local cache area instead of a distributed cache component. This eliminates the need for a cache component, and cached data is stored in the application system's memory. Access to data does not require remote calls, making it more efficient and faster.

[0042] However, the aforementioned cache mechanisms all have their own shortcomings, which are described in detail below.

[0043] Database caching mechanism: (1) Database components need to match completely identical Structured Query Language (SQL) instruction lines before accessing the cache, which has low flexibility; (2) When the data or table structure of the original database component table involved in the cached data changes, all cached data generated by the table will become invalid, which has certain limitations; (3) Enabling the database caching mechanism will occupy the computing resources of the database component, which is not conducive to the stability of the overall business;

[0044] Regarding the distributed caching mechanism: (1) the overall process is carried out in a typical serial manner, that is, it is necessary to access the cache component first and then the database component, which has certain unnecessary performance overhead; (2) the entire process logic needs to be coded and maintained, which increases the complexity of business coding; (3) when a piece of data is designed to be in the cache, it will always be maintained in the cache. When the cache component fails or the response becomes slow, even if the database component has considerable free resources, they cannot be reasonably utilized.

[0045] Regarding the system cache mechanism: (1) the entire process logic needs to be coded and maintained, which increases the complexity of business coding; (2) when a piece of data is designed to be in the cache, it will always be maintained in the cache area. Even if the database component has considerable free resources at a certain time, it cannot be reasonably utilized; (3) the cache area is shared with the application system memory, which may affect the stability of the application system due to excessive cache loading.

[0046] In addition, the above-mentioned cache mechanisms all accelerate and optimize data in a cache manner, but there are no good and complete improvements in resource utilization, performance improvement and system stability.

[0047] Based on this, an embodiment of the present application provides a data reading method, the basic idea of ​​which is: receiving a data query request for data to be queried; based on the data query request, starting a first query thread and a second query thread in parallel; wherein, the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component; when the first query thread queries the data to be queried, the first query thread is controlled to send a first stop signal to the second query thread, and read the data to be queried from the database component; wherein, the first stop signal is used to stop the second query thread; when the second query thread queries the data to be queried, the second query thread is controlled to send a second stop signal to the first query thread, and read the data to be queried from the cache component; wherein, the second stop signal is used to stop the first query thread. In this way, by starting the first query thread and the second query thread in parallel, the data to be queried can be queried in the database component and the cache component at the same time, and the reading method can be flexibly scheduled, thereby avoiding the latency problem of cache reading and improving data reading performance and resource utilization.

[0048] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0049] In one embodiment of the present application, see Figure 1 , which shows a flow chart of a data reading method provided by an embodiment of the present application. Figure 1 As shown, the method may include:

[0050] S101: Receive a data query request for data to be queried.

[0051] It should be noted that the data reading method provided in the embodiments of the present application can be hosted on a functional module, which can be installed in a variety of systems, such as a distributed storage system or a non-distributed storage system. Furthermore, the functional module can be installed on a user device or on a backend server, and the embodiments of the present application are not limited thereto.

[0052] It should be noted that, in the data reading method of the embodiment of the present application, after receiving a data query request for the data to be queried, the process of reading the data to be queried is initiated. Here, the essence of the data query request is an instruction to read the data to be queried. The querying party sending the data query request can be a user or an upper-level module in the system architecture.

[0053] S102: Based on the data query request, start a first query thread and a second query thread in parallel.

[0054] It should be noted that the first query thread is used to query the database component for the data being queried, while the second query thread is used to query the cache component for the data being queried. Here, the cache component refers to a reserved storage space in system memory that temporarily stores input or output data. Generally speaking, compared to the database component, which specifically stores and manages data, the cache component has a faster read speed.

[0055] In the related art, the data to be queried is generally first queried in the cache component. If the data cannot be found, another thread is serially started to query the data to be queried from the database component, but this reduces the efficiency of data reading. In addition, when the cache component is in an unhealthy operating state, its data reading speed will drop significantly. However, according to the reading scheme in the related art, the data to be queried will still be read from the cache component first. As a result, although the data to be queried can be read faster from the database component, the system still obtains data from the cache component, thereby reducing data utilization.

[0056] In an embodiment of the present application, after receiving a data query request, the first query thread and the second query thread are started in parallel, and the data to be queried can be queried in the database component and the cache component at the same time, which improves the speed of data reading and also improves data utilization.

[0057] S103: When the first query thread finds the data to be queried, the first query thread is controlled to send a first stop signal to the second query thread, and the data to be queried is read from the database component.

[0058] It should be noted that because the first and second query threads are executed in parallel, when one query thread finds the data to be queried, the query thread that has found the data to be queried sends a stop signal to the other query thread, thereby stopping the other query thread and avoiding wasting additional system processing capacity. The stop signal can be implemented as a preset semaphore or other technical means that can achieve the same purpose.

[0059] Therefore, if the first query thread finds the data to be queried in the database component, the first query thread sends a first stop signal to the second query thread, so that the second query thread terminates. Here, the first stop signal is used to stop the second query thread.

[0060] S104: When the second query thread finds the data to be queried, control the second query thread to send a second stop signal to the first query thread, and read the data to be queried from the cache component.

[0061] It should be noted that if the second query thread finds the data to be queried in the cache component, the second query thread sends a second stop signal to the first query thread, so that the first query thread terminates; wherein the second stop signal is used to stop the first query thread.

[0062] It should be noted that after the first query thread / second query thread finds the data to be queried, the data to be queried is read and then the read data to be queried is returned to the querying party. Here, if a read failure occurs when reading the data to be queried, a read failure message is returned to the querying party.

[0063] It should also be noted that, in order to improve query efficiency, a data index area may also be provided in a general system. Therefore, in some embodiments, before starting the first query thread and the second query thread in parallel based on the data query request, the method may further include:

[0064] Determine whether the data index area stores the location index information of the data to be queried; wherein the location index information of the data to be queried is used to indicate that the data to be queried is stored in the database component or the cache component;

[0065] When the judgment result is yes, obtaining the position index information of the data to be queried from the data index area, and reading the data to be queried based on the index position information of the data to be queried;

[0066] When the judgment result is no, based on the data query request, the first query thread and the second query thread are started in parallel.

[0067] It should be noted that the data index area is a separate area in the system that stores multiple location index information. Here, the data index area can be set in various locations, such as memory, database, or other devices. Considering the small amount of data and the frequent access to the data index area, it is generally set in system memory.

[0068] The location index information can indicate whether the corresponding data (or data block) is stored in a database component or a cache component. That is, the location index information of the data to be queried is used to indicate whether the data to be queried is stored in a database component or a cache component.

[0069] In addition, the data index area only records whether a data block is stored in the database component or the cache component, without specifying the specific storage location of the data block. Therefore, the index location information either points to the database component or the cache component. It is worth noting that the data index area is set up for query convenience. For data stored in both the data and cache components, only one index location information is retained.

[0070] Therefore, after receiving the query instruction, the index position information of the data to be queried is queried in the data index area; if the index position information of the data to be queried can be found in the data index area, then the corresponding query thread can be directly started according to the index position information of the data to be queried, and the data to be queried can be read and returned to the querying party; if the index position information of the data to be queried cannot be found in the data index area, then step S102 is executed, that is, the first query thread and the second query thread are started in parallel.

[0071] That is to say, assuming that the index location information of the data to be queried stored in the data index area points to the database component, then the first query thread can be directly started to query the data to be queried in the database component without starting the second query thread in parallel; assuming that the index location information of the data to be queried stored in the data index area points to the cache component, then the second query thread can be directly started to query the data to be queried in the cache component without starting the first query thread in parallel.

[0072] In addition, since the data stored in the cache component may be cleared periodically, the following situation may occur: although the index location information of the data to be queried is found in the data index area and points to the cache component, the second query thread does not find the data to be queried in the cache component. At this time, the first query thread will be started serially, and the first query thread will eventually find the data to be queried in the database component (all data in the system is ultimately stored in the database), and load the data to be queried into the cache component, and return the data to be queried to the querying party.

[0073] It should be noted that if the data index area does not store the index location information of the data to be queried, the first query thread and the second query thread will be started in parallel to simultaneously query the data to be queried in the database component and the cache component. After the first query thread or the second query thread finds the data to be queried, the actual storage location of the data to be queried is added to the data index area to facilitate subsequent queries of the data to be queried. Therefore, in some embodiments, when the first query thread finds the data to be queried, the method may further include:

[0074] Creating first location index information of the data to be queried according to the query result of the first query thread;

[0075] The first position index information is stored in the data index area; wherein the first position index information is used to indicate that the data to be queried is stored in the database component.

[0076] It should be noted that for the parallel first query thread and second query thread, if the first query thread queries the data to be queried in the database component, then the first position index information of the data to be queried is created. At this time, the first position index information points to the database component, and then the first position index information is stored in the data index area.

[0077] In this way, if an instruction to query the data to be queried is received later, the first position index information can be found in the data index area, so that the index position information of the data to be queried can be determined as a database component, and the data to be queried can be directly queried in the database component through the first query thread.

[0078] Furthermore, in some embodiments, when the second query thread queries the data to be queried, the method further includes:

[0079] Creating second location index information of the data to be queried according to the query result of the second query thread;

[0080] The second position index information is stored in the data index area; wherein the second position index information is used to indicate that the data to be queried is stored in the cache component.

[0081] It should be noted that, similarly, if the second query thread queries the data to be queried in the cache component, then in the data index area, a second position index information of the data to be queried is created. At this time, the second position index information points to the cache component, and the second position index information is stored in the data index area.

[0082] In this way, by storing the second position index information in the data index area, the processing load of subsequent queries is saved, and the query speed is increased, thereby improving the user experience.

[0083] To sum up, through the parallel first query thread and the second query thread, it is possible to query the data to be queried in the database component and the cache component at the same time. The query thread that queries the data to be queried first reads the data to be queried and returns the read data to be queried to the querying party, avoiding the delay problem of cache reading and improving data reading performance and resource utilization.

[0084] Furthermore, the data index area can be regularly optimized and rewritten to speed up data reading. Therefore, in some embodiments, the method may further include:

[0085] For the plurality of location index information stored in the data index area, determining in sequence whether the plurality of location index information meets a rewriting condition;

[0086] When it is determined that one of the position index information meets the rewrite condition, a rewrite operation is performed on the one of the position index information; wherein the rewrite operation indicates that the position index information is rewritten from indicating that the data is stored in the database component to indicating that the data is stored in the cache component, or from indicating that the data is stored in the cache component to indicating that the data is stored in the database component.

[0087] It should be noted that the data index area is periodically maintained. Specifically, for the multiple index position information stored in the data index area, the multiple index position information is sequentially determined to determine whether they meet the rewrite condition. If it is determined that one of the index position information meets the rewrite condition, the index position information is rewritten.

[0088] That is, if a certain index location information indicates that data is stored in a database component, and after determination, it is found that the index location information meets the rewrite condition, the index location information will be rewritten from indicating that the data is stored in the database component to indicating that the data is stored in the cache component. Similarly, if the index location information indicates that data is stored in a cache component, and after determination, it is found that the index location information meets the rewrite condition, the index location information will be rewritten from indicating that the data is stored in the cache component to indicating that the data is stored in the database component. This ensures that if an operational problem occurs in the database component or the cache component, the index location information can be updated in a timely manner, thereby optimizing data reading speed and data utilization efficiency.

[0089] Furthermore, in some embodiments, before sequentially determining whether the plurality of location index information satisfy the rewriting condition, the method may further include:

[0090] Calculating a database score based on historical operating data of the database component in the previous time period;

[0091] Calculating a cache score based on the historical operating data of the cache component in the previous time period;

[0092] Determining, based on historical query data of the first query thread in a previous time period, first response times for each of the plurality of location index information; wherein different location index information corresponds to different first response times, and the first response time is used to indicate the time required to retrieve the data indicated by the corresponding location index information from the database component;

[0093] determining, based on historical query data of the second query thread in a previous time period, a second response time for each of the plurality of location index information; wherein different location index information corresponds to different second response times, and the second response time is used to indicate a time required to retrieve data indicated by the corresponding location index information from the cache component;

[0094] The step of sequentially determining whether the plurality of location index information satisfy a rewriting condition is performed based on the database score, the cache score, the first response time of each of the plurality of location index information, and the second response time of each of the plurality of location index information.

[0095] It should be noted that the data index area is maintained periodically. In other words, the system has a pre-set periodic duration, which is determined based on actual usage requirements, such as 10 minutes, 30 minutes, 60 minutes, etc.

[0096] To ensure that the index location information in the data index area is the most reasonable, at the beginning of a new time period, the historical operation data of the database component and the cache component in the previous time period are obtained. The database score is calculated based on the historical operation data of the database component, and the cache score is calculated based on the historical operation data of the cache component.

[0097] In other words, the database score can indicate the operating status of the database component in the previous time period, and the cache score can indicate the operating status of the cache component in the previous operating cycle. The better the operating status of the database component / cache component, the higher the corresponding database score / cache score. In addition, the specific calculation of the database score and cache score can include a variety of algorithms, which are not specifically limited in the embodiments of this application.

[0098] It should also be noted that historical operating data may include operating data of various attributes, such as the number of queries per second, CPU utilization, memory utilization, disk throughput, or network throughput. Therefore, in some embodiments, calculating the database score based on the historical operating data of the database component in the previous time period may include:

[0099] Determining multiple operating status values ​​of the database component based on historical operating data of the database component in a previous time period; wherein the multiple operating status values ​​include at least one of the following: number of queries per second, central processing unit utilization, memory utilization, disk throughput, and network throughput;

[0100] Calculating the various operating status values ​​of the database component respectively to obtain a plurality of sub-scores of the database component;

[0101] A weighted sum operation is performed on multiple sub-scores of the database component according to preset weights to obtain the database score.

[0102] Similarly, in some embodiments, calculating the cache score based on the historical operating data of the cache component in the previous time period may include:

[0103] Determining multiple operating status values ​​of the cache component based on historical operating data of the cache component in a previous time period; wherein the multiple operating status values ​​include at least one of the following: number of queries per second, central processing unit utilization, memory utilization, disk throughput, and network throughput;

[0104] Calculating the various operating status values ​​of the cache component respectively to obtain a plurality of sub-scores of the cache component;

[0105] A weighted sum operation is performed on multiple sub-scores of the cache component according to preset weights to obtain the cache score.

[0106] It should be noted that the database score / cache score is derived based on historical operational data from the previous period. First, based on this historical operational data, various operational status values ​​for the database / cache component are determined. Second, based on these various operational status values, multiple sub-scores for each database / cache component are derived. Finally, the weighted sum of these sub-scores is used to determine the database score / cache score. These multiple operational status values ​​include at least one of the following: queries per second, CPU utilization, memory utilization, disk throughput, and network throughput.

[0107] In a specific embodiment, the process of calculating the database score / cache score can be: (1) comparing the operating data of multiple attributes with their respective preset business thresholds to obtain multiple sub-scores; (2) performing weighted summation on the multiple sub-scores according to preset weights to obtain the database score / cache score.

[0108] It should be noted that, for multiple sub-scores, the weight of each sub-score may be the same or different; if the weights of the multiple sub-scores are different, the preset weight includes the weights of the multiple sub-scores.

[0109] It should also be noted that in addition to the database score and cache component score, the first response time and second response time corresponding to each location index information must also be obtained. Here, different location index information corresponds to different first response times, which indicate the time required to retrieve the data indicated by the corresponding location index information from the database component; and different location index information corresponds to different second response times, which indicate the time required to retrieve the data indicated by the corresponding location index information from the cache component.

[0110] Furthermore, in some embodiments, determining the first response time of each of the plurality of location index information based on historical query data of the first query thread in the previous time period may include:

[0111] For one of the plurality of location index information, determining, based on historical query data of the first query thread in a previous time period, a plurality of first historical response times for querying the data indicated by the one of the location index information by the first query thread;

[0112] An average value of the plurality of first historical response times is calculated, and the calculated average value is determined as the first response time of one of the location index information, so as to obtain the first response time of each of the plurality of location index information.

[0113] Similarly, in some embodiments, determining the second response time of each of the plurality of location index information based on historical query data of the second query thread in the previous time period may include:

[0114] For one of the plurality of location index information, determining, based on historical query data of the second query thread in a previous time period, a plurality of second historical response times for querying the data indicated by the one of the location index information by the second query thread;

[0115] An average value of the plurality of second historical response times is calculated, and the calculated average value is determined as the second response time of one of the location index information, so as to obtain the second response time of each of the plurality of location index information.

[0116] It should be noted that the first response time can be the average response time required by the database component to query the data corresponding to the index location information during the previous time period. Therefore, for a specific location index information, based on the historical query data of the first query thread, multiple first historical response times for the data indicated by the first query thread during the previous time period corresponding to the location index information are determined, and the average of these multiple first historical response times is used as the first response time for the location index information. In this way, the first response time of each of the multiple location index information can be calculated.

[0117] The second response time can be the average response time required by the cache component to query the data corresponding to the index location information during the previous time period. Therefore, for a specific location index information, based on the historical query data of the second query thread, multiple second historical response times for the data indicated by the second query thread during the previous time period corresponding to the location index information are determined, and the average of these multiple second historical response times is used as the second response time for the location index information. In this way, the second response time of each of the multiple location index information can be calculated.

[0118] In this way, after obtaining the database score, cache score, first response time of each of the plurality of location index information and second response time of each of the plurality of location index information, it is determined in turn whether the plurality of location index information meets the rewriting condition based on this information.

[0119] Furthermore, in some embodiments, when one of the location index information indicates that data is stored in the database component, when it is determined that one of the location index information meets the rewrite condition, rewriting the one of the location index information includes:

[0120] When the database score is less than the cache score, and the first response time of one of the location index information is greater than a preset business threshold, and the second response time of one of the location index information is less than the preset business threshold, the one of the location index information is rewritten from indicating that the data is stored in the database component to indicating that the data is stored in the cache component.

[0121] It should be noted that for index location information indicating data stored in database components, the rewrite conditions are:

[0122] (1) The database score is less than the cache score;

[0123] (2) The first response time of the index location information is greater than a preset service threshold;

[0124] (3) The second response time of the index location information is less than a preset service threshold.

[0125] Here, the preset business threshold is the maximum response time that the business system can accept. For index location information indicating that data is stored in a database component, when the index location information satisfies the above conditions at the same time, it is determined that the index location information meets the rewriting condition.

[0126] It should also be noted that, for the index location information indicating that data is stored in the database component, the rewriting operation refers to rewriting the index location information to indicate that the data is stored in the cache component.

[0127] Furthermore, in some embodiments, when one of the location index information indicates that data is stored in the cache component, when it is determined that one of the location index information meets the rewrite condition, rewriting the one of the location index information includes:

[0128] When the database score is greater than the cache score, and the first response time of one of the location index information is less than a preset business threshold, and the second response time of one of the location index information is greater than the preset business threshold, one of the location index information is rewritten from indicating that the data is stored in the cache component to indicating that the data is stored in the database component.

[0129] It should be noted that for index location information indicating that data is stored in a cache component, the rewrite conditions are:

[0130] (1) The database score is greater than the cache score;

[0131] (2) The first response time of the index location information is less than a preset service threshold;

[0132] (3) The second response time of the index location information is greater than a preset service threshold.

[0133] Here, the preset business threshold is the maximum response time that the business system can accept. For index location information indicating that data is stored in the cache component, when the index location information satisfies the above conditions at the same time, it is determined that the index location information satisfies the rewrite condition, and the index location information is rewritten to indicate that the data is stored in the database component.

[0134] It should also be noted that since all data is definitely stored in the database component, and the cache component indicates the temporary storage area, for some data blocks, although the data index area stores the index location information pointing to the cache component, it is possible that the cache component does not store the data or the data has been cleared. Then, during the query process, if the second query thread cannot find the data in the cache component, the first query thread will be started serially to query the data from the database component and load the data into the cache component.

[0135] In this way, by regularly self-organizing the index data in the data index area, the index position can be dynamically adjusted, and the resources of the database component / cache component can be reasonably utilized to achieve the goal of improving overall performance and data utilization.

[0136] In summary, there are currently no effective technical solutions or low-cost methods to address the serial latency issue of cache reads in the business support field. In the embodiments of this application, queries to the database component and cache component are performed using two types of threads, and these two types of threads are carried out in parallel, which largely solves this problem.

[0137] In addition, in the data index area, the index location information is periodically calculated and dynamically adjusted, which greatly improves the overall system utilization. When one of the database components / cache components cannot respond or responds slowly, it can be dynamically adjusted to another component, which makes the data reading process quite fault-tolerant.

[0138] In other words, the key points of the embodiments of the present application are:

[0139] (1) The concurrent reading concept of the first query thread and the second query thread;

[0140] (2) Periodically calculating and rewriting index location information not only satisfies the data reading function, but also greatly improves the overall system utilization.

[0141] An embodiment of the present application provides a data reading method, which receives a data query request for data to be queried; based on the data query request, starts a first query thread and a second query thread in parallel; wherein, the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component; when the first query thread queries the data to be queried, controls the first query thread to send a first stop signal to the second query thread, and reads the data to be queried from the database component; wherein, the first stop signal is used to stop the second query thread; when the second query thread queries the data to be queried, controls the second query thread to send a second stop signal to the first query thread, and reads the data to be queried from the cache component; wherein, the second stop signal is used to stop the first query thread. In this way, by starting the first query thread and the second query thread in parallel, it is possible to query the data to be queried in the database component and the cache component at the same time, flexibly schedule the reading method, thereby avoiding the latency problem of cache reading and improving data reading performance and resource utilization.

[0142] In another embodiment of the present application, see Figure 2 , which shows a schematic diagram of the structure of a data reading system provided by an embodiment of the present application. Figure 2 As shown, the data reading system includes a data query interface (QueryInterface) 201, a data index area (Index) 202, a collector (Collector) 203, a data self-organizing module (DataSelf-organizing Module, DSOM) 204, a database component (Database) 205 and a cache component (Cache) 206.

[0143] Data query interface 201, responsible for connecting to external data query requests;

[0144] The data index area 202 is used to store the location mark of the data block (the location mark of the data block is equivalent to the aforementioned index location information), thereby directing the current data query request to the retrieval database component 205 or the cache component 206;

[0145] Collector 203, used to collect the response time of each data query request falling on the database component 205 and cache component 206, as well as the operating performance data of the two components;

[0146] The data self-organizing module 204 is used to obtain the index position information of the data block through a series of algorithm calculations based on the various indicators collected by the collector 203.

[0147] Furthermore, based on the data reading system, the present application embodiment also provides another data reading method, see Figure 3 , which shows a flow chart of another data reading method provided by an embodiment of the present application. Figure 3 As shown, the method may include:

[0148] S301: The data query request of the querying party arrives at the data query interface;

[0149] It should be noted that the data query request from the query party arrives at the data query interface to start the data query process.

[0150] S302: The data query interface queries the data index area to determine whether the data index area stores index location information of the data to be queried.

[0151] Here, for step S302, if the judgment result is no, step S303 is executed.

[0152] It should be noted that the data query interface first searches the data index area to determine whether the data to be queried hits the data index area.

[0153] If the data index area stores the index position information (Index(D)) of the data to be queried, subsequent queries can be performed directly according to the index position information of the data to be queried; if the data index area does not store the index position information of the data to be queried, execute step S303.

[0154] S303: The data query interface simultaneously initiates two types of query threads: a database query thread and a cache query thread.

[0155] It should be noted that the database query thread (Database_Query_Thread) is equivalent to the aforementioned first query thread, which is used to query data from the database component; the cache query thread (Cache_Query_Thread) is equivalent to the aforementioned second query thread, which is used to query data from the cache component.

[0156] It should be noted that the data query interface initiates the database query thread and the cache query thread in parallel, thereby achieving synchronous query of the data to be queried in the database component and the cache component.

[0157] S304: After any thread obtains the data, it transmits a "thread termination" semaphore to the other thread, and the other thread stops querying. The data query interface obtains the data returned by the thread that queried the data faster.

[0158] It's important to note that for both the database query thread and the cache query thread, once either thread retrieves data, it initiates thread communication with the other thread, passing the "thread terminate" semaphore, causing the other thread to stop querying. The data query interface retrieves data returned by the thread that retrieved the data faster, and the entire process is a parallel process.

[0159] S305: According to the location of the data to be queried, the data query interface maintains index location information of the data index area.

[0160] It should be noted that the data query interface maintains the data index area based on the location of the data being queried (database component or cache component), that is, creates the index location information of the data being queried. This way, if you subsequently query the data, you can directly obtain its index location information from the data index area and then directly query it using the corresponding query thread.

[0161] S306: The data query interface returns the data to be queried to the querying party.

[0162] It should be noted that the data query interface obtains the data returned by the thread that queries the data faster and returns the data to be queried to the querying party.

[0163] In this way, for the data to be queried that cannot be indexed in the data index area, by starting two query threads in parallel, it can be queried in the database component and the cache component at the same time, thereby improving the speed and efficiency of data query.

[0164] An embodiment of the present application provides a data reading method. Through the detailed explanation of the above-mentioned embodiment in this embodiment, it can be seen that by starting the first query thread and the second query thread in parallel, the data to be queried can be queried in the database component and the cache component at the same time, and the reading method can be flexibly scheduled, thereby avoiding the delay problem of cache reading and improving data reading performance and resource utilization.

[0165] Based on the aforementioned data reading system, see Figure 4 , which shows a flow chart of another data reading method provided by an embodiment of the present application. Figure 4 As shown, the method may include:

[0166] S401: The data query request of the querying party arrives at the data query interface.

[0167] It should be noted that the data query request from the query party reaches the data query interface, which is the data query process.

[0168] S402: The data query interface queries the data index area to determine whether the data index area stores index location information of the data to be queried;

[0169] Here, for step S402, if the judgment result is yes, step S403 is executed.

[0170] It should be noted that the data query interface first searches the data index area to determine whether the data to be queried hits the data index area. If the data index area stores the index position information of the data to be queried, subsequent queries can be performed directly according to the index position information of the data to be queried.

[0171] S403: Initiate different query threads according to the index location information of the data to be queried, and the query object is a database component or a cache component.

[0172] It should be noted that when the index location information of the data to be queried points to the database component, a database query thread is initiated, and the data to be queried is only queried in the database component; when the index location information of the data to be queried points to the cache component, a cache component query thread is initiated, and the data to be queried is only queried in the cache component.

[0173] S404: Obtain the data to be queried and return it to the querying party.

[0174] It should be noted that after reading the data to be queried, the data query interface returns the data to be queried to the querying party.

[0175] An embodiment of the present application provides a data reading method, which can query the data to be queried in the database component and the cache component at the same time by starting the first query thread and the second query thread in parallel, thereby avoiding the latency problem of cache reading and improving data reading performance and resource utilization.

[0176] Based on the aforementioned data reading system, see Figure 5 , which shows a flow chart of another data reading method provided by an embodiment of the present application. Figure 5 As shown, the method may include:

[0177] S501: The collector regularly monitors the running status of the database component and the cache component and collects data.

[0178] It should be noted that, during the operation of the system, the data self-organizing module dynamically adjusts the index position every T time, where T is a preset time period, for example, 10 minutes.

[0179] It should also be noted that during each time period, the collector monitors the operating status of the database and cache components and collects data. Specific operating status data collected may include: queries per second (the field name can be QPS), CPU utilization (the field name can be cpu), memory utilization (the field name can be memory), disk throughput (the field name can be disk_io), and network throughput (the field name can be net_io). The collector also collects the response time of all database query threads and cache component query threads that successfully retrieved data within the time range T.

[0180] S502: The data self-organizing module obtains the data collected by the collector in the previous cycle, calculates the health of the database component, the health of the cache component and the thread response time corresponding to each position index information in the data index area.

[0181] It should be noted that the health of the database component is equivalent to the aforementioned database score, and the health of the cache component is equivalent to the aforementioned database score. Furthermore, the thread response time includes the first response time for a database query thread to query the data block corresponding to the location index information, and the second response time for a cache component to query the data block corresponding to the location index information.

[0182] Specifically, the calculation of data can be divided into two parts: the calculation of running status data and the calculation of thread response time.

[0183] (1) Calculation of operating status data

[0184] Each piece of operating status data is compared with the corresponding preset business threshold, and then the health of the current component is calculated based on the weight (the higher the health score, the healthier the component). An example is as follows:

[0185] The QPS value health score is calculated as: S(QPS)=Fun1(QPS);

[0186] The CPU health score is calculated as: S(cpu) = Fun2(cpu);

[0187] The memory health score is calculated as: S(memory) = Fun3(memory);

[0188] The disk_io value health score is calculated as: S(disk_io) = Fun4(disk_io);

[0189] The health score of net_io value is calculated as: S(net_io) = Fun5(net_io);

[0190] It should be noted that Fun1, Fun2, Fun3, Fun4, and Fun5 are the preset QPS threshold, preset CPU threshold, preset memory threshold, preset disk_io threshold, and preset net_iod threshold, respectively. Furthermore, calculating the health score by threshold comparison is only one specific algorithm, and other algorithms in related technologies may also be used for calculation, which is not limited in this embodiment of the present application.

[0191] After that, the weights of QPS, CPU, memory, disk_io, and net_io are W1, W2, W3, W4, and W5 respectively. The component health score is:

[0192] S_total=S(QPS)*W1+S(cpu)*W2+S(memory)*W3+S(disk_io)*W4+S(net_io)*W5.

[0193] Through the above calculations, we can obtain the health of the database component and the health of the cache component respectively.

[0194] (2) Calculation of operating status data

[0195] For each running state data, the data self-organizing module calculates the thread response time of each index position information by a preset thread response time calculation method. Here, the preset thread response time calculation method can adopt a variety of algorithms, which are not limited in this application.

[0196] In a specific embodiment, for a specific location index information, the average response time value of the database component in the previous cycle querying the data block corresponding to the location index information can be used as the first response time RT (Database_Query_Thread), and the average response time value of the cache component in the previous cycle querying the data block corresponding to the location index information can be used as the second response time RT (Cache_Query_Thread).

[0197] S503: Each index position information in the data index area is judged, and if one of the index position information meets the rewriting condition, the one of the index position information is rewritten.

[0198] It should be noted that: the index position information in the data index area is judged, and if one of the index position information meets the rewrite condition, the index position information is rewritten. Here, the rewrite condition can be determined according to actual use requirements and is not limited in this embodiment of the present application.

[0199] For index location information pointing to a database component, the rewrite operation refers to rewriting the index location information to point to a cache component; for index location information pointing to a cache component, the rewrite operation refers to rewriting the index location information to point to a database component.

[0200] In a specific embodiment, step S503 may include the following specific steps:

[0201] First, a preset service threshold (RT_MAX) is determined, that is, the maximum RT value acceptable to the service.

[0202] Secondly, when the RT of one component (database component or cache component) is higher than RT_MAX, the RT of another component is lower than RT_MAX, and the health of the component is lower than that of the other component, the index position is rewritten to point to the other component.

[0203] In other words, the index location information is judged and rewritten. If the index location information indicates that the data is stored in the cache component, after T time of operation, when the following three conditions are met, the index location information is rewritten to indicate that the data is stored in the database component:

[0204] (1) RT (Cache_Query_Thread) is greater than RT_MAX (the maximum RT acceptable to the business);

[0205] (2) RT (Database_Query_Thread) is less than RT_MAX (the maximum RT acceptable to the business);

[0206] (3) S_total(Database) is greater than S_total(Cache), which means that the health of the Database is better than that of the Cache.

[0207] Similarly, if the index location information indicates that the data is stored in the database component, after T time of operation, when the following three conditions are met, the index location information is rewritten to point to the cache component:

[0208] (1) RT (Cache_Query_Thread) is less than RT_MAX (the maximum RT acceptable to the business);

[0209] (2) RT (Database_Query_Thread) is greater than RT_MAX (the maximum RT acceptable to the business);

[0210] (3) S_total(Database) is greater than S_total(Cache), which means that the health of the Database is better than that of the Cache.

[0211] In this way, by continuously rewriting the data index area within the T period, optimizing the data block reference position (i.e., index position information), and rationally utilizing the resources of each component, the overall performance and utilization rate can be improved.

[0212] In the business support field, there are currently no effective technical methods or low-cost solutions to the serial latency problem of cache reads. In this solution, database and cache queries are performed through two types of threads in a parallel manner, which largely solves this problem.

[0213] In an embodiment of the present application, the data self-organizing module periodically calculates and dynamically adjusts the data reading method based on the data collected by the collector, which greatly improves the overall utilization of the system. When a query to a certain component fails to respond or responds slowly, it can be dynamically adjusted to another component, and it also has a certain degree of fault tolerance.

[0214] Therefore, the embodiment of the present application provides a self-organizing data reading method model based on thread communication, the key points of which are (1) the concurrent reading concept of the Database_Query_Thread and Cache_Query_Thread query threads; (2) with the help of the data self-organizing module, the reading method is regularly self-organized and adjusted, which not only meets the data reading function but also greatly improves the overall system utilization.

[0215] In summary, in the field of operator business support and software architecture, the protection points of this application are: (1) a parallel data reading method based on thread communication, in which the Database query thread and the Cache query thread are executed concurrently and simultaneously, and communicate with each other through inter-thread semaphores; (2) determining the reading thread connection object in the system-built data index area, and calculating and implementing self-organization adjustment of the data reading method through the data self-organization module; (3) a software system implemented based on the above method; (4) a hardware system, device or equipment implemented based on the above method.

[0216] An embodiment of the present application provides a data reading method. Through the detailed explanation of the above embodiment in this embodiment, it can be seen that, on the one hand, the index position information of the data to be queried is queried with the help of the data index area. If the index position of the data to be queried is stored in the data index area, the corresponding query can be directly performed according to the index position of the data to be queried, thereby improving the efficiency and speed of data reading; if the index position of the data to be queried is not stored in the data index area, by starting the first query thread and the second query thread in parallel, the data to be queried can be queried in the database component and the cache component at the same time, thereby avoiding the delay problem of cache reading and improving data reading performance and resource utilization; on the other hand, by continuously rewriting the index position information of the data index area, the resources of each component are reasonably utilized to achieve the purpose of improving overall performance and utilization.

[0217] In yet another embodiment of the present application, see Figure 6 , which shows a schematic diagram of the composition structure of another data reading device 60 provided in an embodiment of the present application. Figure 6 As shown, the data reading device 60 includes a receiving unit 601, a starting unit 602, a first reading unit 603 and a second reading unit 604, wherein:

[0218] A receiving unit 601 is configured to receive a data query request for data to be queried;

[0219] The starting unit 602 is configured to start a first query thread and a second query thread in parallel based on the data query request; wherein the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component;

[0220] The first reading unit 603 is configured to, when the first query thread finds the data to be queried, control the first query thread to send a first stop signal to the second query thread, and read the data to be queried from the database component; wherein the first stop signal is used to stop the second query thread;

[0221] The second reading unit 604 is configured to control the second query thread to send a second stop signal to the first query thread and read the data to be queried from the cache component when the second query thread queries the data to be queried; wherein the second stop signal is used to stop the first query thread.

[0222] In some embodiments, as Figure 7 As shown, the data reading device 60 also includes an index unit 605, which is configured to determine whether the data index area stores the location index information of the data to be queried; wherein the location index information of the data to be queried is used to indicate that the data to be queried is stored in the database component or the cache component; when the judgment result is yes, the location index information of the data to be queried is obtained from the data index area, and the data to be queried is read based on the index location information of the data to be queried; when the judgment result is no, based on the data query request, the first query thread and the second query thread are started in parallel.

[0223] In some embodiments, as Figure 7 As shown, the data reading device 60 also includes a creation unit 606, which is configured to create first position index information of the data to be queried based on the query results of the first query thread; store the first position index information in the data index area; wherein the first position index information is used to indicate that the data to be queried is stored in the database component.

[0224] In some embodiments, the creation unit 606 is further configured to create second location index information of the data to be queried based on the query results of the second query thread; and store the second location index information in the data index area; wherein the second location index information is used to indicate that the data to be queried is stored in the cache component.

[0225] In some embodiments, as Figure 7 As shown, the data reading device 60 also includes a rewrite unit 607, which is configured to determine, in sequence, whether the multiple position index information stored in the data index area meet the rewrite conditions; when it is determined that one of the position index information meets the rewrite condition, rewrite the one of the position index information; wherein the rewrite operation represents rewriting the position index information from indicating that the data is stored in the database component to indicating that the data is stored in the cache component, or rewriting the indicating that the data is stored in the cache component to indicating that the data is stored in the database component.

[0226] In some embodiments, the rewrite unit 607 is further configured to calculate a database score based on the historical operation data of the database component in the previous time period; calculate a cache score based on the historical operation data of the cache component in the previous time period; determine a first response time of each of the multiple location index information based on the historical query data of the first query thread in the previous time period; wherein different location index information corresponds to different first response times, and the first response time is used to indicate the time required to query the corresponding data indicated by the location index information from the database component; determine a second response time of each of the multiple location index information based on the historical query data of the second query thread in the previous time period; wherein different location index information corresponds to different second response times, and the second response time is used to indicate the time required to query the corresponding data indicated by the location index information from the cache component; and perform the step of sequentially determining whether the multiple location index information meet the rewrite condition based on the database score, the cache score, the first response time of each of the multiple location index information, and the second response time of each of the multiple location index information.

[0227] In some embodiments, when one of the location index information indicates that the data is stored in the database component, the rewriting unit 607 is further configured to rewrite the one of the location index information from indicating that the data is stored in the database component to indicating that the data is stored in the cache component when it is determined that the database score is less than the cache score, and the first response time of the one of the location index information is greater than the preset business threshold, and the second response time of the one of the location index information is less than the preset business threshold.

[0228] In some embodiments, when one of the location index information indicates that the data is stored in the cache component, the rewriting unit 607 is further configured to rewrite the one of the location index information from indicating that the data is stored in the cache component to indicating that the data is stored in the database component when it is determined that the database score is greater than the cache score, and the first response time of the one of the location index information is less than the preset business threshold, and the second response time of the one of the location index information is greater than the preset business threshold.

[0229] In some embodiments, the rewrite unit 607 is further configured to determine multiple operating status values ​​of the database component based on historical operating data of the database component in the previous time period; wherein the multiple operating status values ​​include at least one of the following: number of queries per second, CPU utilization, memory utilization, disk throughput, and network throughput; the multiple operating status values ​​of the database component are calculated separately to obtain multiple sub-scores of the database component; and the multiple sub-scores of the database component are weighted and summed according to preset weights to obtain the database score.

[0230] In some embodiments, the rewrite unit 607 is further configured to determine multiple operating status values ​​of the cache component based on historical operating data of the cache component in the previous time period; wherein the multiple operating status values ​​include at least one of the following: number of queries per second, CPU utilization, memory utilization, disk throughput, and network throughput; the multiple operating status values ​​of the cache component are calculated separately to obtain multiple sub-scores of the cache component; and the multiple sub-scores of the cache component are weighted and summed according to preset weights to obtain the cache score.

[0231] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0232] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0233] Therefore, this embodiment provides a computer storage medium, which stores a data reading program. When the data reading program is executed by at least one processor, the steps of the method in any one of the above embodiments are implemented.

[0234] Based on the above-mentioned composition of a data reading device 60 and computer storage medium, see Figure 8 , which shows a specific hardware structure diagram of a data reading device 60 provided in an embodiment of the present application, such as Figure 8 As shown, the data reading device 60 may include: a communication interface 701, a memory 702 and a processor 703; each component is coupled together via a bus device 704. It is understood that the bus device 704 is used to achieve connection and communication between these components. In addition to the data bus, the bus device 704 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus devices 704. Among them, the communication interface 701 is used to receive and send signals in the process of sending and receiving information between other external network elements;

[0235] Memory 702, used to store computer programs that can be run on processor 703;

[0236] The processor 703 is configured to, when running the computer program, execute:

[0237] Receiving a data query request for data to be queried;

[0238] Based on the data query request, a first query thread and a second query thread are started in parallel; wherein the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component;

[0239] When the first query thread finds the data to be queried, controlling the first query thread to send a first stop signal to the second query thread, and reading the data to be queried from the database component; wherein the first stop signal is used to stop the second query thread;

[0240] When the second query thread finds the data to be queried, the second query thread is controlled to send a second stop signal to the first query thread, and reads the data to be queried from the cache component; wherein the second stop signal is used to stop the first query thread.

[0241] It is understood that the memory 702 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 702 of the apparatus and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0242] The processor 703 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the processor 703. The above-mentioned processor 703 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 702 , and the processor 703 reads the information in the memory 702 and completes the steps of the above method in combination with its hardware.

[0243] It is understood that the embodiments described herein may be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, or other electronic units or combinations thereof for performing the functions described herein.

[0244] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0245] Optionally, as another embodiment, the processor 703 is further configured to execute the steps of the method in any one of the aforementioned embodiments when running the computer program.

[0246] Based on the composition and hardware structure diagram of the data reading device 60, see Figure 9 , which shows a schematic diagram of the composition structure of another data reading system 80 provided in an embodiment of the present application. Figure 9 As shown, the data reading system 80 at least includes the data reading device 60 according to any one of the aforementioned embodiments.

[0247] For the data reading system 80, by starting the first query thread and the second query thread in parallel, the data to be queried can be queried in the database component and the cache component at the same time, thereby avoiding the latency problem of cache reading and improving data reading performance and resource utilization.

[0248] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

[0249] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0250] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0251] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0252] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0253] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0254] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data reading method, characterized in that: The method comprises: Receiving a data query request for data to be queried; Based on the data query request, a first query thread and a second query thread are started in parallel; wherein the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component; When the first query thread finds the data to be queried, controlling the first query thread to send a first stop signal to the second query thread, and reading the data to be queried from the database component; wherein the first stop signal is used to stop the second query thread; When the second query thread finds the data to be queried, controlling the second query thread to send a second stop signal to the first query thread, and reading the data to be queried from the cache component; wherein the second stop signal is used to stop the first query thread; The data index area stores a plurality of location index information, wherein the location index information is used to indicate whether the corresponding data is stored in the database component or the cache component; the method further includes: For the plurality of position index information stored in the data index area, determining in sequence whether the plurality of position index information meets a rewriting condition; When it is determined that one of the position index information meets the rewrite condition, a rewrite operation is performed on the one of the position index information; wherein the rewrite operation indicates that the position index information is rewritten from indicating that the data is stored in the database component to indicating that the data is stored in the cache component, or from indicating that the data is stored in the cache component to indicating that the data is stored in the database component.

2. The data reading method according to claim 1, wherein: Before starting the first query thread and the second query thread in parallel based on the data query request, the method further includes: Determine whether the data index area stores the location index information of the data to be queried; wherein the location index information of the data to be queried is used to indicate that the data to be queried is stored in the database component or the cache component; When the judgment result is yes, obtaining the position index information of the data to be queried from the data index area, and reading the data to be queried based on the index position information of the data to be queried; When the judgment result is no, based on the data query request, the first query thread and the second query thread are started in parallel.

3. The data reading method according to claim 2, wherein: When the first query thread finds the data to be queried, the method further includes: Creating first location index information of the data to be queried according to the query result of the first query thread; The first position index information is stored in the data index area; wherein the first position index information is used to indicate that the data to be queried is stored in the database component.

4. The data reading method according to claim 2, wherein: When the second query thread finds the data to be queried, the method further includes: Creating second location index information of the data to be queried according to the query result of the second query thread; The second position index information is stored in the data index area; wherein the second position index information is used to indicate that the data to be queried is stored in the cache component.

5. The data reading method according to claim 1, wherein: Before sequentially determining whether the plurality of position index information satisfy a rewriting condition, the method further includes: Calculating a database score based on historical operating data of the database component in the previous time period; Calculating a cache score based on the historical operating data of the cache component in the previous time period; Determining, based on historical query data of the first query thread in a previous time period, first response times for each of the plurality of location index information; wherein different location index information corresponds to different first response times, and the first response time is used to indicate the time required to retrieve the data indicated by the corresponding location index information from the database component; determining, based on historical query data of the second query thread in a previous time period, a second response time for each of the plurality of location index information; wherein different location index information corresponds to different second response times, and the second response time is used to indicate a time required to retrieve data indicated by the corresponding location index information from the cache component; The step of sequentially determining whether the plurality of location index information satisfy a rewriting condition is performed based on the database score, the cache score, the first response time of each of the plurality of location index information, and the second response time of each of the plurality of location index information.

6. The data reading method according to claim 5, characterized in that: In a case where one of the location index information indicates that data is stored in the database component, when it is determined that one of the location index information meets the rewrite condition, rewriting the one of the location index information includes: When it is determined that the database score is less than the cache score, and the first response time of one of the location index information is greater than the preset business threshold, and the second response time of one of the location index information is less than the preset business threshold, one of the location index information is rewritten from indicating that the data is stored in the database component to indicating that the data is stored in the cache component.

7. The data reading method according to claim 5, characterized in that: In a case where one of the location index information indicates that data is stored in the cache component, when it is determined that one of the location index information meets the rewrite condition, rewriting the one of the location index information includes: When it is determined that the database score is greater than the cache score, and the first response time of one of the location index information is less than the preset business threshold, and the second response time of one of the location index information is greater than the preset business threshold, one of the location index information is rewritten from indicating that the data is stored in the cache component to indicating that the data is stored in the database component.

8. A data reading device, characterized in that: The data reading device includes a receiving unit, a starting unit, a first reading unit and a second reading unit, wherein: The receiving unit is configured to receive a data query request for data to be queried; The starting unit is configured to start a first query thread and a second query thread in parallel based on the data query request; wherein the first query thread is used to query the data to be queried in the database component, and the second query thread is used to query the data to be queried in the cache component; The first reading unit is configured to, when the first query thread finds the data to be queried, control the first query thread to send a first stop signal to the second query thread and read the data to be queried from the database component; wherein the first stop signal is used to stop the second query thread; The second reading unit is configured to, when the second query thread finds the data to be queried, control the second query thread to send a second stop signal to the first query thread and read the data to be queried from the cache component; wherein the second stop signal is used to stop the first query thread; The data index area stores a plurality of location index information, wherein the location index information is used to indicate whether the corresponding data is stored in the database component or the cache component; the data reading device further includes a rewriting unit; The rewrite unit is configured to determine, in sequence, whether the multiple location index information stored in the data index area meet the rewrite conditions; when it is determined that one of the location index information meets the rewrite conditions, perform a rewrite operation on the one of the location index information; wherein the rewrite operation represents rewriting the location index information from indicating data stored in the database component to indicating data stored in the cache component, or from indicating data stored in the cache component to indicating data stored in the database component.

9. A data reading device, characterized in that The data reading device includes a memory and a processor; wherein, The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the steps of the method according to any one of claims 1 to 7 when running the computer program.

10. A computer storage medium, characterized in that The computer storage medium stores a data reading program, and when the data reading program is executed by at least one processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A data reading system, characterized in that: The data reading system at least includes the data reading device according to claim 8 or 9.

Images