Data acquisition method and device based on multi-level cache, equipment and storage medium

Abstract

The invention discloses a multi-level cache-based data acquisition method and device, equipment and a storage medium, and relates to the technical field of big data, the method comprises the following steps: after a data acquisition request is received, determining a data type and a data identifier of to-be-acquired data by analyzing the data acquisition request; when the data type is static data, acquiring target data corresponding to the data acquisition request in a first cache module; when the data type is dynamic data, target data corresponding to the data obtaining request is obtained in a multi-level cache system, and the multi-level cache system is composed of a second cache module, a local cache and a backup cache. According to the technical scheme, the acquisition speed of the static data and the dynamic data is increased, and the static data and part of the dynamic data are acquired only in the reverse proxy servers corresponding to the first cache module and the second cache module, so that the access pressure of a back-end server is reduced, and the access efficiency and the access stability are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of big data technology, and in particular to a data acquisition method, device, equipment and storage medium based on multi-level cache. Background Art

[0002] Multi-level caching is an architectural strategy for optimizing data acquisition performance. It reduces direct access to backend data sources (such as databases) by setting up caches at different levels, thereby speeding up data acquisition.

[0003] In the prior art, the commonly used multi-level cache system is composed of a client cache, a middleware cache and a database. Once the middleware fails and the cache data becomes invalid, a large number of data requests will impact the database and affect the normal operation of the database. Summary of the invention

[0004] The present invention provides a data acquisition method, device, equipment and storage medium based on multi-level cache, so as to improve the efficiency and stability of data acquisition.

[0005] In a first aspect, an embodiment of the present invention provides a data acquisition method based on a multi-level cache, comprising:

[0006] After receiving the data acquisition request, determining the data type and data identifier of the data to be acquired by parsing the data acquisition request;

[0007] When the data type is static data, obtaining target data corresponding to the data acquisition request in the first cache module according to the data identifier;

[0008] When the data type is dynamic data, the target data corresponding to the data acquisition request is acquired in a multi-level cache system of dynamic data according to the data identifier, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache.

[0009] The technical solution of an embodiment of the present invention provides a data acquisition method based on multi-level cache, including: after receiving a data acquisition request, determining the data type and data identifier of the data to be acquired by parsing the data acquisition request; when the data type is static data, acquiring the target data corresponding to the data acquisition request in a first cache module according to the data identifier; when the data type is dynamic data, acquiring the target data corresponding to the data acquisition request in a multi-level cache system of dynamic data according to the data identifier, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache. The above technical scheme can first parse the received data acquisition request. Through the analysis, the data type and data identifier of the data to be acquired can be determined. Data of different data types are stored in different locations. Therefore, the determination of the data type improves the speed and efficiency of data acquisition. When the data type of the data to be acquired is determined to be static data, the target data of the data acquisition request data can be determined in the first cache module according to the data identifier of the data to be acquired, thereby improving the acquisition speed of static data. When the data type of the data to be acquired is determined to be dynamic data, data can be searched in each cache of the multi-level cache storing dynamic data according to the data identifier of the data to be acquired until the target data corresponding to the data identifier is found, thereby improving the acquisition speed of dynamic data. In addition, the acquisition of static data and part of dynamic data only needs to be performed on the reverse proxy servers corresponding to the first cache module and the second cache module, which reduces the access pressure of the back-end server and further improves the access efficiency and access stability.

[0010] Furthermore, it also includes:

[0011] After receiving the data storage request, determining the data type of the data to be stored by parsing the data storage request;

[0012] When the data type is static data, storing the data to be stored in the first cache module;

[0013] When the data type is dynamic data, the data to be stored is stored in the second cache module and the backup cache, the amount of data stored in the second cache module is monitored, and when the data amount reaches a storage threshold, the target dynamic data is determined according to the prediction strategy, and the target dynamic data is moved into the local cache.

[0014] Furthermore, after storing the data to be stored in the second cache module, the method further includes:

[0015] Monitor the number of accesses to each dynamic data stored in the second cache module.

[0016] Furthermore, target dynamic data is determined according to the prediction strategy, including:

[0017] Sorting the dynamic data in descending order of the number of accesses;

[0018] The dynamic data sorted after the M dynamic data is determined as the target dynamic data, where M is the storage threshold.

[0019] Further, acquiring target data corresponding to the data acquisition request in a multi-level cache system of dynamic data according to the data identifier includes:

[0020] Searching for data corresponding to the data identifier in the second cache module and the local cache in sequence;

[0021] If the search result is failure, the target data corresponding to the data acquisition request is determined in the backup cache based on the data identifier.

[0022] Further, searching the second cache module and the local cache for data corresponding to the data identifier in sequence includes:

[0023] Searching in the second cache module based on the data identifier;

[0024] If the data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the data is searched in the local cache based on the data identifier;

[0025] If the data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the data is searched in the backup cache based on the data identifier;

[0026] If the data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the search result is determined to be a failure.

[0027] In a second aspect, an embodiment of the present invention further provides a data acquisition device based on a multi-level cache, comprising:

[0028] A parsing module, configured to determine the data type and data identifier of the data to be acquired by parsing the data acquisition request after receiving the data acquisition request;

[0029] A first acquisition module, configured to acquire, when the data type is static data, target data corresponding to the data acquisition request in a first cache module according to the data identifier;

[0030] The second acquisition module is used to acquire the target data corresponding to the data acquisition request in the multi-level cache system of dynamic data according to the data identifier when the data type is dynamic data, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache.

[0031] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0032] at least one processor; and a memory communicatively coupled to the at least one processor;

[0033] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-level cache-based data acquisition method as described in any one of the first aspects.

[0034] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute a multi-level cache-based data acquisition method as described in any one of the first aspects.

[0035] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed on a computer, the computer executes the multi-level cache-based data acquisition method provided in the first aspect.

[0036] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the data acquisition device based on the multi-level cache, or may be packaged separately from the processor of the data acquisition device based on the multi-level cache, and this application does not limit this.

[0037] The description of the second, third, fourth and fifth aspects in this application can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0038] In this application, the name of the above-mentioned multi-level cache-based data acquisition device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalent technologies.

[0039] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A flowchart of a data acquisition method based on multi-level cache provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of another data acquisition method based on multi-level cache provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the structure of a data acquisition device based on multi-level cache provided by an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0046] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0047] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0048] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0049] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0050] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0051] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0052] Figure 1 A flowchart of a data acquisition method based on a multi-level cache is provided in an embodiment of the present invention. This embodiment is applicable to situations where the efficiency and stability of data acquisition need to be improved. The method can be executed by a data acquisition device based on a multi-level cache, such as Figure 1 As shown, the specific steps include:

[0053] Step 110: After receiving the data acquisition request, determine the data type and data identifier of the data to be acquired by parsing the data acquisition request.

[0054] Among them, the data acquisition request is a request sent by a user terminal or other device for data acquisition. The data acquisition request can indicate the data type and data identifier of the data to be acquired. The data type can be static data or dynamic data, and the data identifier can be understood as a unique identifier of the data.

[0055] Specifically, after receiving a data acquisition request sent by a user terminal or other device, the data acquisition request can be parsed to determine the data type and data identifier of the data to be acquired, that is, it can be determined whether the data to be acquired is static data or dynamic data, and the unique identifier of the data to be acquired can also be determined.

[0056] In the embodiment of the present invention, the data type and data identifier of the data to be acquired can be determined by parsing the received data acquisition request. Data of different data types are stored in different locations. Therefore, the determination of the data type improves the speed and efficiency of data acquisition.

[0057] Step 120: When the data type is static data, the target data corresponding to the data acquisition request is acquired in the first cache module according to the data identifier.

[0058] Among them, static data usually refers to data that remains relatively fixed and does not change frequently within a certain period of time.

[0059] In most distributed architectures, the reverse proxy server is the first node where external requests arrive. The first cache module can be the local cache of the reverse proxy server. When the user terminal or other device initiates the same request to obtain the data stored in the local cache of the reverse proxy server, the reverse proxy server can directly extract the data from the local cache and respond to the user terminal or other device without having to obtain it from the back-end server (such as application server, database, etc.), which greatly improves the response speed, reduces the load pressure on the back-end server, and improves the performance and user experience of the entire website or application.

[0060] If the static data is stored in the backend server, the acquisition of the static data also needs to be based on the backend server, which will bring unnecessary performance consumption. Therefore, the static data can be stored in the first cache module.

[0061] Specifically, when it is determined that the data type of the data to be obtained is static data, it can be determined that the data to be obtained is stored in the first cache module. Therefore, data search can be performed in the first cache module according to the data identifier of the data to be obtained, and the data corresponding to the data identifier in the first cache module is determined as the target data corresponding to the data acquisition request, so that the reverse proxy server closer to the device that issues the data acquisition request returns the static data corresponding to the data acquisition request, thereby improving the response speed and alleviating the access pressure on the background server.

[0062] In the embodiment of the present invention, the static data acquired by the data acquisition request is determined in the first cache module, so that the acquisition speed of the static data is improved and the access pressure of the background server is reduced.

[0063] Step 130: When the data type is dynamic data, the target data corresponding to the data acquisition request is acquired in a multi-level cache system of dynamic data according to the data identifier.

[0064] The multi-level cache system is composed of a second cache module, a local cache and a backup cache.

[0065] The second cache module can provide a mechanism for sharing data between multiple worker processes of the reverse proxy server. The second cache module can use a key-value pair storage structure to store data to improve the reading speed. In addition, the second cache module manages cache or memory resources of limited capacity based on the least used elimination rule. The second cache module is used to store hot data that will be frequently accessed. When the storage capacity of the second cache module reaches the storage threshold and space needs to be freed up to store new data, the least used elimination rule can select the least frequently used hot data for removal.

[0066] The local cache can be understood as the local cache of the backend server, which is used to store the hot data removed from the second cache module, and the backup cache is used to back up all dynamic data.

[0067] For dynamic data, first, it can be determined whether it is hot data based on the number of accesses. After it is determined to be hot data, it is stored in the second cache module and the backup cache. By storing the hot data in the second cache module, the cache and read and write efficiency of the hot data can be improved. After it is determined that it is not hot data, it is stored in the backup cache. Of course, when it is determined that the amount of data stored in the second cache module exceeds the storage threshold, the data with the least number of accesses can be removed to the local cache to ensure that the hot data with higher access volume is always stored in the second cache module, further improving the response speed.

[0068] Specifically, when it is determined that the data type of the data to be acquired is dynamic data, it can be determined that the data to be acquired is stored in a multi-level cache system, and then data search can be performed in each level of cache in the multi-level cache system based on the data identifier of the data to be acquired until the target data corresponding to the data identifier is found.

[0069] In the embodiment of the present invention, dynamic data acquired by a data acquisition request is determined in a multi-level cache, thereby improving the acquisition speed of the dynamic data.

[0070] The data acquisition method based on multi-level cache provided in an embodiment of the present invention includes: after receiving a data acquisition request, determining the data type and data identifier of the data to be acquired by parsing the data acquisition request; when the data type is static data, acquiring the target data corresponding to the data acquisition request in the first cache module according to the data identifier; when the data type is dynamic data, acquiring the target data corresponding to the data acquisition request in the multi-level cache system of dynamic data according to the data identifier, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache. The above technical scheme can first parse the received data acquisition request. Through the analysis, the data type and data identifier of the data to be acquired can be determined. Data of different data types are stored in different locations. Therefore, the determination of the data type improves the speed and efficiency of data acquisition. When the data type of the data to be acquired is determined to be static data, the target data of the data acquisition request data can be determined in the first cache module according to the data identifier of the data to be acquired, thereby improving the acquisition speed of static data. When the data type of the data to be acquired is determined to be dynamic data, data can be searched in each cache of the multi-level cache storing dynamic data according to the data identifier of the data to be acquired until the target data corresponding to the data identifier is found, thereby improving the acquisition speed of dynamic data. In addition, the acquisition of static data and part of dynamic data only needs to be performed on the reverse proxy servers corresponding to the first cache module and the second cache module, which reduces the access pressure of the back-end server and further improves the access efficiency and access stability.

[0071] Figure 2 The flowchart of another data acquisition method based on multi-level cache provided by the embodiment of the present invention is specific based on the above embodiment. Figure 2 As shown, in this embodiment, the method may further include:

[0072] Step 210: After receiving the data storage request, determine the data type of the data to be stored by parsing the data storage request.

[0073] The data storage request is a request for data storage sent by a user terminal or other device, and the data storage request may include data to be stored.

[0074] Specifically, after receiving the data storage request, the data storage request can be parsed to obtain the data to be stored, and then the data type of the data to be stored can be determined, that is, it can be determined whether the data to be stored is static data or dynamic data.

[0075] In the embodiment of the present invention, the data type of the data to be stored is determined by parsing the data storage request.

[0076] Step 220: When the data type is static data, the data to be stored is stored in the first cache module.

[0077] When the reverse proxy server of the distributed architecture is an Nginx server, the first cache module may be an Nginx cache.

[0078] Specifically, in order to improve the response speed and reduce the pressure on the backend server, the static data can be stored in the Nginx cache, and this can be achieved by replacing the static access path (the storage address of the static data in the Nginx cache) with a specified static path.

[0079] In the embodiment of the present invention, static data is stored in the Nginx cache.

[0080] Step 230: When the data type is dynamic data, the data to be stored is stored in the second cache module and the backup cache, the amount of data stored in the second cache module is monitored, and when the data amount reaches a storage threshold, the target dynamic data is determined according to the prediction strategy, and the target dynamic data is moved into the local cache.

[0081] Among them, the multi-level cache system is composed of a second cache module, a local cache and a backup cache. When the reverse proxy server is an Nginx server, the second cache module can be an Nginx shared dictionary. In the multi-level cache system, the Nginx shared dictionary stores hot data with high access volume, the local cache stores other hot data, and the backup cache stores all dynamic data.

[0082] In one embodiment, after storing the data to be stored in the second cache module, the method further includes:

[0083] Monitor the number of accesses to each dynamic data stored in the second cache module.

[0084] Accordingly, target dynamic data is determined according to the prediction strategy, including:

[0085] The dynamic data are sorted in descending order of the number of accesses; and the dynamic data sorted after the M dynamic data are determined as the target dynamic data, where M is the storage threshold.

[0086] Specifically, after determining that the data type of the data to be stored is dynamic data, the data to be stored can first be stored in the Nginx shared dictionary and the backup cache. As the amount of data stored in the Nginx shared dictionary increases, when it is determined that the amount of data stored in the Nginx shared dictionary reaches the storage threshold, the target dynamic data is determined according to the prediction strategy. Specifically, the dynamic data in the Nginx shared dictionary can be sorted from large to small according to the number of visits to them, and the dynamic data exceeding the storage threshold can be determined as the target dynamic data. When the storage threshold is M, the dynamic data sorted after M dynamic data can be determined as the target dynamic data, and then the target dynamic data can be moved into the local cache.

[0087] Of course, if the access volume of dynamic data stored in the local cache is less than the access volume threshold after a preset time period, it can be moved out of the local cache. Since these dynamic data are also stored in the backup cache, moving them out of the local cache will not affect normal access to them, and can also reduce the cache volume of the local cache, further improving access efficiency.

[0088] In the embodiment of the present invention, storage of dynamic data is achieved based on a multi-level cache system.

[0089] Step 240: After receiving the data acquisition request, determine the data type and data identifier of the data to be acquired by parsing the data acquisition request.

[0090] Specifically, after receiving the data acquisition request, the data acquisition request can be parsed to determine the data type and data identifier of the data to be acquired, that is, to determine whether the data to be acquired is static data or dynamic data, and to determine the unique identifier of the data to be acquired.

[0091] In the embodiment of the present invention, the data type and data identifier of the data to be acquired can be determined by parsing the received data acquisition request. Data of different data types are stored in different locations. Therefore, the determination of the data type improves the speed and efficiency of data acquisition.

[0092] Step 250: When the data type is static data, the target data corresponding to the data acquisition request is acquired in the first cache module according to the data identifier.

[0093] When the data type of the data to be acquired is determined to be static data as described in the first embodiment, data can be searched in the Nginx cache according to the data identifier of the data to be acquired, and the data corresponding to the data identifier in the Nginx cache is determined as the target data corresponding to the data acquisition request, thereby determining the target data.

[0094] In the embodiment of the present invention, it is achieved that static data obtained by a data acquisition request is determined in an Nginx cache, the acquisition speed of the static data is improved, and the access pressure of a background server is reduced.

[0095] Step 260: When the data type is dynamic data, the target data corresponding to the data acquisition request is acquired in a multi-level cache system of dynamic data according to the data identifier.

[0096] In one implementation, obtaining target data corresponding to the data acquisition request in a multi-level cache system for dynamic data according to the data identifier includes:

[0097] Search the second cache module and the local cache for data corresponding to the data identifier in turn; if the search result is failure, determine the target data corresponding to the data acquisition request in the backup cache based on the data identifier.

[0098] Further, searching the second cache module and the local cache for data corresponding to the data identifier in sequence includes:

[0099] A search is performed in the second cache module based on the data identifier; if data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, a search is performed in the local cache based on the data identifier; if data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, a search is performed in the backup cache based on the data identifier; if data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the search result is determined to be a failure.

[0100] Specifically, when it is determined that the data type of the data to be obtained is dynamic data, it can be determined that the data to be obtained is stored in a multi-level cache system, and then the data can be searched in each level of cache in the multi-level cache system based on the data identifier of the data to be obtained. Specifically, first, the Nginx shared dictionary can be searched based on the data identifier of the data to be obtained. If the data is found, the found data is determined as the target data corresponding to the data acquisition request. If the data is not found, the local cache can be searched based on the data identifier of the data to be obtained. If the data is found, the found data is determined as the target data corresponding to the data acquisition request. If the data is not found, the backup cache can be searched based on the data identifier of the data to be obtained; if the data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the search result is determined to be a failure.

[0101] In the embodiment of the present invention, dynamic data acquired by a data acquisition request is determined in a multi-level cache, thereby improving the acquisition speed of the dynamic data.

[0102] The data acquisition method based on multi-level cache provided by an embodiment of the present invention includes: after receiving a data storage request, determining the data type of the data to be stored by parsing the data storage request; when the data type is static data, storing the data to be stored in the first cache module; when the data type is dynamic data, storing the data to be stored in the second cache module and the backup cache, monitoring the amount of data stored in the second cache module, and determining the target dynamic data according to the prediction strategy when the data amount reaches the storage threshold, and moving the target dynamic data into the local cache; after receiving a data acquisition request, determining the data type and data identifier of the data to be acquired by parsing the data acquisition request; when the data type is static data, acquiring the target data corresponding to the data acquisition request in the first cache module according to the data identifier; when the data type is dynamic data, acquiring the target data corresponding to the data acquisition request in the multi-level cache system of dynamic data according to the data identifier. The above technical solution parses the received data storage request. Through the analysis, the data type of the data to be stored can be determined. The different locations of data storage of data of different data types. Therefore, the determination of the data type improves the speed and efficiency of data storage. When the data type of the data to be stored is determined to be static data, the data to be stored can be stored in the first cache module. When the data type of the data to be stored is determined to be dynamic data, the dynamic data can be stored based on a multi-level cache system of the second cache module + local cache + backup cache to realize partitioned storage of data. After receiving the data acquisition request, the received data acquisition request can be parsed first. Through the analysis, the data type and data identifier of the data to be acquired can be determined. Data of different data types are stored in different The same position, therefore, the determination of the data type improves the speed and efficiency of data acquisition. When it is determined that the data type of the data to be acquired is static data, the target data of the data acquisition request data can be determined in the first cache module according to the data identifier of the data to be acquired, thereby improving the acquisition speed of static data. When it is determined that the data type of the data to be acquired is dynamic data, data can be searched in each cache of the multi-level cache storing dynamic data according to the data identifier of the data to be acquired until the target data corresponding to the data identifier is found, thereby improving the acquisition speed of dynamic data. In addition, the acquisition of static data and part of dynamic data only needs to be performed on the reverse proxy servers corresponding to the first cache module and the second cache module, which reduces the access pressure of the back-end server and further improves the access efficiency and access stability.

[0103] Figure 3A schematic diagram of a data acquisition device based on a multi-level cache provided in an embodiment of the present invention is provided. The device can be applied to situations where the efficiency and stability of data acquisition need to be improved. The device can be implemented by software and / or hardware and is generally integrated in an electronic device, such as a computer device.

[0104] like Figure 3 As shown, the device comprises:

[0105] The parsing module 310 is used to determine the data type and data identifier of the data to be obtained by parsing the data acquisition request after receiving the data acquisition request;

[0106] A first acquisition module 320, configured to acquire target data corresponding to the data acquisition request in a first cache module according to the data identifier when the data type is static data;

[0107] The second acquisition module 330 is used to acquire the target data corresponding to the data acquisition request in the multi-level cache system of dynamic data according to the data identifier when the data type is dynamic data, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache.

[0108] The data acquisition device based on multi-level cache provided in this embodiment determines the data type and data identifier of the data to be acquired by parsing the data acquisition request after receiving a data acquisition request; when the data type is static data, the target data corresponding to the data acquisition request is acquired in the first cache module according to the data identifier; when the data type is dynamic data, the target data corresponding to the data acquisition request is acquired in the multi-level cache system of dynamic data according to the data identifier, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache. The above technical scheme can first parse the received data acquisition request. Through the analysis, the data type and data identifier of the data to be acquired can be determined. Data of different data types are stored in different locations. Therefore, the determination of the data type improves the speed and efficiency of data acquisition. When the data type of the data to be acquired is determined to be static data, the target data of the data acquisition request data can be determined in the first cache module according to the data identifier of the data to be acquired, thereby improving the acquisition speed of static data. When the data type of the data to be acquired is determined to be dynamic data, data can be searched in each cache of the multi-level cache storing dynamic data according to the data identifier of the data to be acquired until the target data corresponding to the data identifier is found, thereby improving the acquisition speed of dynamic data. In addition, the acquisition of static data and part of dynamic data only needs to be performed on the reverse proxy servers corresponding to the first cache module and the second cache module, which reduces the access pressure of the back-end server and further improves the access efficiency and access stability.

[0109] Based on the above embodiment, the device further includes:

[0110] A storage module is used to, after receiving a data storage request, determine the data type of the data to be stored by parsing the data storage request; when the data type is static data, store the data to be stored in the first cache module; when the data type is dynamic data, store the data to be stored in the second cache module and the backup cache, monitor the amount of data stored in the second cache module, and when the data amount reaches a storage threshold, determine the target dynamic data according to the prediction strategy, and move the target dynamic data into the local cache.

[0111] Based on the above embodiment, the storage module is further used for:

[0112] After the data to be stored is stored in the second cache module, the number of accesses to each dynamic data stored in the second cache module is monitored.

[0113] Accordingly, target dynamic data is determined according to the prediction strategy, including:

[0114] The dynamic data are sorted in descending order of the number of accesses; and the dynamic data sorted after the M dynamic data are determined as the target dynamic data, where M is the storage threshold.

[0115] Based on the above embodiment, the second acquisition module 330 is specifically used for:

[0116] When the data type is dynamic data, the data corresponding to the data identifier is searched in the second cache module and the local cache in turn; if the search result is failure, the target data corresponding to the data acquisition request is determined in the backup cache based on the data identifier.

[0117] In one implementation, searching the second cache module and the local cache for data corresponding to the data identifier in sequence includes:

[0118] A search is performed in the second cache module based on the data identifier; if data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, a search is performed in the local cache based on the data identifier; if data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, a search is performed in the backup cache based on the data identifier; if data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the search result is determined to be a failure.

[0119] The multi-level cache-based data acquisition device provided in the embodiment of the present invention can execute the multi-level cache-based data acquisition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the multi-level cache-based data acquisition method.

[0120] It is worth noting that in the above-mentioned embodiment of the data acquisition device based on multi-level cache, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0121] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 4 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0122] like Figure 4 As shown, the electronic device 4 is in the form of a general purpose computing electronic device. The components of the electronic device 4 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0123] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0124] The electronic device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 4, including volatile and non-volatile media, removable and non-removable media.

[0125] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0126] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0127] The electronic device 4 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the electronic device 4, and / or any device that enables the electronic device 4 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 4 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. Figure 4 As shown, the network adapter 20 communicates with other modules of the electronic device 4 via the bus 18. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 4, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0128] The processing unit 16 executes various functional applications and page displays by running the programs stored in the system memory 28, for example, implementing the data acquisition method based on multi-level cache provided in an embodiment of the present invention, the method comprising:

[0129] After receiving the data acquisition request, determining the data type and data identifier of the data to be acquired by parsing the data acquisition request;

[0130] When the data type is static data, obtaining target data corresponding to the data acquisition request in the first cache module according to the data identifier;

[0131] When the data type is dynamic data, the target data corresponding to the data acquisition request is acquired in a multi-level cache system of dynamic data according to the data identifier, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache.

[0132] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the data acquisition method based on multi-level cache provided by any embodiment of the present invention.

[0133] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a data acquisition method based on a multi-level cache provided in an embodiment of the present invention is implemented. The method includes:

[0134] After receiving the data acquisition request, determining the data type and data identifier of the data to be acquired by parsing the data acquisition request;

[0135] When the data type is static data, obtaining target data corresponding to the data acquisition request in the first cache module according to the data identifier;

[0136] When the data type is dynamic data, the target data corresponding to the data acquisition request is acquired in a multi-level cache system of dynamic data according to the data identifier, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache.

[0137] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0138] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0139] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0140] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0142] In addition, the acquisition, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations.

[0143] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data acquisition method based on multi-level cache, characterized in that: include: After receiving the data acquisition request, determining the data type and data identifier of the data to be acquired by parsing the data acquisition request; When the data type is static data, obtaining target data corresponding to the data acquisition request in the first cache module according to the data identifier; When the data type is dynamic data, the target data corresponding to the data acquisition request is acquired in a multi-level cache system of dynamic data according to the data identifier, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache.

2. The data acquisition method based on multi-level cache according to claim 1, characterized in that: Also includes: After receiving the data storage request, determining the data type of the data to be stored by parsing the data storage request; When the data type is static data, storing the data to be stored in the first cache module; When the data type is dynamic data, the data to be stored is stored in the second cache module and the backup cache, the amount of data stored in the second cache module is monitored, and when the data amount reaches a storage threshold, the target dynamic data is determined according to the prediction strategy, and the target dynamic data is moved into the local cache.

3. The data acquisition method based on multi-level cache according to claim 2, characterized in that: After storing the data to be stored in the second cache module, the method further includes: Monitor the number of accesses to each dynamic data stored in the second cache module.

4. The data acquisition method based on multi-level cache according to claim 3 is characterized in that: Determine target dynamic data based on prediction strategy, including: Sorting the dynamic data in descending order of the number of accesses; The dynamic data sorted after the M dynamic data is determined as the target dynamic data, where M is the storage threshold.

5. The data acquisition method based on multi-level cache according to claim 1, characterized in that: Acquiring target data corresponding to the data acquisition request in a multi-level cache system for dynamic data according to the data identifier includes: Searching for data corresponding to the data identifier in the second cache module and the local cache in sequence; If the search result is failure, the target data corresponding to the data acquisition request is determined in the backup cache based on the data identifier.

6. The data acquisition method based on multi-level cache according to claim 5, characterized in that: Searching for data corresponding to the data identifier in the second cache module and the local cache in sequence includes: Searching in the second cache module based on the data identifier; If the data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the data is searched in the local cache based on the data identifier; If the data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the data is searched in the backup cache based on the data identifier; If the data is found, the found data is determined as the target data corresponding to the data acquisition request; otherwise, the search result is determined to be a failure.

7. A data acquisition device based on multi-level cache, characterized in that: include: A parsing module, configured to determine the data type and data identifier of the data to be acquired by parsing the data acquisition request after receiving the data acquisition request; A first acquisition module, configured to acquire, when the data type is static data, target data corresponding to the data acquisition request in a first cache module according to the data identifier; The second acquisition module is used to acquire the target data corresponding to the data acquisition request in the multi-level cache system of dynamic data according to the data identifier when the data type is dynamic data, wherein the multi-level cache system is composed of a second cache module, a local cache and a backup cache.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-level cache-based data acquisition method as described in any one of claims 1-6.

9. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the multi-level cache-based data acquisition method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the data acquisition method based on a multi-level cache according to any one of claims 1 to 6.

Images