Method and system for solving cache penetration problem, medium and computing equipment

Abstract

The invention relates to the field of big data processing, and discloses a method and system for solving a cache penetration problem, a medium and computing equipment, and the method comprises the following steps: determining a first query result of a client request in a cache system, and judging whether to perform database query according to the first query result; determining a second query result of the client request in the database, and judging whether the query is finished or not according to the second query result; if not, null caching is carried out, and when the client request appears again, direct feedback is carried out by the caching system. The problem that the database pressure is too large can be effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of big data processing, and in particular to a method (cache null method), system, medium and computing device for solving a cache penetration problem. Background Art

[0002] With the development of big data technology, more and more queries need to access the database. As the business develops, enterprises will pay more and more attention to the response time of data queries and the richness of communication data to meet more data indicator requirements and faster interaction speed. When it is necessary to frequently query some data, such as user names, enumeration values, etc., in order to quickly query them, the data is often cached in the memory, so that the user can directly obtain the data from the cache the next time he visits, without frequently requesting the database. This can obtain access data faster and reduce the pressure on the database, such as Figure 2 However, in actual scenarios, the cache system may crash or users may frequently query data that is not in the system. In this case, the query does not read data from the cache, but goes directly to the database for data query, resulting in excessive database pressure. Summary of the invention

[0003] In view of the above problems, the purpose of the present invention is to provide a method, system, medium and computing device for solving the cache penetration problem, which effectively solves the problem of excessive database pressure.

[0004] To achieve the above-mentioned purpose, the present invention adopts the following technical solutions: a method for solving the cache penetration problem, which includes: determining a first query result of a client request in a cache system, and determining whether to perform a database query based on the first query result; determining a second query result of the client request in a database, and determining whether the query is completed based on the second query result; if not completed, caching null, and when the client request appears again, the cache system provides direct feedback.

[0005] Further, determining a first query result of the client request in the cache system, and determining whether to perform a database query according to the first query result, includes: After receiving a client request, query the client request in the cache system, and use the query result as the first query result; Determine whether the first query result is empty. If so, perform a database query.

[0006] Furthermore, when the first query result exists, the query is terminated; otherwise, when the first query result is empty, the client request is queried in the database.

[0007] Further, determining a second query result of the client request in the database, and determining whether the query is finished according to the second query result, includes: Performing a data query in a database based on the client's request, and using the query result as a second query result; Determine whether the second query result is empty. If not, end the query, otherwise proceed to the next step.

[0008] Furthermore, if the request is not completed, the cache is null. When the client request appears again, the cache system will provide direct feedback, including: When the second query result is empty, a new null object is created and stored in the cache system; When the client request appears again, it will be directly queried in the cache system, and the cache system can directly feedback the null object.

[0009] Furthermore, a validity period is set for the null object stored in the cache system, and the null object expires after the validity period has expired.

[0010] Further, the effective time is set to five minutes.

[0011] A system for solving the cache penetration problem includes: a first judgment module, which determines the first query result of a client request in a cache system, and determines whether to perform a database query based on the first query result; a second judgment module, which determines the second query result of the client request in the database, and determines whether the query is completed based on the second query result; a cache null module, which caches null if it is not completed, and when the client request appears again, the cache system directly feedbacks.

[0012] A computer-readable storage medium storing one or more programs includes the one or more programs including instructions, which, when executed by a computing device, cause the computing device to perform any of the above methods.

[0013] A computing device comprises: one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the above methods.

[0014] The present invention adopts the above technical solution, which has the following advantages: 1. The present invention solves the problem of cache penetration and avoids excessive database pressure, which in turn causes a series of problems.

[0015] 2. Compared with the Bloom filter in the prior art, the present invention has an error rate that increases with the number of elements stored in the Bloom filter, while the present invention does not have the error rate problem. In addition, the Bloom filter has the problem of difficulty in deleting elements, while the present invention does not have the problem of difficulty in deleting elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is an overall flow chart of a method for solving the cache penetration problem in an embodiment of the present invention; Figure 2 It is a working principle diagram of the cache system under normal circumstances in the prior art; Figure 3 This is a schematic diagram of a situation where cache penetration occurs in the prior art. DETAILED DESCRIPTION

[0017] The cache system fails or users frequently query information that is not in the cache system. In this case, the query does not read data from the cache, but goes directly to the database for data query, causing excessive pressure on the database.

[0018] For example, a user initiates a request for a data item named lisi, but the data item does not exist in the cache system. Therefore, the data item cannot be found in each query, and the query will be made to the database. Figure 3 As shown in the figure, since lisi itself is not stored in the system, each query returns a null value, causing frequent queries to bypass the cache system and directly request the database. Such high-frequency database queries will cause a lot of database pressure, which may lead to other problems.

[0019] In order to solve the problem of cache penetration, that is, the cache system crashes or users frequently query information that is not in the system, the query does not read data from the cache, but goes directly to the database for data query, resulting in excessive database pressure, which leads to a series of problems. The present invention provides a method, system, medium and computing device for solving the problem of cache penetration, which includes: determining the first query result of a client request in the cache system, and determining whether to perform a database query based on the first query result; determining the second query result of the client request in the database, and determining whether the query is completed based on the second query result; if it is not completed, caching null, and when the client request appears again, the cache system will provide direct feedback.

[0020] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0021] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0022] In one embodiment of the present invention, a method for solving the cache penetration problem is provided. In this embodiment, Figure 1 As shown, the method comprises the following steps: 1) Determine the first query result of the client request in the cache system, and determine whether to perform a database query based on the first query result; 2) determining a second query result of the client's request in the database, and determining whether the query is terminated according to the second query result; 3) If the request is not completed, the cache will be null. When the client request appears again, the cache system will provide direct feedback.

[0023] When the present invention is used, the problem of cache penetration is effectively solved by querying step by step and caching null, thereby overcoming a series of problems caused by excessive database pressure. For example, in an e-commerce system, when a user queries for product details, if the query is a non-existent product ID, the system will normally return that the product does not exist. However, if a user or an attacker repeatedly queries a large number of non-existent product IDs, each query can bypass the cache and directly query the database, and the database will be overloaded due to these invalid query operations. The survival time of the details of a popular product in an e-commerce platform in the cache has expired. Just in the big promotion event, tens of thousands of users query this product information at almost the same time. At this time, the cache has expired, and all requests are directly hit on the back-end database, which may cause the database to be overloaded instantly, respond slowly, or even temporarily unavailable. Suppose a news application uses a cache to store popular news content. If the caches of these popular news fail at the same time at 12 o'clock at night, and this coincides with the user peak, a large number of query requests will hit the database. Since the processing capacity of the database is much lower than that of the cache, a sudden increase may greatly increase the database response time, causing service delays or even service interruptions. Malicious attacks are invalid: hackers will attack the system and use a non-existent ID to query data, which will generate a large number of requests to the database. This may cause the database to crash due to excessive pressure.

[0024] In the above step 1), determining the first query result of the client request in the cache system, and determining whether to perform a database query according to the first query result, includes the following steps: 1.1) After receiving a client request, perform a data query in the cache system for the client request, and use the query result as the first query result; 1.2) Determine whether the first query result is empty. If so, perform a database query; Specifically, when the first query result exists, the query is terminated; otherwise, when the first query result is empty, the customer request is queried in the database.

[0025] In the above step 2), determining the second query result of the client request in the database, and determining whether the query is finished according to the second query result, includes the following steps: 2.1) Query the data requested by the customer in the database, and use the query result as the second query result; 2.2) Determine whether the second query result is empty. If not, end the query, otherwise proceed to the next step.

[0026] In the above step 3), if it is not finished, the cache is null. When the client request appears again, the cache system will directly feedback, including the following steps: 3.1) When the second query result is empty, a new null object is created and stored in the cache system; when a new null object is created, the query data may not exist or the system may fail.

[0027] In this embodiment, a valid time is set for the null object stored in the cache system, and the null object expires after the valid time expires.

[0028] Preferably, the effective time is set to five minutes.

[0029] 3.2) When the client request appears again, it will directly query the cache system, and the cache system can directly feedback the null object.

[0030] In summary, the present invention solves the cache penetration problem by caching null method. The cache null method is: if a query returns null, the null result is still cached, but its expiration time will be very short. When the user queries the data again, null is directly returned without continuing to access the database, which can ensure the security of the database.

[0031] The present invention records in the cache whether a temporary data exists in the system. If it does not exist, null is directly returned without querying the database, thereby avoiding the cache from penetrating into the database.

[0032] The present invention does not have the problem of miscalculation rate, nor does it have the problem of deletion difficulty.

[0033] In one embodiment of the present invention, a system for solving the cache penetration problem is provided, which includes: A first judgment module determines a first query result of a client request in the cache system, and determines whether to perform a database query based on the first query result; A second judgment module determines a second query result of the client's request in the database, and determines whether the query is finished according to the second query result; Cache null module. If it does not end, cache null. When the client request appears again, the cache system will give direct feedback.

[0034] In the above embodiment, determining a first query result of a client request in a cache system, and determining whether to perform a database query according to the first query result, includes: When receiving a client request, querying data in the cache system for the client request, and using the query result as the first query result; Determine whether the first query result is empty. If it is empty, perform a database query.

[0035] In this embodiment, when the first query result exists, the query is terminated; otherwise, when the first query result is empty, the client request is queried in the database.

[0036] In the above embodiment, determining the second query result of the client request in the database, and determining whether the query is finished according to the second query result, includes: Performing a data query in a database based on the client's request, and using the query result as a second query result; Determine whether the second query result is empty. If not, end the query, otherwise proceed to the next step.

[0037] In the above embodiment, if the request is not completed, the cache is null. When the client request appears again, the cache system directly responds, including: When the second query result is empty, a new null object is created and stored in the cache system; When the client request appears again, it will be directly queried in the cache system, and the cache system can directly feedback the null object.

[0038] In this embodiment, a valid time is set for the null object stored in the cache system, and the null object expires after the valid time expires.

[0039] Among them, preferably, the effective time is set to five minutes.

[0040] The system provided in this embodiment is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for specific processes and detailed contents, which will not be repeated here.

[0041] A computing device provided in one embodiment of the present invention may be a terminal, which may include: a processor, a communications interface, a memory, a display screen, and an input device. The processor, the communications interface, and the memory communicate with each other through a communication bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory, wherein the non-volatile storage medium stores an operating system and a computer program, and the computer program is executed by the processor to implement the methods in the above embodiments; the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a management network, NFC (near field communication) or other technologies. The display screen may be a liquid crystal display screen or an electronic ink display screen, and the input device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computing device, or an external keyboard, touchpad or mouse, etc. The processor may call the logic instructions in the memory.

[0042] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0043] In one embodiment of the present invention, a computer program product is provided, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments.

[0044] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores server instructions, and the computer instructions enable a computer to execute the methods provided in the above embodiments.

[0045] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0046] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0047] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0048] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for solving the cache penetration problem, characterized in that: include: Determine a first query result of the client request in the cache system, and determine whether to perform a database query based on the first query result; Determine a second query result of the client request in the database, and determine whether the query is terminated according to the second query result; If it is not finished, the cache will be null. When the client request appears again, the cache system will give direct feedback.

2. The method for solving the cache penetration problem as claimed in claim 1, characterized in that: Determining a first query result of a client request in a cache system, and determining whether to perform a database query according to the first query result, including: After receiving a client request, query the client request in the cache system, and use the query result as the first query result; Determine whether the first query result is empty. If so, perform a database query.

3. The method for solving the cache penetration problem as claimed in claim 2, characterized in that: When the first query result exists, the query is terminated; otherwise, when the first query result is empty, the client request is queried in the database.

4. The method for solving the cache penetration problem as claimed in claim 1, characterized in that: Determining a second query result of the client request in the database, and determining whether the query is terminated according to the second query result, including: Performing a data query in a database based on the client's request, and using the query result as a second query result; Determine whether the second query result is empty. If not, end the query, otherwise proceed to the next step.

5. The method for solving the cache penetration problem as claimed in claim 1, characterized in that: If it is not finished, the cache is null. When the client request appears again, the cache system will give direct feedback, including: When the second query result is empty, a new null object is created and stored in the cache system; When the client request appears again, it will be directly queried in the cache system, and the cache system can directly feedback the null object.

6. The method for solving the cache penetration problem as claimed in claim 5, characterized in that: Set the validity period for the null object stored in the cache system. After the validity period, the null object will expire.

7. The method for solving the cache penetration problem as claimed in claim 6, characterized in that: The validity period is set to five minutes.

8. A system for solving the cache penetration problem, characterized in that: include: A first judgment module determines a first query result of a client request in a cache system, and determines whether to perform a database query based on the first query result; A second judgment module determines a second query result of the client's request in the database, and determines whether the query is finished according to the second query result; Cache null module. If it does not end, cache null. When the client request appears again, the cache system will give direct feedback.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 7.

10. A computing device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods described in claims 1 to 7.

Images