Method for realizing concurrent read-write and self-defined sorting functions by utilizing redis cache algorithm

Abstract

The invention provides a method for realizing concurrent read-write and self-defined sorting functions by using a redis cache algorithm, the implementation method comprises two parts of middleware selection and algorithm selection, in the middleware selection process, redis is selected to perform data storage and query, and the redis comprises a single redis or a redis 3 master 3 slave cluster scheme; in the algorithm selection process, the definition of the score value of the zset in the redis data structure is utilized to realize various sorting and top setting functions. According to the method, a zset data structure of middleware redis is utilized to realize a brand-new dynamic cache structure, and real-time paging query and high-concurrency read-write can be carried out, so that the problems of an original cache system are thoroughly solved. All target data can be subjected to unified cache management, cache breakdown is avoided, unified management is facilitated, the operation and maintenance cost is greatly reduced, the cost of middleware adopting an open source scheme is zero, the server requirement is low, no special requirement for hardware such as a CPU and a hard disk exists, and the method is suitable for being used in small and medium-sized projects.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a method for realizing concurrent reading and writing and custom sorting functions by utilizing a redis cache algorithm. Background Art

[0002] In general, the homepage recommendation list, product list, video list, circle of friends and other content are not highly personalized content. The content seen by all users is basically the same or is cached information classified based on user tags.

[0003] The cache design based on the above type in the existing technology will lead to the following disadvantages: 1. The cache is not real-time enough. Information needs to be updated in batches at regular intervals so that users can refresh new content. In addition, the system load is high during batch refresh, which affects the user access speed. 2. In order to avoid cache breakdown, the cache will be made more complex and bloated, and it must be used with multiple loads, which increases the complexity of system development and the cost of subsequent maintenance. Especially in small and medium-sized systems, complex cache systems are a nightmare for subsequent maintenance. 3. Since it is an active cache, the cache content is basically within a small range. If the query content exceeds the range, the performance will drop significantly and the system response time will increase. 4. Some systems have also introduced niche middleware to solve the cache problem, but because it is not a commonly used technology in the industry, the learning cost is high. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for realizing concurrent reading and writing and custom sorting functions by using the redis cache algorithm, so as to solve the problems raised in the above background technology. The present invention utilizes the zset data structure of the middleware redis to realize a new dynamic cache structure, and can perform real-time paging query and high concurrent reading and writing, completely solving the problems of the original cache system.

[0005] In order to achieve the above-mentioned purpose, the present invention is implemented through the following technical scheme: a method for realizing concurrent reading and writing and custom sorting functions by using redis cache algorithm, the implementation method includes two parts: middleware selection and algorithm selection. In the middleware selection process, redis is selected for data storage and query, and redis includes a single redis or a redis 3 master 3 slave cluster solution; in the algorithm selection process, the definition of the score value of zset in the redis data structure is used to realize various sorting and top functions.

[0006] Furthermore, it includes utilizing redis's own high-speed reading and writing capabilities to support high concurrency requirements.

[0007] Furthermore, it also includes sorting and pinning, waterfall paging query, data division of different topics or categories, and high-concurrency reading and writing.

[0008] Furthermore, in the process of sorting and pinning, the reverse order of the timestamps of product release is used as the sorting value, and newly released products are displayed first by default. If a product needs to be recommended and pinned for n days, the pinning is achieved by adding its timestamp + the number of milliseconds corresponding to n days. After n days, newly released products with a timestamp greater than the pinning timestamp will be displayed first.

[0009] Furthermore, the results of waterfall paging queries will be used in high-concurrency areas such as the home page or recommendation lists.

[0010] Furthermore, a query combination of "timestamp + size of the last record smaller than the currently displayed result" is used to prevent users from viewing duplicate data.

[0011] Furthermore, the division of data into different topics or categories is achieved by establishing different zset data.

[0012] Furthermore, the high-concurrency reading and writing is based on the new paging method and the score sorting rules of zset, and data reading and writing are completely completed by redis.

[0013] Beneficial effects of the present invention:

[0014] 1. This method of using the redis cache algorithm to achieve concurrent reading and writing and custom sorting functions uses the zset data structure of the middleware redis to achieve a new dynamic cache structure, and can perform real-time paging query and high concurrent reading and writing, completely solving the problems of the original cache system.

[0015] 2. This method can uniformly cache all target data, avoid cache breakdown, and facilitate unified management, greatly reducing operation and maintenance costs, and improving high-concurrency read and write performance.

[0016] 3. The overall learning cost of this method is basically 0, the maintenance cost is extremely low, the cost of the middleware using open source solution is 0, the server requirements are low, you only need to pay attention to the memory to meet business needs, there are no special requirements for hardware such as CPU and hard disk, the overall cost is extremely low, and it is suitable for small and medium-sized projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of the method for realizing concurrent reading and writing and custom sorting functions by using the redis cache algorithm of the present invention. DETAILED DESCRIPTION

[0018] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0019] See also Figure 1 The present invention provides the following technical solutions: a method for realizing concurrent reading and writing and custom sorting functions by using the redis cache algorithm. The method utilizes the zset data structure of the middleware redis to realize a brand-new dynamic cache structure, and can perform real-time paging query and high concurrent reading and writing, which completely solves the problems of the original cache system. And the target data falls into the cache in real time, and no longer needs to be refreshed regularly. The target data is only queried in the cache, and no database query is performed, which eliminates cache breakdown and avoids the complexity caused by multiple loads. If the cache is disordered due to bugs and other reasons, it only needs to re-establish the cache once. The target data supports dynamic paging query and parameter query. Through different groupings, multi-list, multi-target, and multi-source data query can be realized, and high efficiency can be maintained at any time. And it can achieve zero learning cost, good performance, and low maintenance cost.

[0020] This embodiment also provides a specific implementation scheme of the above method:

[0021] Middleware selection: Select redis for data storage and query. You can choose a single redis or a redis 3-master 3-slave cluster solution. Use redis's own high-speed reading and writing capabilities to support high concurrency requirements. The server requirements are very low, and even a low-end server can achieve the function. Compared with the existing high-cost solutions that require multiple high-end servers and complex middleware, the economic and learning costs are greatly reduced.

[0022] Algorithm selection: Use the definition of the score value of zset in the redis data structure to implement various sorting, top-level functions, etc.

[0023] This embodiment also provides a simple recommendation list waterfall flow to illustrate the above technical solution, and its content is as follows:

[0024] The waterfall flow list of products on the mobile terminal of a small points mall, based on the customer's requirements for fluency, cooperates with Alibaba Cloud OSS to store product images and perform the following process:

[0025] 1. Sorting and pinning:

[0026] First, use the reverse order of the product release timestamp as the sorting value. By default, newly released products are given priority. If a product needs to be recommended and pinned to the top for n days, you can add the number of milliseconds corresponding to its timestamp + n days to achieve the pinning operation. After n days, the newly released product will have a timestamp greater than the pinning timestamp and will naturally be displayed first.

[0027] 2. Waterfall flow paging query:

[0028] Since the query results will be used in high-concurrency areas such as the homepage or recommendation list, if duplicate data appears in the two paging results, it will greatly affect the user's viewing experience. Therefore, the traditional "page+size" query combination is abandoned and the "timestamp of the last record smaller than the currently displayed result+size" query combination is adopted to ensure that users will not see duplicate data and the query efficiency is extremely high.

[0029] 3. Data of different topics and categories:

[0030] You just need to create different zset data.

[0031] 4. High concurrent reading and writing:

[0032] Based on the new paging method and the score sorting rules of zset, data reading and writing are completely completed by redis, the business system is stress-free, and cache breakdown and database downfall are prevented.

[0033] Through the above solution, it is measured that under 200qps, the Internet return time of the waterfall paging query interface is less than 1 second, thus meeting the user's requirements of continuous finger movement and waterfall flow.

[0034] The basic principles and main features of the present invention and the advantages of the present invention are shown and described above. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0035] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for realizing concurrent reading and writing and custom sorting functions by using redis cache algorithm, characterized in that: The implementation method includes two parts: middleware selection and algorithm selection. In the middleware selection process, redis is selected for data storage and query, and redis includes a single redis or a redis 3-master 3-slave cluster solution; in the algorithm selection process, the definition of the score value of zset in the redis data structure is used to implement various sorting and top functions.

2. According to claim 1, the method for realizing concurrent reading and writing and custom sorting functions by using the redis cache algorithm is characterized in that: This includes leveraging redis's own high-speed reading and writing capabilities to support high concurrency requirements.

3. According to claim 1, the method for realizing concurrent reading and writing and custom sorting functions by using the redis cache algorithm is characterized in that: It also includes sorting and pinning, waterfall paging query, data division of different topics or categories, and high-concurrency reading and writing.

4. According to claim 3, the method for realizing concurrent reading and writing and custom sorting functions by using the redis cache algorithm is characterized in that: During the sorting and pinning process, the reverse order of the product release timestamps is used as the sorting value. By default, newly released products are given priority. If a product needs to be recommended and pinned for n days, the pinning is achieved by adding its timestamp to the number of milliseconds corresponding to n days. After n days, newly released products with a timestamp greater than the pinning timestamp will be given priority.

5. According to claim 3, the method for realizing concurrent reading and writing and custom sorting functions by using the redis cache algorithm is characterized in that: The results of waterfall paging queries will be used in high-concurrency areas such as the home page or recommendation list.

6. The method for realizing concurrent reading and writing and custom sorting functions by using redis cache algorithm according to claim 5 is characterized in that: The query combination of "timestamp + size of the last record smaller than the currently displayed result" is used to prevent users from viewing duplicate data.

7. The method for realizing concurrent reading and writing and custom sorting functions by using redis cache algorithm according to claim 3 is characterized in that: The division of data into different topics or categories is achieved by establishing different zset data.

8. The method for realizing concurrent reading and writing and custom sorting functions by using redis cache algorithm according to claim 3 is characterized in that: The high-concurrency reading and writing is based on the new paging method and the score sorting rules of zset, and data reading and writing are completely completed by redis.

Images