Like data compensation processing method and device, computer equipment and readable storage medium

Abstract

The invention discloses a like data compensation processing method and device, computer equipment and a readable storage medium, and the method comprises the steps: firstly obtaining data generated by related like operation of a user, and storing the data into a Redis metering pool; and then judging whether the Redis compensation pool has current user compensation data, if so, synchronizing the data to the DB database and then clearing the data, and if not, directly operating the DB database to store the data. A Redsson distributed lock is used for controlling concurrent operation of a DB database, and database and cache synchronization is completed according to user ID table division operation. The DB database is monitored through a section, if synchronization fails, compensation data are put into the compensation pool for compensation again, and it is ensured that data synchronization succeeds. According to the design, the like data processing efficiency can be effectively improved, the data consistency is guaranteed, the database pressure is relieved, and the system stability is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, device, computer equipment and readable storage medium for compensating like data. Background Art

[0002] Implementing the "Like" feature in current internet applications presents numerous challenges. In high-concurrency scenarios, a large number of "Like" requests flood in simultaneously, easily overloading the database and causing service unavailability. Furthermore, network fluctuations or system failures can cause "Like" data loss or inconsistency, impacting user experience and statistical accuracy. Furthermore, as the user base and the volume of "Like" data continue to grow, traditional non-table storage methods significantly increase database query and maintenance costs. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, device, computer equipment and readable storage medium for compensating like data.

[0004] In a first aspect, an embodiment of the present invention provides a method for compensating likes data, comprising:

[0005] Obtain the like data generated by the current user based on user like-related operations and record the like data in the Redis metering pool; the user like-related operations include like operations, cancel like operations, or refresh page operations. The metering pool uses a bitmap structure, uses setBit to mark likes or cancel likes, and uses getbit and bitcount to return the like status and total number of likes in real time;

[0006] Determine whether the compensation data of the current user exists in the Redis compensation pool; if the compensation data exists, immediately synchronize the data to the DB database and then clear the compensation data; if the compensation data does not exist, directly operate the DB database to save the current operation data; the DB database uses Redisson distributed locks to control concurrent operations to prevent excessive requests to operate the DB database;

[0007] Split the DB database into tables according to user ID to synchronize data between the database and cache.

[0008] The DB database is monitored in sections. If the data synchronization fails, the compensation data is placed in the compensation pool for re-compensation until the compensation data is synchronized to the DB database.

[0009] In a possible implementation, the bitmap structure generates a redis key for storing like data using the like content ID. When a user clicks a like, the position corresponding to the user ID in the bitmap of the corresponding redis key is set to 1, and when the like is canceled, it is set to 0.

[0010] In one possible implementation, the Redis compensation pool is a HashMap structure, and the compensation pool data includes objects and key-value pairs. The object represents a specific like content id, the key in the key-value pair represents the user id, and the value represents the current user's latest like status, 1 represents like, and 0 represents cancel like.

[0011] In one possible implementation, using Redisson distributed locks to control concurrent operations includes:

[0012] Specify a key as the lock tag and store it in Redis, use the unique user ID as the value, and set the expiration time;

[0013] After processing the business, verify the value and clear the key to release the lock, ensuring that only one client process obtains the lock at the same time.

[0014] In a possible implementation, the operation of partitioning the DB database into tables according to user IDs includes:

[0015] Determine the number of shard tables and calculate the start and end IDs of each shard table based on the number of shard tables;

[0016] The user ID is mapped to the corresponding shard table by taking the modulus of the number of shard tables by the user ID.

[0017] In a possible implementation, the number of sub-tables is set to 10, the sub-table identifier is generated according to the last two digits of the user ID, and the sub-table number is determined by performing a modulo operation of the user ID on 100.

[0018] In a possible implementation, the performing aspect monitoring on the DB database and placing the compensation data into the compensation pool for re-compensation if the data synchronization fails includes:

[0019] By utilizing the AfterThrowing exception notification of the AOP aspect, the compensation data is put back into the compensation pool for further compensation after the database operation throws an exception.

[0020] In a second aspect, an embodiment of the present invention provides a like data compensation processing device, comprising:

[0021] The acquisition module is used to obtain the like data generated by the current user based on the user's like-related operations and record the like data in the Redis metering pool; the user like-related operations include like operations, cancel like operations, or refresh page operations. The metering pool uses a bitmap structure, uses setBit to mark likes or cancel likes, and uses getbit and bitcount to return the like status and total number of likes in real time;

[0022] The compensation module is used to determine whether the compensation data of the current user exists in the compensation pool of Redis; if the compensation data exists, immediately synchronize the data to the DB database and then clear the compensation data; if the compensation data does not exist, directly operate the DB database to save the current operation data; the DB database uses Redisson distributed locks to control concurrent operations to prevent excessive requests to operate the DB database; perform table partitioning operations on the DB database according to user ID to complete data synchronization between the database and the cache; perform aspect monitoring on the DB database, and if the data synchronization fails, put the compensation data into the compensation pool for re-compensation until the compensation data is synchronized to the DB database.

[0023] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor and a non-volatile memory storing computer instructions, wherein when the computer instructions are executed by the processor, the computer device executes the method described in the first aspect.

[0024] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method described in the first aspect.

[0025] Compared with the prior art, the beneficial effects provided by the present invention include: using a likes data compensation processing method, device, computer equipment and readable storage medium disclosed by the present invention, including: first obtaining the data generated by the user's likes-related operations and storing it in the Redis metering pool. Then determine whether the Redis compensation pool has the current user compensation data. If so, synchronize it to the DB database and then clear it. If not, directly operate the DB database to save it. Use Redisson distributed locks to control the concurrent operations of the DB database, and complete the synchronization of the database and cache by table operation according to the user ID. Monitor the DB database through the aspect. If the synchronization fails, put the compensation data into the compensation pool for compensation again to ensure that the data synchronization is successful. Such a design can effectively improve the efficiency of likes data processing, ensure data consistency, reduce database pressure, and improve system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the drawings required for use in the embodiments. It should be understood that the following drawings illustrate only certain embodiments of the present invention and should not be construed as limiting the scope of the present invention. Those skilled in the art can, without inventive effort, derive other relevant drawings from these drawings.

[0027] Figure 1 A schematic flow chart of the steps of the likes data compensation processing method provided by an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of the architecture of a likes data compensation processing system provided by an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of a bitmap structure provided by an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of the compensation pool data structure provided by an embodiment of the present invention;

[0031] Figure 5 A schematic diagram of the distributed lock execution process provided by an embodiment of the present invention;

[0032] Figure 6 A schematic diagram of a user database provided by an embodiment of the present invention;

[0033] Figure 7 A schematic block diagram of the structure of a likes data compensation processing device provided by an embodiment of the present invention;

[0034] Figure 8 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0036] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0037] In order to solve the technical problems in the above background technology, Figure 1 This is a flow chart of the likes data compensation processing method provided in an embodiment of the present disclosure. The likes data compensation processing method is introduced in detail below.

[0038] Step S201: Obtain the like data generated by the current user based on the user's like-related operations, and record the like data in a Redis metering pool; the user's like-related operations include like operations, cancel like operations, or refresh page operations. The metering pool uses a bitmap structure, uses setBit to mark likes or cancel likes, and uses getbit and bitcount to return the like status and total number of likes in real time;

[0039] Step S202: Determine whether the compensation data of the current user exists in the compensation pool of Redis; if the compensation data exists, immediately synchronize the data to the DB database and then clear the compensation data; if the compensation data does not exist, directly operate the DB database to save the current operation data; the DB database uses Redisson distributed locks to control concurrent operations to prevent excessive requests to operate the DB database;

[0040] Step S203: Split the database into tables according to the user ID to complete data synchronization between the database and the cache;

[0041] Step S204: perform aspect monitoring on the DB database. If the data synchronization fails, the compensation data is placed in the compensation pool for re-compensation until the compensation data is synchronized to the DB database.

[0042] In the embodiment of the present invention, for example, it is assumed that the server runs a social media application, and many users can like and unlike various content (such as photos, articles, etc.) on the platform, and also frequently refresh the page to obtain the latest data.

[0043] User A, while browsing an article, finds the content fascinating and clicks the Like button. The server then receives User A's Like request, which is the "like data generated by the current user's Like action." The server then processes this data.

[0044] The metering pool uses a bitmap structure, using the article ID to generate a Redis key for storing like data, for example, "like_article:{article_id}". Suppose the article ID is 500 and user A's ID is 2000. The server executes the Redis setBit operation, setting the 2000th bit of "like_article:500" to 1, indicating that user A liked the article. Simultaneously, other users' likes on the article at the same time are also recorded in the same bitmap structure. When user B (with ID 2001) also likes the article, the 2001st bit of "like_article:500" is also set to 1.

[0045] When other users want to know whether User A has liked an article, the server can use the getbit operation to return the like status in real time. For example, when User C views the article details, the server executes redis.getbit("like_article:500",2000) to immediately obtain User A's like status (a value of 1 indicates a like). To count the total number of likes for the article, the server executes redis.bitcount("like_article:500") to obtain the total number of likes in real time.

[0046] After liking the article, user A reads it again and finds some parts that don't align with their own opinions, so they decide to cancel their like. The server receives user A's request to cancel the like and obtains the relevant data for this cancellation.

[0047] Based on the same Redis key "like_article:500," the server executes the setBit operation, but this time sets the 2000th bit to 0, indicating that user A has un-liked the article. At this point, other users who query user A's like status through the getBit operation will receive a value of 0, indicating no likes. The total number of likes for the article is also updated and decremented in real time through the bitcount operation.

[0048] User B is browsing a list of articles he or she follows and clicks the refresh button to get the latest likes. The server receives user B's refresh request and treats this action as "likes generated by the refresh."

[0049] Based on the page display logic, the server needs to retrieve the likes data for all articles followed by the user. For example, user B follows three articles with IDs 500, 501, and 502. The server uses the getbit operation to retrieve user B's like status for each article based on the Redis key corresponding to each article (e.g., "like_article:500," "like_article:501," and "like_article:502"), and the bitcount operation to retrieve the total number of likes for each article. This data is then collated and returned to user B's client to display the latest likes information.

[0050] Suppose that at some point in the past, user C liked a photo with the ID 300. However, due to network fluctuations or other reasons, the database operation failed to complete successfully. At this point, the server stores the data related to user C's like in the Redis compensation pool. The compensation pool is a HashMap structure, where the object represents the photo ID (here, 300); the key in the key-value pair represents the user ID (user C's ID (assuming it is 1500), and the value represents the user's most recent like status, with 1 indicating a like.

[0051] When user C performs another like-related action (such as liking another photo, canceling a previous like, or refreshing the page), the server obtains the like data and records it in the metering pool, then starts to check the compensation pool. The server finds that user C's compensation data (for the like data of photo ID 300) exists in the compensation pool.

[0052] The server immediately synchronizes the compensation data with the database. During the synchronization process, the database uses Redisson distributed locks to control concurrent operations. For example, if a large number of users are simultaneously performing operations such as "like" and "cancel like", Redisson distributed locks will ensure that only one client process can access the database at a time, preventing excessive simultaneous requests from causing a database crash.

[0053] Specifically, the Redisson distributed lock specifies a key as the lock tag and stores it in Redis, with user C's unique identifier (such as user C's ID 1500) as the value and an expiration time, such as 10 seconds. When other requests attempt to acquire the lock, they discover that the key already exists and the value is different from their own identifier (or the key exists but has expired). They are unable to acquire the lock and must wait. After the server completes processing user C's data synchronization, it verifies the value (confirming it is user C's ID 1500) and clears the key, releasing the lock. This ensures that only one client process can acquire the lock at a time.

[0054] After the data synchronization is completed, the server clears the compensation data of user C for the photo ID 300 in the compensation pool to free up space in preparation for receiving new compensation data that may appear.

[0055] User D likes the video with the video ID 400. After the server obtains the like data and records it in the metering pool, it checks the compensation pool and finds that there is no compensation data for user D.

[0056] The server directly operates the database to save the data of user D's "like" action on video ID 400. Similarly, when operating the database, Redisson distributed locks are used to control concurrency. Redisson distributed locks specify a key as a lock tag stored in Redis, with user D's unique identifier (assuming user D's ID is 1800) as the value and an expiration time (for example, 15 seconds).

[0057] After successfully acquiring the lock, the server inserts the like data into the database. After the operation is complete, the value is verified (confirming that it is user D's ID 1800) and the key is cleared to release the lock, preventing too many requests from operating the database simultaneously.

[0058] Suppose the system decides to split the database into 10 tables to store likes data. The server calculates the start and end IDs for each table based on the number of tables. For example, if user IDs range from 1 to 10,000, each table will store data for an average of 1,000 users. The first table will have a start ID of 1 and an end ID of 1000; the second table will have a start ID of 1001 and an end ID of 2000, and so on.

[0059] For example, user E's ID is 3567. The server calculates the user ID modulo the number of shard tables, 10, resulting in 3567 / 10 = 7. The result, 7, is the shard table index for that user ID. Therefore, user E's likes data will be stored in the eighth table (indexed starting at 0).

[0060] When the server synchronizes data, whether synchronizing data from the Redis metering pool or the compensation pool to the database, it follows this table partitioning rule. For example, if user E likes a picture with ID 600, when synchronizing the like data to the database, the server finds the corresponding table 8 based on user E's ID and inserts the data into that table. At the same time, the database and cache data are synchronized. When reading data from the database to update the cache, the corresponding table is also retrieved according to the table partitioning rule.

[0061] Suppose user F likes an audio file with the ID 700. The server records the like data in the metering pool. After determining that the compensation pool does not contain user F's compensation data, it directly operates the database to save the data. During the save process, due to a temporary failure of the database server, the data insertion operation fails and an exception is thrown.

[0062] The server uses the AfterThrowing exception notification of the AOP aspect to catch the exception thrown by the database operation. At this point, the server puts the compensation data for user F's like operation back into the compensation pool. The data structure in the compensation pool is still a HashMap structure, with the object being the audio id 700, the key being user F's id (assuming it's 2300), and the value being the like status 1.

[0063] The server continuously monitors the compensation pool. When it detects data in the compensation pool, it attempts to synchronize the data with the database again. During each synchronization attempt, Redisson distributed locks are used to control concurrency and ensure database stability. If synchronization fails again, the compensation data remains in the compensation pool until the next attempt is made and the data is successfully synchronized to the database.

[0064] For example, after the first synchronization fails, the server waits for a period of time (e.g., 5 seconds) and tries again to obtain user F's data from the compensation pool for synchronization. If the second synchronization succeeds, the server clears user F's compensation data from the compensation pool. If the second synchronization fails again, the server waits and tries again until it succeeds, thus ensuring data consistency and integrity.

[0065] Through the above detailed scenario examples, the specific implementation process of the likes data compensation processing method in practical applications is fully demonstrated, ensuring that the likes data can be processed and stored efficiently, stably and accurately.

[0066] In an embodiment of the present invention, the bitmap structure generates a redis key for storing like data using the like content ID. When a user likes a content, the position corresponding to the user ID in the bitmap of the corresponding redis key is set to 1, and when the like is canceled, it is set to 0.

[0067] In the embodiment of the present invention, for example, it is assumed that the server runs a picture and text sharing platform, and users can like or cancel the like operation on the picture and text content on the platform.

[0068] There's a picture and article on the platform with the likes content ID 101. User "Yangguang," whose user ID is 20230101, browses this picture and article, finds the content interesting, and clicks the like button. After receiving "Yangguang's" like request for the picture and article with ID 101, the server, following the rules, uses the likes content ID to generate a Redis key for storing the likes data: "like_content:101." The server then sets the bitmap corresponding to user ID 20230101 to 1 in the bitmap structure corresponding to "like_content:101," indicating that "Yangguang" has liked the picture and article. At this point, if other users want to query whether "Yangguang" has liked the picture and article, the server executes the Redis getbit operation (redis.getbit("like_content:101",20230101)). This will immediately return a like status of 1, indicating that "Yangguang" has liked the picture and article.

[0069] Later, "Sunshine" re-examined the post and decided it wasn't as good as he initially thought, so he decided to cancel his like. Upon receiving the cancel request, the server also resets the bitmap corresponding to the user ID 20230101 in the Redis key "like_content:101" from 1 to 0, indicating that "Sunshine" has canceled his like. When another query is received, redis.getbit("like_content:101",20230101) is executed, and the returned like status is 0, indicating that "Sunshine" no longer likes the post.

[0070] If the platform wants to count the total number of likes for a post, the server can execute the redis.bitcount("like_content:101") operation to obtain the total number of likes. This method of generating a Redis key based on the ID of the liked content and marking the user's like or cancel status by setting 1 or 0 in the bitmap structure allows for efficient and convenient recording and management of like data.

[0071] In an embodiment of the present invention, the Redis compensation pool is a HashMap structure, and the compensation pool data includes objects and key-value pairs. The object represents a specific like content id, the key in the key-value pair represents the user id, and the value represents the current user's latest like status, 1 represents like, and 0 represents cancel like.

[0072] In the embodiment of the present invention, illustratively, a program for processing user likes data is running on a certain video sharing server.

[0073] While browsing videos, user "Xingchen" saw a video with a like (ID 202) and liked it. However, due to a momentary network fluctuation, the like data failed to be successfully synchronized to the database. The server then stored the like data in the Redis compensation pool. The compensation pool is a HashMap structure, and the object here is the specific like content ID, 202. In the key-value pair, the key is "Xingchen's" user ID, assuming it is 30303, and the value is 1, representing the latest status of "Xingchen's like".

[0074] A moment later, Xingchen saw another video and liked content with ID 203. This time, the like operation was successful, and the data was synchronized to the database and cache. Xingchen then refreshed the page. The server retrieved the like data generated by Xingchen's refresh operation and recorded it in the Redis metering pool. The server then checked the compensation pool and found compensation data for content with ID 202.

[0075] The server immediately synchronizes the data from the compensation pool, syncing all of "Xingchen's" likes for video 202 to the database. During the synchronization process, Redisson distributed locks are used to control concurrency and ensure database stability. After successful synchronization, the server clears the data from the compensation pool for "Xingchen's" likes for content with ID 202 to free up space.

[0076] Let's consider another user, "Dahai," whose user ID is 40404. "Dahai" un-likes a video with content ID 204. Due to a brief database busy period, the un-like data wasn't updated in time. The server then stores this data in the compensation pool. In the compensation pool, the object is 204, the key is "Dahai's" user ID 40404, and the value is 0, indicating the un-like status.

[0077] When Dahai likes another video, the server records the likes data in the metering pool. It then checks the compensation pool and finds the compensation data for Dahai's liked content with ID 204. It then synchronizes this data to the database and clears the compensation data after completion. This way, the compensation pool effectively ensures the consistency and integrity of the likes data even in unusual situations.

[0078] In the embodiment of the present invention, the use of Redisson distributed locks to control concurrent operations can be implemented through the following examples.

[0079] Specify a key as the lock tag and store it in Redis, use the unique user ID as the value, and set the expiration time;

[0080] After processing the business, verify the value and clear the key to release the lock, ensuring that only one client process obtains the lock at the same time.

[0081] In the embodiment of the present invention, for example, it is assumed that the server supports a popular short video platform, and a large number of users perform operations such as liking and unliking at the same time, and the concurrency is extremely high.

[0082] While browsing short videos, user "Dream Chaser" sees a video with the like content ID 505 and immediately likes it. After receiving the like request, the server begins to use Redisson distributed locks to control concurrent operations.

[0083] First, the server stores a designated lock tag in Redis, for example, "lock:video:505." This key is associated with the liked video content, with the unique user ID of "Dream Chaser" (assuming it's 60606) as its value, and an expiration time set, here set to 10 seconds. At this point, if other users also like this video or perform other database-related operations, they attempt to acquire the same lock, storing the same key "lock:video:505" in Redis. Because the key already exists in Redis, these subsequent requests cannot successfully set the lock and cannot acquire the lock, forcing them to wait.

[0084] The "Dream Chaser" likes transaction begins. The server records the likes data in the Redis metering pool and checks whether the relevant data exists in the compensation pool, among other operations. Once all transactions are complete, successfully syncing the likes data to the database, the server verifies the value and confirms that it is "Dream Chaser"'s user ID 60606, indicating that the user who originally acquired the lock completed the transaction. The server then clears the key "lock:video:505," releasing the lock.

[0085] Shortly after "Dream Chaser" released the lock, user "Explorer" (user id 70707) also liked the video with the content id 505. The server also executed the lock acquisition process, specifying "lock:video:505" as the key, "Explorer's" user id 70707 as the value, and setting the expiration time to 10 seconds. Since the previous lock had been released, "Explorer" successfully acquired the lock and began to process the like business. After the processing is completed, the value is verified again to be 70707, and the key is cleared to release the lock, ensuring that only one client process can obtain the lock at the same time, effectively avoiding too many requests operating on the database at the same time under high concurrency, and preventing the database from crashing due to excessive pressure.

[0086] In the embodiment of the present invention, the operation of partitioning the DB database into tables according to user IDs can be implemented through the following examples.

[0087] Determine the number of shard tables and calculate the start and end IDs of each shard table based on the number of shard tables;

[0088] The user ID is mapped to the corresponding shard table by taking the modulus of the number of shard tables by the user ID.

[0089] In an embodiment of the present invention, for example, the number of sub-tables is first determined to be 10. The platform calculates the start ID and end ID of each sub-table based on the distribution of user IDs. Since user IDs are assigned continuously, and assuming that the current user ID range is roughly from 1 to 100,000. Then each table is expected to store an average of 10,000 users' likes data. The start ID of the first table is set to 1 and the end ID is 10,000; the start ID of the second table is 10,001 and the end ID is 20,000, and so on. The start ID of the tenth table is 90,001 and the end ID is 100,000.

[0090] For example, user "Qingfeng" has a user ID of 35789. When the server performs table partitioning, it takes the modulus of the user ID and the number of partitions, i.e., 35789%10=9 (the remainder is obtained from the modulo operation). This indicates that "Qingfeng's" likes data should be mapped to the 10th table (counting starts at 0, and the remainder 9 corresponds to the 10th table).

[0091] When Qingfeng likes an article, the server retrieves the like data and records it in the Redis metering pool. After processing the relevant logic, it prepares to store the like data in the database. Based on the above table partitioning rules, the server determines that the data should be stored in the 10th table and inserts Qingfeng's like data for the article into this table, completing the data storage.

[0092] Let's look at user "Mingyue" (Mingyue), whose user ID is 12345. The server calculates 12345%10=5, indicating that Mingyue's likes data should be stored in table 6. When Mingyue likes a video, the server accurately stores the likes data in the corresponding table 6 according to the table partitioning rules. This table partitioning allows the server to more efficiently manage and store the likes data of massive amounts of users, improving overall database performance.

[0093] In the embodiment of the present invention, the number of sub-tables is set to 10, the sub-table identifier is generated according to the last two digits of the user ID, and the sub-table number is determined by performing a modulo operation of the user ID on 100.

[0094] In an embodiment of the present invention, for example, it is assumed that the server is responsible for managing the likes data of a content sharing platform with a huge number of users. In order to store and manage the data more efficiently, a table partitioning operation is adopted, the number of tables is set to 10, and the table number is determined by performing a modulo operation of 100 on the user ID, and the table identifier is generated by the last two digits of the user ID.

[0095] User "Rainbow" browses a great post on the platform and decides to like it. Rainbow's user ID is 12345678. After receiving the like request, the server performs a modulo operation on 100 using the user ID: 12345678 % 100 = 78. This indicates that Rainbow's like data should be assigned to the table identified by 78. Under this table partitioning rule, this table is the final partition that stores Rainbow's like data.

[0096] Next, the server follows the process, recording the likes data in the Redis metering pool. It then performs a series of processing to determine whether the Redis compensation pool contains compensation data for "Rainbow." If all goes well, the server prepares to persist the data to the database. Based on the previously determined table number, "Rainbow"'s likes for the post are inserted into table number 78.

[0097] Let's look at user "Fanxing" (Fanxing). Their user ID is 98765432. The server also performs a modulo operation on "Fanxing's" user ID: 98765432 % 100 = 32. When "Fanxing" likes a video on the platform, after recording the data in the Redis metering pool and determining the compensation pool, the server stores "Fanxing's" like data in the subtable identified by 32.

[0098] By generating a table identifier based on the last two digits of the user ID and using modulo operations to determine the table number, the server can sequentially store different users' likes data in their corresponding tables. Even in high-concurrency scenarios, with numerous users simultaneously submitting likes, the server can methodically and accurately store the data in the corresponding tables according to established rules. This significantly improves database storage and query efficiency, ensuring the stable operation of the platform's likes feature.

[0099] In the embodiment of the present invention, the DB database is monitored in a section manner. If the data synchronization fails, the compensation data is placed in the compensation pool for re-compensation. This can be implemented through the following example.

[0100] By utilizing the AfterThrowing exception notification of the AOP aspect, the compensation data is put back into the compensation pool for further compensation after the database operation throws an exception.

[0101] In the embodiment of the present invention, for example, it is assumed that the server is running an information application, and users can like various news articles.

[0102] User "Chenyang" browses a news article. The article's "like" content has the ID 808, and user "Chenyang"'s ID is 90909. Chenyang finds the article valuable and clicks the "Like" button. After receiving the "Like" request, the server first records the "Like" data in the Redis metering pool. Then, using the "setBit" operation on the bitmap structure, it sets the bit corresponding to user ID 90909 to 1 in the Redis key "like_news:808" generated based on the "Like" content ID.

[0103] The server then determined that there was no compensation data for "Chen Yang" in the Redis compensation pool, and directly operated the database to save the likes data. However, when executing the database insert operation, due to a temporary resource bottleneck on the database server, the operation threw an exception.

[0104] The server catches this exception using the AfterThrowing exception notification of the AOP aspect. At this point, the server puts the compensation data for this like action back into the compensation pool. The compensation pool is a HashMap structure with the like content ID 808 as the object, the user ID 90909 for "Chen Yang" as the key, and a value of 1, indicating the like status.

[0105] Afterwards, the server continuously monitors the compensation pool. If it detects compensation data for article 808 from Chenyang, it will attempt to synchronize the data with the database again. During this synchronization, Redisson distributed locks will be used to control concurrent operations and ensure database stability.

[0106] For example, after waiting for 5 seconds, the server retrieves Chen Yang's like data for article 808 from the compensation pool and attempts to insert it into the database. This time, the database server resources return to normal, and the data is successfully inserted into the database. After confirming that the data synchronization is successful, the server clears Chen Yang's compensation data for article 808 from the compensation pool.

[0107] If the second synchronization fails, the server will continue to wait and try again, with each wait time being extended appropriately, such as 10 seconds, until the data is successfully synchronized to the database. In this way, by utilizing the AfterThrowing exception notification of the AOP aspect, the server can promptly place the compensation data into the compensation pool when data synchronization fails, and perform further compensation, thus ensuring that the likes data is ultimately stored accurately in the database, ensuring data integrity and consistency.

[0108] In order to more clearly describe the solution provided by the present invention, a relatively complete implementation method is provided below. Figure 2 , Figure 2 A schematic diagram of the architecture of the likes data compensation processing system provided in an embodiment of the present invention.

[0109] Step 01: Install project tools such as IDEA, JDK 8, Maven, Git, and MySQL, introduce JAR packages, Redis bitmap storage metering pool, aspect monitoring process, Redisson distributed lock and other technologies.

[0110] Step 02: When a user likes or cancels a post or refreshes the page, the user's like data is obtained. The server queries the Redis bitmap algorithm metering pool to cache getbit and bitcount, returning the results in real time. The backend processes the operation data asynchronously.

[0111] Step 03: Asynchronously synchronize the cache and database, determining whether the current user's compensation data exists in the Redis compensation pool. If so, immediately execute the data synchronization rules. After saving, remove the current user's data from the compensation pool. If not, directly operate the database to perform data operations.

[0112] Step 04: Use Redisson distributed lock technology to control concurrency and prevent excessive requests from directly operating the database, causing the database to crash.

[0113] Step 05: Split the database into tables according to the user ID, such as the table name + the last two digits as an identifier, to complete the data synchronization between the database and the cache.

[0114] Step 06: The aspect monitors the process of operating the database. If it fails, it continues to enter the compensation pool for compensation again, and the operation is cyclical.

[0115] Please refer to Figure 3 , Figure 3 The schematic diagram of the bitmap structure provided in the embodiment of the present invention, bitmap for short, is an array composed of multiple binary bits. Each binary bit in the array has a corresponding offset. These offsets can be used to operate on one or more binary bits specified in the bitmap. 2, 4, 6, and 7 are easily represented, that is, they are stored in the corresponding positions in b[0], and the corresponding positions are set to 1. For numbers N greater than 7, the corresponding b[n] is found by rounding down n=1+N / 8, and then the remainder operation is performed on 8 with N, and the remainder is the storage position. To determine whether a number exists, it depends on whether the bit corresponding to the number is 1. To describe it bluntly, it is similar to an array. The values of the array only exist in two cases: 0 and 1. If the value is 1, it means that the value of the corresponding group index exists.

[0116] Like / Unlike:

[0117] Assume that the user's numeric id is 1000, and he likes the photo with id 100. First, generate a Redis key based on the photo id to store the like data. For example, if the generation strategy is like_photo:{photo_id}, and the user with id 1000 likes the photo, we only need to set the 1000th bit of like_photo:100 to 1 (or set it to 0 if the user does not like the photo).

[0118] The time complexity of the redissetbit operation is O(1), so this way of giving likes is very efficient.

[0119] redis.setbit("like_photo:100",1000,1);

[0120] Is this a Like?

[0121] When a user opens an image, they need to check whether they have liked the photo. This can be done using the redisgetbit operation. For example, to check whether the user with user ID 1000 has liked the photo with photo ID 100, you only need to retrieve the value of bit 1000 of the like_photo:100 bitmap.

[0122] The time complexity of the redisgetbit operation is also O(1).

[0123] redis.getbit("like_photo:100",1000);

[0124] Query the total number of likes:

[0125] For example, if you need to display the number of likes for a photo with the photo id 100, you only need to perform a bitmap counting operation on like_photo:100bitmap.

[0126] Although the time complexity of the redis bitcount operation is O(N), you don't need to worry about the bitcount efficiency issue in most data volumes. For example, redis.bitcount("like_photo:100");

[0127] Please refer to Figure 4 , Figure 4This is a diagram of the compensation pool data structure provided by an embodiment of the present invention. The compensation pool is a Redis HashMap structure. Object 1 represents the name of a specific photo with ID 1, user1 represents the user with ID 1, and value represents the current user's most recent like status, with 1 indicating a like and 0 indicating a cancel. If compensation data does not exist for an object, the current object does not exist. This continues in this order, with compensation data for N objects.

[0128] When the user clicks or refreshes, the compensation pool is checked to see if the current object data exists. If so, the data in the current compensation pool is immediately saved to the DB to ensure data consistency. After saving, the compensation pool is cleared to free up space. If the data does not exist, the data of the current operation is saved in the DB. When operating the database, the AOP aspect AfterThrowing exception notification is used to re-execute the compensation operation after the method throws an exception.

[0129] Please refer to Figure 5 , Figure 5 Provide your distributed lock execution flow diagram for the embodiment of the present invention, specify a key as the lock mark, store it in Redis, and specify a unique user identifier as the value.

[0130] The value can only be set when the key does not exist, ensuring that only one client process obtains the lock at the same time, satisfying the mutual exclusion feature.

[0131] Set an expiration time to prevent the key from being deleted due to system abnormalities, and meet the anti-deadlock feature.

[0132] After processing the business, you need to clear the key to release the lock. When clearing the key, you need to verify the value. It must be satisfied that only the person who locked the lock can release it.

[0133] To ensure that a method or property can only be executed by the same thread at the same time in high concurrency situations.

[0134] Please refer to Figure 6 , Figure 6 Schematic diagram of the user database provided for an embodiment of the present invention. Determine the number of sub-tables: First, determine that the table should be divided into 10 tables, which can be distinguished according to the range of user IDs. Calculate the sub-table boundaries: Based on the number of sub-tables, calculate the start ID and end ID of each sub-table. Map the user ID to the sub-table: In order to map the user ID to the corresponding sub-table, a modulo operation can be used. The result of taking the modulo operation of the user ID and the number of sub-tables is the index of the sub-table where the user ID is located. When inserting data, find the corresponding sub-table according to the user ID, and then insert the data into the table.

[0135] Please refer to Figure 7 , Figure 7 A like data compensation processing device 110 provided in an embodiment of the present invention includes:

[0136] The acquisition module is used to obtain the like data generated by the current user based on the user's like-related operations and record the like data in the Redis metering pool; the user like-related operations include like operations, cancel like operations, or refresh page operations. The metering pool uses a bitmap structure, uses setBit to mark likes or cancel likes, and uses getbit and bitcount to return the like status and total number of likes in real time;

[0137] The compensation module is used to determine whether the compensation data of the current user exists in the compensation pool of Redis; if the compensation data exists, immediately synchronize the data to the DB database and then clear the compensation data; if the compensation data does not exist, directly operate the DB database to save the current operation data; the DB database uses Redisson distributed locks to control concurrent operations to prevent excessive requests to operate the DB database; perform table partitioning operations on the DB database according to user ID to complete data synchronization between the database and the cache; perform aspect monitoring on the DB database, and if the data synchronization fails, put the compensation data into the compensation pool for re-compensation until the compensation data is synchronized to the DB database.

[0138] It should be noted that the implementation principles of the aforementioned likes data compensation processing device can be referenced to the implementation principles of the aforementioned likes data compensation processing method, and will not be elaborated here. It should be understood that the division of the various modules of the aforementioned device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity, or physically separated. Furthermore, these modules can be implemented entirely in the form of software called by a processing element; or entirely in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the likes data compensation processing device can be a separate processing element, or it can be integrated into a chip of the aforementioned device. Furthermore, it can be stored in the form of program code in the memory of the aforementioned device, and called and executed by a processing element of the aforementioned device. The implementation of the other modules is similar. Furthermore, these modules can be fully or partially integrated together, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, the steps of the aforementioned method or the above modules can be completed by hardware integrated logic circuits in the processor element or by software instructions.

[0139] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code on a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0140] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned like data compensation processing device. Figure 8 As shown, Figure 8 This is a block diagram of the computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a likes data compensation processing device, a memory 111, a processor 112, and a communication unit 113.

[0141] In order to realize the transmission or interaction of data, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly. For example, the electrical connection between these components can be achieved through one or more communication buses or signal lines. The likes data compensation processing device includes at least one software function module that can be stored in the memory 111 in the form of software or firmware or solidified in the operating system (OS) of the computer device 100. The processor 112 is used to execute the likes data compensation processing device stored in the memory 111, such as the software function modules and computer programs included in the likes data compensation processing device.

[0142] An embodiment of the present invention provides a readable storage medium, which includes a computer program. When the computer program is running, it controls the computer device where the readable storage medium is located to execute the aforementioned like data compensation processing device.

[0143] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in light of the above teachings. These embodiments have been selected and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the present disclosure and to utilize various embodiments with various modifications as appropriate for the specific application contemplated.

Claims

1. A method for compensating likes data, characterized in that: include: Get the like data generated by the current user based on the user's like-related operations, and record the like data in the Redis metering pool; The user like-related operations include like operation, cancel like operation or refresh page operation. The metering pool adopts a bitmap structure, uses setBit to mark like or cancel like, and uses getbit and bitcount to return the like status and total number of likes in real time; Determine whether the compensation data of the current user exists in the compensation pool of Redis; if the compensation data exists, immediately synchronize the data to the DB database and then clear the compensation data; if the compensation data does not exist, directly operate the DB database to save the current operation data; The DB database uses Redisson distributed locks to control concurrent operations to prevent excessive requests from operating the DB database; Split the DB database into tables according to user ID to synchronize data between the database and cache. The DB database is monitored in sections. If the data synchronization fails, the compensation data is placed in the compensation pool for re-compensation until the compensation data is synchronized to the DB database.

2. The method according to claim 1, characterized in that The bitmap structure generates a redis key for storing like data using the like content ID. When a user clicks a like, the position corresponding to the user ID in the bitmap of the corresponding redis key is set to 1, and when the like is canceled, it is set to 0.

3. The method according to claim 1, characterized in that The Redis compensation pool is a HashMap structure, and the compensation pool data includes objects and key-value pairs. The object represents a specific like content id, the key in the key-value pair represents the user id, and the value represents the current user's latest like status, 1 represents like, and 0 represents cancel like.

4. The method according to claim 1, wherein The use of Redisson distributed locks to control concurrent operations includes: Specify a key as the lock tag and store it in Redis, use the unique user ID as the value, and set the expiration time; After processing the business, verify the value and clear the key to release the lock, ensuring that only one client process obtains the lock at the same time.

5. The method according to claim 1, characterized in that The operation of partitioning the DB database into tables according to the user ID includes: Determine the number of shard tables and calculate the start and end IDs of each shard table based on the number of shard tables; The user ID is mapped to the corresponding shard table by taking the modulus of the number of shard tables by the user ID.

6. The method according to claim 5, characterized in that The number of sub-tables is set to 10, and the sub-table identifier is generated according to the last two digits of the user ID. The sub-table number is determined by performing a modulo operation on the user ID by 100.

7. The method according to claim 1, characterized in that The section monitoring of the DB database and placing the compensation data into the compensation pool for re-compensation if the data synchronization fails include: By utilizing the AfterThrowing exception notification of the AOP aspect, the compensation data is put back into the compensation pool for further compensation after the database operation throws an exception.

8. A likes data compensation processing device, characterized in that: include: The acquisition module is used to obtain the like data generated by the current user based on the user's like-related operations and record the like data in the Redis metering pool; The user like-related operations include like operation, cancel like operation or refresh page operation. The metering pool adopts a bitmap structure, uses setBit to mark like or cancel like, and uses getbit and bitcount to return the like status and total number of likes in real time; The compensation module is used to determine whether the compensation data of the current user exists in the compensation pool of Redis; if the compensation data exists, immediately synchronize the data to the DB database and then clear the compensation data; if the compensation data does not exist, directly operate the DB database to save the current operation data; The DB database uses Redisson distributed locks to control concurrent operations to prevent excessive requests to operate the DB database; the DB database is partitioned according to the user ID to complete the data synchronization between the database and the cache; The DB database is monitored in sections. If the data synchronization fails, the compensation data is placed in the compensation pool for re-compensation until the compensation data is synchronized to the DB database.

9. A computer device, characterized in that: The computer device includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device executes the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images