Implementation method for acquiring bullet screen messages in real time

Abstract

The invention belongs to the technical field of real-time acquisition of bullet screen messages, and discloses an implementation method for real-time acquisition of bullet screen messages. The implementation method comprises the following specific steps: S1, data initialization and memory management; s2, Ehcache space distribution and management are carried out; s3, setting a Redis cache room number; s4, starting a message synchronization task; s5, synchronous judgment and processing based on the cache; s6, the user enters the room to interact with the information; and S7, obtaining and returning the message. By introducing Ehcache and other measures, data can be stored in a local memory, the speed is improved, dynamic monitoring and other performance and resource utilization are optimized, the synchronization number can be controlled, and a confirmation mechanism is added to optimize experience, enhance system performance and the like; 2, the system has excellent expandability, disperses Redis pressure, is stable in QPS and can be infinitely and transversely expanded; and thirdly, when the client acquires the message, the connection overhead with the Redis is reduced, the interface response speed and stability are improved, and the experience is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-time acquisition of barrage messages, and specifically relates to a method for implementing real-time acquisition of barrage messages. Background Art

[0002] Danmu (bullet comments) are subtitles used as commentary on online videos and live streaming platforms. Highly concurrent, real-time bullet comments enhance the interactive experience. Its system design architecture prioritizes high stability, high availability, and low latency. Danmu messages can be distributed and retrieved via AJAX polling, AJAX persistent connection polling, and WebSocket. Danmu messages come in a variety of types, including text, images, and interactive elements like gifts, which can be customized. When the message volume is high, messages can be prioritized by type, with high, medium, or low priority. A maximum threshold for returned messages can also be set to accommodate complex scenarios and client presentation requirements.

[0003] Autohome uses diffuse reading to distribute bullet comments, and uses AJAX long connection interface polling to obtain them. As the number of viewers increases, the number of requests increases. The Redis-based storage and retrieval solution has the expected effect when the live broadcast room and the number of online users are small. The process is as follows: the client sends a bullet comment, and the server uses Redis key to store the message according to specific information and generates the maximum ID; after the user enters the live broadcast room, the client first obtains the maximum message ID set of each level; then, it retrieves messages at a fixed time, and the server multi-threads retrieves bullet comments according to priority, aggregates them, and returns the latest message ID set.

[0004] The existing barrage solution can ensure interface stability, system availability, and stability in scenarios with small barrage volume and few online users. However, during large-scale live broadcasts, due to Redis's single-threaded nature, increased QPS can degrade its performance, leading to system instability, message loss, increased latency, and even program freezes. Furthermore, although the server uses multiple threads to aggregate and return messages based on priority, when the barrage interface QPS exceeds 5,000, the interface performance is difficult to meet expectations due to factors such as new thread acquisition, thread switching, and the limit on the number of threads on a single machine. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for real-time acquisition of barrage messages to solve the problems raised in the above background technology.

[0006] In order to achieve the above-mentioned purpose, the present invention provides the following technical solution: A method for implementing real-time acquisition of bullet screen messages has the following specific steps:

[0007] S1: Data initialization and memory management: When the system starts, system parameters and cold data are put into the local memory and stored in hashMap. The data is updated synchronously from the database through scheduled tasks.

[0008] S2: Ehcache space allocation and management: Introduce Ehcache on the external interface server to allocate independent storage space for the live broadcast room, and delete the corresponding space after the live broadcast ends;

[0009] S3: Redis cache room number setting: When the client sends a bullet message, the room number is stored in the Redis cache according to specific rules, with an expiration time of 1 minute (dynamically adjustable), and this is used to determine whether the room is removed from the synchronization list;

[0010] S4: Message synchronization task starts: The external station interface server executes the message synchronization task every second, first querying the live broadcast room list from Redis and performing a loop operation;

[0011] S5: Cache-based synchronization judgment and processing: Determine whether the room is synchronized based on the Redis cache, and synchronize messages from Redis to Ehcache in batches based on priority and messageId relationship (the number of messages can be dynamically configured);

[0012] S6: User enters the room and interacts with information: The user enters the live broadcast room and passes the room number. The external interface server returns the oldestMessageId, the largest ID set of messages of each level.

[0013] S7: Message acquisition and return processing: The client obtains the interface according to the return information request, and the external station interface server compares the messageId and processes it. After acquisition, the aggregated and sorted message list and oldesMessageId are returned to the client.

[0014] Preferably, the specific steps of data initialization and memory management in S1 are as follows:

[0015] Step 1: Initial placement of cold data: When the system starts, all system parameters and cold data with low update frequency are uniformly placed in the local memory to prepare for subsequent data use and processing, ensuring that relevant data can be quickly called;

[0016] Step 2: Setting up a scheduled update mechanism: By setting up a scheduled task, updated data will be synchronized from the database to the local memory at a certain time period, so as to ensure that the data in the local memory can always be kept relatively up to date to meet the system operation requirements;

[0017] Step 3: Select the data structure for storage: Determine to use a hashMap data structure to store the data in the local memory, set the parameter name as the key, and the corresponding parameter value as the value, to facilitate subsequent quick search, access, and operation of the data in the memory.

[0018] Preferably, the Ehcache space allocation and management in S2 refers to the introduction of an efficient Ehcache caching mechanism on the external station interface server, accurately allocating independent storage space for each live broadcast room, so as to effectively manage and quickly access the live broadcast data, and deleting the corresponding space immediately after the live broadcast ends to release resources.

[0019] Preferably, the specific steps for setting the Redis cache room number in S3 are as follows:

[0020] Step 1: Cache storage when sending messages: When the client sends a message through the barrage message sending interface, the corresponding room number information must be stored in the Redis cache to lay the foundation for subsequent related data processing and synchronization operations;

[0021] Step 2: Cache key: According to the established rules, [message_chatroom_live:room number] is used as the key to accurately identify the stored content. At the same time, the cache expiration time is set to 1 minute, and this time can be dynamically adjusted based on the parameters in S1;

[0022] Step 3: Synchronization processing of expired messages: Monitor the room numbers in the cache. If no new messages are sent within the set expiration time range, the room corresponding to the room number will be removed from the synchronization list to optimize the management of synchronization resources.

[0023] Preferably, the message synchronization task initiation in S4 refers to the construction of a message synchronization task module on each external station interface server, with the execution frequency set to once per second. After the task is officially started, the first step is to accurately retrieve the room list information currently in the live broadcast state from the Redis database, and then carry out cyclic deep processing and analysis operations on each room in sequence.

[0024] Preferably, the specific steps of cache-based synchronization judgment and processing in S5 are as follows:

[0025] Step 1: Cache-based synchronization judgment: By referring to the cache-related information involved in S3, a comprehensive judgment is made on whether data synchronization is necessary for the current room, so as to clarify the subsequent data processing direction;

[0026] Step 2: Compare the message IDs based on priority: If the room needs to be synchronized, the message IDs of the latest message in the current room and the largest message ID after the last synchronization are compared.

[0027] Step 3: Synchronize messages according to configuration: When the above size relationship is determined to meet the conditions, batches of messages are synchronized from the Redis cache to the Ehcache according to the dynamically configurable rules set in S1 to ensure the orderly flow and update of data.

[0028] Preferably, the user entering the room and interacting with information in S6 refers to when the user first enters the live broadcast room, passing the room number, and the external station interface server returns the largest message ID set oldestMessageId of messages of all levels in the current room.

[0029] Preferably, the specific steps of message acquisition and return processing in S7 are as follows:

[0030] Step 1: Client request and server judgment: The client requests the barrage message acquisition interface based on the room number and the obtained maximum message ID of each level. The external interface server then compares the message ID of each level passed by the client with the maximum message ID of its own current level.

[0031] Step 2: Message retrieval based on comparison results: Messages are retrieved in batches from Ehcache based on message priority. If the client's largest message messageId is smaller than the server's, the server assembles the corresponding interval messages from Ehcache into a list set and adds the largest message ID in the set to oldestMessageId. If they are equal or greater, the server processes them according to the corresponding rules and updates oldestMessageId.

[0032] Step 3: Message integration and final return: After all levels of messages are obtained, the external station interface server integrates the message lists of each level, sorts them in ascending order according to the reception time, and finally returns the sorted message list together with the oldestMessageId to the client.

[0033] The beneficial effects of the present invention are as follows:

[0034] First, this invention introduces Ehcache and retrieves messages from the current machine, offering numerous benefits. It stores some data in local memory, improving read and response speeds. Second, it dynamically monitors message synchronization to avoid invalid requests, improving performance and resource utilization. Third, it controls the number of synchronizations per session to prevent message storms and optimize the user experience. Fourth, it adds a client-side confirmation mechanism to ensure message accuracy. Overall, this effectively optimizes the message processing process, reduces external dependencies, enhances system performance, stability, and reliability, and improves the user experience.

[0035] 2. The present invention has excellent scalability through this optimization solution. It distributes the pressure of Redis to each external station interface server. Regardless of how the number of live broadcast rooms changes, Redis's QPS can be maintained in a stable range without large fluctuations. It is particularly worth mentioning that this solution can achieve unlimited horizontal expansion, which means that it can support a nearly unlimited number of online users. This undoubtedly provides solid support for live broadcast scenarios with continuous business expansion and growing user scale, helping the business to develop and grow better.

[0036] 3. The present invention can directly obtain data from Ehcache when the client requests a barrage message, greatly reducing the connection overhead with Redis and the number of requests. The interface response speed and stability are greatly improved, effectively ensuring the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of a method for obtaining real-time bullet screen messages according to the present invention;

[0038] Figure 2 This is a flow chart of message storage and acquisition of the present invention;

[0039] Figure 3 This is a functional architecture diagram of the present invention;

[0040] Figure 4 This is a flowchart of the synchronization of barrage messages in the present invention;

[0041] Figure 5 This is a flowchart of the barrage message synchronization of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] like Figures 1 to 5 As shown, the embodiment of the present invention provides a method for obtaining bullet screen messages in real time, and the specific steps are as follows:

[0044] S1: Data initialization and memory management: When the system starts, system parameters and cold data are put into the local memory and stored in hashMap. The data is updated synchronously from the database through scheduled tasks.

[0045] S2: Ehcache space allocation and management: Introducing Ehcache in the external interface server, allocating independent storage space for live room, deleting the corresponding space after the live broadcast ends;

[0046] S3: Redis cache room number setting: When the client sends a barrage message, the room number is stored in Redis cache according to certain rules, with a 1-minute expiration time (which can be dynamically adjusted), and the room is determined whether to be removed from the synchronization list;

[0047] S4: Message synchronization task starts: The external interface server executes the message synchronization task every second, first checks the live room list from Redis and performs a loop operation;

[0048] S5: Synchronization judgment and processing based on cache: Determine whether the room is synchronized according to the Redis cache, and synchronize messages from Redis to Ehcache according to the priority and messageId relationship (the number of messages can be dynamically configured);

[0049] S6: User enters the room and information interaction: The user enters the live room and transmits the room number, and the external interface server returns the maximum id set oldestMessageId of each level message;

[0050] S7: Message acquisition and return processing: The client requests the acquisition interface according to the returned information, and the external interface server processes after comparing the messageId, aggregates and sorts the message list after acquisition, and returns the oldesMessageId to the client.

[0051] Redis is an open source key-value database written in C language, supporting network and in-memory persistence; Ehcache is a pure Java in-process cache framework, characterized by fast and simple, in common terminology, RT refers to response time, covering request sending, network transmission, and server processing time; QPS refers to the number of request transactions processed per second, i.e. throughput. WebSocket is a full-duplex communication protocol on a single TCP connection, facilitating data exchange, and can transmit bidirectionally after handshake. Read diffusion is an active pull subscriber message list, and write diffusion needs to write a message list for each subscriber.

[0052] Among them, the specific steps of data initialization and memory management in S1 are as follows:

[0053] Step 1: Cold data initialization: When the system starts, all system parameters and cold data with low update frequency in the system are placed in the local memory, preparing for subsequent data use and processing, and ensuring that related data can be quickly called;

[0054] Step 2: Setting up a scheduled update mechanism: By setting up a scheduled task, updated data will be synchronized from the database to the local memory at a certain time period, so as to ensure that the data in the local memory can always be kept relatively up to date to meet the system operation requirements;

[0055] Step 3: Select the data structure for storage: Determine to use a hashMap data structure to store the data in the local memory, set the parameter name as the key, and the corresponding parameter value as the value, to facilitate subsequent quick search, access, and operation of the data in the memory.

[0056] Redis is crucial for ensuring system stability, availability, and throughput. To reduce its usage and maintain a stable query per second (QPS), system parameters and infrequently updated cache data are stored in the memory of a single cluster machine at system startup. This memory data is regularly updated using scheduled tasks.

[0057] Among them, the Ehcache space allocation and management in S2 refers to the introduction of an efficient Ehcache caching mechanism on the external station interface server, accurately allocating independent storage space for each live broadcast room, so as to effectively manage and quickly access the live broadcast data, and immediately deleting the corresponding space after the live broadcast ends to release resources.

[0058] Ehcache space allocation and management is to introduce the Ehcache cache mechanism on the external station interface server, allocate independent storage space for each live broadcast room for effective management and fast access to live broadcast data, and delete the corresponding space when the live broadcast ends to achieve the purpose of releasing resources.

[0059] The specific steps for setting the Redis cache room number in S3 are as follows:

[0060] Step 1: Cache storage when sending messages: When the client sends a message through the barrage message sending interface, the corresponding room number information must be stored in the Redis cache to lay the foundation for subsequent related data processing and synchronization operations;

[0061] Step 2: Cache key: According to the established rules, [message_chatroom_live:room number] is used as the key to accurately identify the stored content. At the same time, the cache expiration time is set to 1 minute, and this time can be dynamically adjusted based on the parameters in S1;

[0062] Step 3: Synchronization processing of expired messages: Monitor the room numbers in the cache. If no new messages are sent within the set expiration time range, the room corresponding to the room number will be removed from the synchronization list to optimize the management of synchronization resources.

[0063] In the process of transferring the message acquisition source from Redis to the stand-alone Ehcache, dynamic control is also implemented to determine whether the room messages are synchronized. The judgment is based on whether there are any messages sent in the last minute. If the room has messages sent within this minute, the message synchronization operation is performed to synchronize the messages from Redis to Ehcache; if no messages are sent, the room will be removed from the synchronization list.

[0064] Among them, the message synchronization task startup in S4 refers to the construction of a message synchronization task module on each external station interface server, and its execution frequency is set to once per second. After the task is officially started, the first step is to accurately retrieve the room list information currently in the live broadcast state from the Redis database, and then carry out cyclic deep processing and analysis operations on each room in sequence.

[0065] When the synchronization task is started, the external interface server builds the task module, which is executed once per second. After it is started, the live broadcast room list information is retrieved from the Redis database, and then a loop of deep processing and analysis operations are performed on each room.

[0066] The specific steps of cache-based synchronization judgment and processing in S5 are as follows:

[0067] Step 1: Cache-based synchronization judgment: By referring to the cache-related information involved in S3, a comprehensive judgment is made on whether data synchronization is necessary for the current room, so as to clarify the subsequent data processing direction;

[0068] Step 2: Compare the message IDs based on priority: If the room needs to be synchronized, the message IDs of the latest message in the current room and the largest message ID after the last synchronization are compared.

[0069] Step 3: Synchronize messages according to configuration: When the above size relationship is determined to meet the conditions, batches of messages are synchronized from the Redis cache to the Ehcache according to the dynamically configurable rules set in S1 to ensure the orderly flow and update of data.

[0070] The client made corresponding changes to transfer the message acquisition source from Redis to the stand-alone Ehcache: the cluster machines periodically synchronized the latest messages from Redis to the local Ehcache every second; set the Ehcache message expiration time to one minute, allocated independent storage space for each room, and cleared it after the live broadcast ended; and changed the method of synchronizing messages from Redis from one by one to obtaining multiple messages at a time, thereby reducing the number of Redis requests.

[0071] Among them, the user entering the room and information interaction in S6 refers to when the user enters the live broadcast room for the first time, the room number is passed, and the external station interface server returns the largest message ID set oldestMessageId of messages of all levels in the current room.

[0072] When a user enters a live broadcast room for the first time, he needs to pass the corresponding room number to the system. Then, based on this request, the external interface server will return the largest message ID set of messages of all levels in the current room, that is, the oldestMessageId, to achieve information interaction.

[0073] The specific steps of message acquisition and return processing in S7 are as follows:

[0074] Step 1: Client request and server judgment: The client requests the barrage message acquisition interface based on the room number and the obtained maximum message ID of each level. The external interface server then compares the message ID of each level passed by the client with the maximum message ID of its own current level.

[0075] Step 2: Message retrieval based on comparison results: Messages are retrieved in batches from Ehcache based on message priority. If the client's largest message messageId is smaller than the server's, the server assembles the corresponding interval messages from Ehcache into a list set and adds the largest message ID in the set to oldestMessageId. If they are equal or greater, the server processes them according to the corresponding rules and updates oldestMessageId.

[0076] Step 3: Message integration and final return: After all levels of messages are obtained, the external station interface server integrates the message lists of each level, sorts them in ascending order according to the reception time, and finally returns the sorted message list together with the oldestMessageId to the client.

[0077] Finally, we changed the multi-threaded message acquisition method to a synchronous method, which retrieves message sets from the local Ehcache according to message priority, aggregates them, and returns them to the client. This allows for more efficient management of the message acquisition process.

[0078] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0079] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for obtaining real-time bullet screen messages, characterized by: The specific steps for implementing the method of obtaining the bullet message in real time are as follows: S1: Data initialization and memory management: When the system starts, system parameters and cold data are put into the local memory and stored in hashMap. The data is updated synchronously from the database through scheduled tasks. S2: Ehcache space allocation and management: Introduce Ehcache on the external interface server to allocate independent storage space for the live broadcast room, and delete the corresponding space after the live broadcast ends; S3: Redis cache room number setting: When the client sends a bullet message, the room number is stored in the Redis cache according to specific rules and set to an expiration time of 1 minute. This is used to determine whether the room is removed from the synchronization list; S4: Message synchronization task starts: The external station interface server executes the message synchronization task every second, first querying the live broadcast room list from Redis and performing a loop operation; S5: Cache-based synchronization judgment and processing: Determine whether the room is synchronized based on the Redis cache, and synchronize messages from Redis to Ehcache in batches based on priority and messageId relationship; S6: User enters the room and interacts with information: The user enters the live broadcast room and passes the room number. The external interface server returns the oldestMessageId, the largest ID set of messages of each level. S7: Message acquisition and return processing: The client obtains the interface according to the return information request, and the external station interface server compares the messageId and processes it. After acquisition, the aggregated and sorted message list and oldesMessageId are returned to the client.

2. A method for obtaining real-time bullet screen messages according to claim 1, characterized in that: The specific steps of data initialization and memory management in S1 are as follows: Step 1: Initial placement of cold data: When the system starts, all system parameters and cold data with low update frequency are uniformly placed in the local memory to prepare for subsequent data use and processing, ensuring that relevant data can be quickly called; Step 2: Setting up a scheduled update mechanism: By setting up a scheduled task, updated data will be synchronized from the database to the local memory at a certain time period, so as to ensure that the data in the local memory can always be kept relatively up to date to meet the system operation requirements; Step 3: Select the data structure for storage: Determine to use a hashMap data structure to store the data in the local memory, set the parameter name as the key, and the corresponding parameter value as the value, to facilitate subsequent quick search, access, and operation of the data in the memory.

3. The method for obtaining real-time bullet screen messages according to claim 1, characterized in that: The Ehcache space allocation and management in S2 refers to the introduction of an efficient Ehcache caching mechanism on the external station interface server, accurately allocating independent storage space for each live broadcast room to effectively manage and quickly access live broadcast data, and deleting the corresponding space immediately after the live broadcast ends to release resources.

4. The method for obtaining real-time bullet screen messages according to claim 1, characterized in that: The specific steps for setting the Redis cache room number in S3 are as follows: Step 1: Cache storage when sending messages: When the client sends a message through the barrage message sending interface, the corresponding room number information must be stored in the Redis cache to lay the foundation for subsequent related data processing and synchronization operations; Step 2: Cache key: According to the established rules, [message_chatroom_live:room number] is used as the key to accurately identify the stored content. At the same time, the cache expiration time is set to 1 minute, and this time can be dynamically adjusted according to the parameters in S1; Step 3: Synchronization processing of expired messages: Monitor the room numbers in the cache. If no new messages are sent within the set expiration time range, the room corresponding to the room number will be removed from the synchronization list to optimize the management of synchronization resources.

5. The method for obtaining real-time bullet screen messages according to claim 1, characterized in that: The message synchronization task startup in S4 refers to the construction of a message synchronization task module on each external station interface server, with the execution frequency set to once per second. After the task is officially started, the first step is to accurately retrieve the list of rooms currently in the live broadcast state from the Redis database, and then perform cyclical in-depth processing and analysis operations on each room in sequence.

6. A method for obtaining real-time bullet screen messages according to claim 1, characterized in that: The specific steps of cache-based synchronization judgment and processing in S5 are as follows: Step 1: Cache-based synchronization judgment: By referring to the cache-related information involved in S3, a comprehensive judgment is made on whether data synchronization is necessary for the current room, so as to clarify the subsequent data processing direction; Step 2: Compare the message IDs based on priority: If the room needs to be synchronized, the message IDs of the latest message in the current room and the largest message ID after the last synchronization are compared. Step 3: Synchronize messages according to configuration: When the above size relationship is determined to meet the conditions, batches of messages are synchronized from the Redis cache to the Ehcache according to the dynamically configurable rules set in S1 to ensure the orderly flow and update of data.

7. A method for obtaining real-time bullet screen messages according to claim 1, characterized in that: The user entering the room and interacting with information in S6 refers to when the user first enters the live broadcast room, the room number is passed, and the external station interface server returns the oldestMessageId, the largest message ID set of messages of all levels in the current room.

8. The method for obtaining real-time bullet screen messages according to claim 1, characterized in that: The specific steps of message acquisition and return processing in S7 are as follows: Step 1: Client request and server judgment: The client requests the barrage message acquisition interface based on the room number and the obtained maximum message ID of each level. The external interface server then compares the message ID of each level passed by the client with the maximum message ID of its own current level. Step 2: Message acquisition based on comparison results: Messages are batched from Ehcache based on message priority. If the client's largest message messageId is smaller than the server's, the server assembles the corresponding interval messages from Ehcache into a list set and adds the largest message ID in the set to oldestMessageId. If they are equal or greater, the server processes them according to the corresponding rules and updates oldestMessageId. Step 3: Message integration and final return: After all levels of messages are obtained, the external interface server integrates the message lists of each level, sorts them in ascending order according to the reception time, and finally returns the sorted message list together with the oldestMessageId to the client.

Images