A high-performance reading and writing method, system, device and storage medium for instant messaging

Abstract

The present invention relates to a high-performance read and write method, system, device and storage medium for instant messaging. The overall storage architecture of the read and write method follows the principles of read diffusion and incremental write diffusion, so as to solve the problem of restricted read diffusion table partitioning strategy and write diffusion and write duplication problem. The read diffusion method is used to store messages. When the sender sends a message, the shard is assigned to the Hash modulus value of the sender's primary key and written into the sent message table. Each time a message is sent, it is only stored in one place, which reduces the pressure of writing, and the recipient pulls the message by himself. Incremental write diffusion is used. If the Hash modulus value of the sender's primary key is inconsistent with the Hash modulus value of the recipient's primary key, the shard is assigned to the Hash modulus value of the recipient's primary key and the full amount is copied to the received message table; this solves the problem of restricted table partitioning strategy caused by simple read diffusion.

Description

Technical Field

[0001] The present invention relates to a high-performance reading and writing method, system, device and storage medium for instant messaging, and belongs to the technical field of Internet communications. Background Art

[0002] With the rapid development of mobile Internet and communication technology, online communication has become the most important way for people to work and live. The accuracy and real-time performance of instant messaging conversations directly affect the user experience and feelings. With the surge in users, functions such as instant messaging message storage and message synchronization have become particularly important. Figure 1 As shown, in the traditional instant messaging read-write architecture, when online, the server pushes messages directly to the client; when offline, offline messages are stored in the message synchronization library, and the client pulls the synchronized messages from the message synchronization library to obtain messages, and deletes the synchronized messages in the message synchronization library to ensure that the offline client only has the latest unread messages; the server stores messages in the message storage library, and the client pulls the roaming messages in the message storage library to obtain historical messages; in this architecture, messages are saved by both the message synchronization library and the message storage library.

[0003] In addition, if Figure 2 As shown in the traditional write diffusion model, users copy the same message multiple times and then store them in the subscriber's inbox separately, so that subscribers do not have to go to my outbox to pull the message; but when there are many subscribers, it will have to be written many times, resulting in a large amount of write operations. In the read diffusion model, Figure 3 As shown in the figure, user messages are stored in their outbox. Subscribers actively go to the outbox to pull messages, and only one copy of the message is stored. However, when a subscriber subscribes to many people, all new messages of these people must be read, so the read operation task is very large.

[0004] With the massive user base, both read and write propagation in the message synchronization model generate massive amounts of data, increasing database pressure and significantly reducing read and write performance, making it difficult to ensure real-time and reliable message synchronization. While this architectural bottleneck can be addressed at the database level through methods like read-write splitting and sharding, it also leads to server resource overhead and makes it difficult to ensure data consistency and accuracy. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides a high-performance reading and writing method and system for instant messaging;

[0006] The present invention also provides a computer device and a storage medium.

[0007] The high-performance instant messaging read and write method provided by this invention employs the principles of read diffusion and incremental write diffusion to address the limitations of read diffusion table partitioning strategies and the high write workload. This read and write method is sufficient to support the reading and writing of tens of millions of data points and ensures that messages are transmitted completely and quickly from sender to receiver, preventing roaming message loss, out of order, and duplication. It also enables users to log in to any client and view all historical messages, realizing the unique features of advanced instant messaging systems.

[0008] Explanation of terms:

[0009] 1. Hash modulus value: The absolute value of the remainder of the hashCode value obtained from the sender or receiver's primary key and divided by the number of message tables is used as the hash modulus value. For example, if there are two message tables, the absolute value of either number divided by 2 is 0 or 1. Therefore, when creating a message table, the logical name suffix of the message table should range from 0 to 1.

[0010] The technical solution of the present invention is:

[0011] A high-performance reading and writing method for instant messaging, the method comprising:

[0012] Create two or more message tables in the MySQL database. Use a routing sharding strategy based on the user primary key Hash modulo. Divide the message tables into a receiving message table and a sending message table based on data storage logic. Logically, they belong to the same table and are both used to store messages. The sharding field in the message table is shard. Use this sharding strategy to distribute database pressure.

[0013] When a sender sends a message to a recipient, the shard is assigned the Hash modulo value of the sender's primary key, and the message is stored in a message table with the same suffix as the shard value. The message table at this time serves as the sent message table.

[0014] If the Hash modulus value of the sender's primary key is inconsistent with the Hash modulus value of the recipient's primary key, the shard is assigned the Hash modulus value of the recipient's primary key, and the message stored in the sent message table is copied to another message table with the same suffix as the shard value. The message table at this time is the received message table; this solves the problem of limited table partitioning strategy caused by simple read diffusion. Each time a message is sent, it is only stored in one place to reduce the pressure of writing, and the recipient pulls the message himself; if the Hash modulus value of the sender's primary key is consistent with the Hash modulus value of the recipient's primary key, the message is only stored in the sent message table. This is a read diffusion operation;

[0015] When the sender reads a message with the recipient, or when the recipient reads a message with the sender, the Shard is assigned the Hash modulo value of the primary key of the person being read, and the message table with the same suffix as the shard value in this step is read.

[0016] Preferably, according to the present invention, the name of a message table created in a MySQL database includes a physical table name and a suffix. The physical table name of the message table is im_message, and the suffix is ​​the shard value. Within the range of the shard value, the suffix of the message table is a positive integer that increases sequentially starting from 0; for example, im_message0 and im_message1. The shard value of the same message table is the same.

[0017] Preferably, according to the present invention, when the sender sends a message to the recipient, a version field exists in the message table as the version number. The specific process is: the increment function is used through the redis database to obtain the value value, and the obtained value value is assigned to the version field and loaded into the message sent by the sender to the recipient; and then the message loaded with the unique version number is stored in the message table.

[0018] The version number is guaranteed to be unique in high-concurrency scenarios. The client pulls messages based on the message version number to ensure the order of pulled messages.

[0019] The message table contains a version field. Redis is a non-relational database that stores data in a key-value format. "increment" is a Redis atomic operation. Set a key as the message version number and use it as the parameter of the "increment" method. Call this method to retrieve the value that will be used as the message version number.

[0020] Preferably, according to the present invention, when a sender reads a message with a recipient, by limiting the sender and recipient, all point-to-point message records can be directly read; by limiting the time, message records at different times can be read.

[0021] According to a preferred embodiment of the present invention, when a sender reads a message with a recipient and wants to pull the message records of user A, the sender sets the query condition to A or the recipient value to A to query all message records of user A. The sender then concatenates the version numbers to query all the latest data records that have not been synchronized with user A, and the concatenated version number is greater than the version number of the last viewed. The sender then uses the paging query condition to view messages page by page. This allows the sender to obtain all the unsynchronized records of the user, and synchronize the latest data by paging.

[0022] According to the preferred embodiment of the present invention, when the sender reads the message with the recipient, by limiting the shard value, the sent messages and received messages of the same user stored in the table can be queried, the reading data table can be accurately determined, the linked table query is reduced, and the query efficiency is improved.

[0023] A device for implementing an instant messaging high-performance reading and writing method is used to implement the above-mentioned instant messaging high-performance reading and writing method. The device includes: a storage module and a reading module. The storage module is used to implement steps (1) to (3), and the reading module is used to implement step (4).

[0024] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of a high-performance reading and writing method for instant messaging are realized.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a high-performance reading and writing method for instant messaging.

[0026] The beneficial effects of the present invention are:

[0027] 1. The present invention provides a high-performance reading and writing method for instant messaging, and its overall storage architecture follows the principles of read diffusion and incremental write diffusion, so as to solve the problem of restricted read diffusion sharding strategy and write diffusion and write duplication problem. The read diffusion method is used to store messages. When the sender sends a message, the shard is assigned to the Hash modulus value of the sender's primary key and written into the "send" message table. Each time a message is sent, it is only stored in one place, which reduces the pressure of writing, and the recipient pulls the message by himself; it can solve the problem of write diffusion and write duplication. Incremental write diffusion is used. If the Hash modulus value of the sender's primary key is inconsistent with the Hash modulus value of the recipient's primary key, the shard is assigned to the Hash modulus value of the recipient's primary key, and the full amount (including the complete message content) is copied to the "receive" message table; this solves the problem of restricted sharding strategy caused by simple read diffusion.

[0028] 2. Utilize the Redis increment atomic operation to generate a unique version number, ensuring uniqueness in high-concurrency scenarios. Clients pull messages based on their version numbers, ensuring orderly message retrieval. When pulling messages offline or switching client devices, the client compares version numbers and pulls the latest data in pages, as the full database already exists. This ensures roaming data reliability and multi-device data synchronization.

[0029] 3. The reading and writing method provided by the present invention is sufficient to support the reading and writing of more than tens of millions of data, and ensure that messages are complete and quickly transmitted from the sender to the receiver, and roaming messages are not lost, disordered, or repeated. An account can log in to any client to view all historical messages, realizing the unique functions of advanced instant messaging systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is an instant messaging reading and writing architecture in the prior art;

[0031] Figure 2 Write a diffusion diagram for the prior art;

[0032] Figure 3 It is a schematic diagram of a diffusion model in the prior art;

[0033] Figure 4 A schematic diagram of the storage model for instant messaging provided by the present invention;

[0034] Figure 5 This is a schematic diagram of the instant messaging reading model provided by the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below with reference to the embodiments and the accompanying drawings, but is not limited thereto.

[0036] Example 1

[0037] A high-performance reading and writing method for instant messaging, such as Figure 4 and Figure 5 As shown, the method includes:

[0038] Create two or more message tables in the MySQL database. In this embodiment, two message tables are created in the MySQL database, and the user Hash modulus is 0 or 1; the message table sharding strategy is a routing sharding strategy based on the user primary key Hash modulus; according to the data storage logic, it is divided into a receiving message table and a sending message table, which logically belong to the same table and are both used to store messages, and the sharding field in the message table is shard; the database pressure is dispersed through the sharding strategy.

[0039] The name of a message table consists of the physical table name and a suffix. The physical table name of the message table is im_message, and the suffix is ​​the shard value. Within the range of shard values, the suffix of the message table name is a positive integer starting from 0 and increasing in sequence. For example, in this embodiment, two or more message tables are created in the MySQL database, and the user primary key hash modulo value is 0 or 1. The two message tables are named im_message0 and im_message1. The shard value of the same message table is the same.

[0040] In this embodiment, user A is the sender and user B is the recipient. When user A sends a message to user B, the shard is assigned the hash modulo value of user A's primary key, and the message is stored in a message table with the same suffix as the shard value. The message table at this time serves as the sent message table.

[0041] When user A sends a message to user B, a version field exists in the message table as the version number. The specific process is as follows: the increment function is used through the Redis database to obtain the value, and the obtained value is assigned to the version field and loaded into the message sent by the sender to the recipient; then the message loaded with the unique version number is stored in the message table.

[0042] The version number is guaranteed to be unique in high-concurrency scenarios. The client pulls messages based on the message version number to ensure the order of pulled messages.

[0043] The message table contains a version field. Redis is a non-relational database that stores data in a key-value format. "increment" is a Redis atomic operation. Set a key as the message version number and use it as the parameter of the "increment" method. Call this method to retrieve the value that will be used as the message version number.

[0044] If the Hash modulus value of user A's primary key is inconsistent with the Hash modulus value of user B's primary key, the shard value will be assigned to the Hash modulus value of user B's primary key, and the messages stored in the sent message table will be copied to another message table with the same suffix as the shard value. The message table at this time is the received message table. This solves the problem of limited table partitioning strategy caused by simple read diffusion. Each sent message is only stored in one place, reducing the pressure of writing, and the recipient pulls the message himself. If the Hash modulus value of user A's primary key is consistent with the Hash modulus value of user B's primary key, the message is only stored in the sent message table. This is a read diffusion operation.

[0045] When user A reads a chat with user B, or user B reads a chat with user A, assign the shard value to the hash modulo value of the primary key of the user being read, and read the message table with the same suffix as the shard value in this step. Create two message tables in the MySQL database, each storing all messages for a single user, regardless of whether they are sent or received.

[0046] By limiting the sender and recipient, you can directly read all point-to-point message records; by limiting the time, you can read message records at different times.

[0047] To retrieve user A's message history, set the query condition to "Sender: User A" or "Recipient: User A" to retrieve all of user A's message history. Then, concatenate the version numbers to retrieve all of user A's latest unsynchronized data records, ensuring the concatenated version number is greater than the last viewed version number. Use the paging query condition to view messages page by page. This retrieves all of a user's unsynchronized records, allowing you to synchronize with the latest data by paging.

[0048] Specifically, user A is the sender and user B is the recipient, such as Figure 5 As shown in the figure, the specific process is: 1) User A pulls the chat with user B, assigns the Shard value to the Hash modulo of user B's primary key to determine which message table to read the message from;

[0049] 2) User B pulls the chat with user A and assigns the shard value to the modulo of the hash of user A's primary key to determine which message table to read the message from.

[0050] 3) (sender = user A and recipient = user B) or (sender = user B and recipient = user A)" query statement can directly obtain the entire content of the point-to-point chat; you can also limit the earliest record that the user can pull by adding a time filter.

[0051] 4) Add the "version>:begin" and "version<=:end" conditions to retrieve all the user's unsynchronized records. You can then synchronize to the latest data by pulling records by page.

[0052] Step (4) also includes: by limiting the shard value, it is possible to query the sent messages and received messages of the same user stored in the table, accurately determine the read data table, reduce the table query and improve the query efficiency.

[0053] Example 2

[0054] A device for implementing a high-performance reading and writing method for instant messaging is used to implement the high-performance reading and writing method for instant messaging provided in Example 1. The device includes: a storage module and a reading module. The storage module is used to implement steps (1) to (3), and the reading module is used to implement step (4).

[0055] Example 3

[0056] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the instant messaging high-performance reading and writing method provided in Example 1 are implemented.

[0057] Example 4

[0058] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the instant messaging high-performance reading and writing method provided in Example 1.

Claims

1. A high-performance reading and writing method for instant messaging, characterized in that: The method includes: (1) Create two or more message tables in the MySQL database. The message table partitioning strategy is the routing partitioning strategy based on the user primary key Hash modulo; and the partitioning field in the message table is shard; (2) When the sender sends a message to the recipient, the shard is assigned the Hash modulo value of the sender's primary key, and the message is stored in a message table with the same suffix as the shard value. The message table at this time serves as the sent message table; (3) If the Hash modulus value of the sender's primary key is inconsistent with the Hash modulus value of the recipient's primary key, the shard is assigned the Hash modulus value of the recipient's primary key, and the message stored in the sent message table is copied to another message table with the same suffix as the shard value. The message table at this time is the received message table; (4) When the sender reads the message with the recipient, or when the recipient reads the message with the sender, the Shard is assigned the Hash modulo value of the primary key of the person being read, and the message table with the same suffix as the shard value in this step is read.

2. The high-performance reading and writing method for instant messaging according to claim 1, characterized in that: The name of the message table created in the MySQL database consists of the physical table name and the suffix. The physical table name of the message table is im_message, and the suffix is ​​the shard value. The suffix of the message table is a positive integer that increases from 0 within the range of the shard value. The suffix of the message table is the same.

3. The high-performance reading and writing method for instant messaging according to claim 1, characterized in that: When a sender sends a message to a recipient, a version field is created in the message table as the version number. The specific process is as follows: the increment function is used to obtain the value from the Redis database, and the obtained value is assigned to the version field and loaded into the message sent from the sender to the recipient; the message with the unique version number is then stored in the message table.

4. The high-performance reading and writing method for instant messaging according to claim 1, characterized in that: When the sender reads the message with the recipient, by limiting the sender and recipient, all point-to-point message records can be directly read; by limiting the time, message records at different times can be read.

5. The high-performance reading and writing method for instant messaging according to claim 1, characterized in that: When the sender reads the message with the recipient, if you want to pull the message records of user A, you can query all the message records of user A by setting the query condition to A or the recipient value to A; then you can query all the latest data records of user A that have not been synchronized by concatenating the version numbers, and the concatenated version number must be greater than the version number last viewed; and you can view messages page by page using the paging query condition.

6. The high-performance reading and writing method for instant messaging according to claim 1, characterized in that: When the sender reads the message with the recipient, by limiting the shard value, the sent and received messages of the same user stored in the table can be queried.

7. A device for implementing a high-performance reading and writing method for instant messaging, characterized in that: The device is used to implement the high-performance instant messaging reading and writing method described in any one of claims 1-6, and includes: a storage module and a reading module, the storage module is used to implement steps (1) to (3), and the reading module is used to implement step (4).

8. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the instant messaging high-performance reading and writing method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the instant messaging high-performance reading and writing method according to any one of claims 1 to 6 are implemented.

Images