Message processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a message processing method and device, electronic equipment and a storage medium, and belongs to the technical field of computers. The message processing method comprises the following steps: receiving a user message; storing the user message in a preset data structure; based on the delay time, obtaining a user message in the delay time from a preset data structure as a target message; and processing the target message. According to the embodiment of the invention, the comprehensiveness and integrity of message processing can be improved, message delay processing is facilitated, flow flood peaks are prevented, the message processing efficiency and stability are improved, and the efficiency of a system is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a message processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In related technologies, when a system processes messages, there may be traffic surges. The system needs to process messages in batches within a short period of time. During traffic surges, the system capacity decreases, data processing efficiency is low, and the system may become overloaded.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this application is to provide a message processing method, apparatus, electronic device, and storage medium, which aims to improve message processing speed and system efficiency.

[0005] To achieve the above objectives, one aspect of this application provides a message processing method, which includes the following steps:

[0006] Receive user messages;

[0007] The user messages are stored in a preset data structure;

[0008] Based on the delay time, the user message within the delay time is obtained from the preset data structure as the target message;

[0009] The target message is processed.

[0010] In some embodiments, storing the user message in a preset data structure includes:

[0011] Query the preset data structure to see if there is a historical message corresponding to the user message;

[0012] If a historical message corresponding to the user message exists in the preset data structure, the data content of the historical message is updated in the preset data structure according to the user message, and the write time information of the historical message is retained.

[0013] If there is no historical message corresponding to the user message in the preset data structure, then the user message is added to the preset data structure.

[0014] In some embodiments, the preset data structure includes a first data structure, a second data structure, a third data structure, and a fragment file. The first data structure is used to store index information corresponding to the user message. The second data structure is used to store the write time information of the user message. The third data structure is used to store fragment file information and offset information corresponding to the user message. The fragment file is used to store the data content of the user message. The third data structure is associated with the first data structure and the second data structure.

[0015] In some embodiments, storing the user message in a preset data structure includes:

[0016] The user message is stored in the fragment file, and the first fragment file information and the first offset information corresponding to the user message are recorded.

[0017] Query the preset data structure to see if there is a historical message corresponding to the user message;

[0018] If a historical message corresponding to the user message exists in the preset data structure, then the second fragment file information corresponding to the historical message is replaced by the first fragment file information, and the second offset information corresponding to the historical message is replaced by the first offset information.

[0019] In some embodiments, the number of fragment files is multiple; storing the user message into the fragment files includes:

[0020] Based on a pre-defined hash function, the user message is hashed to obtain the corresponding hash value;

[0021] Based on the hash value, a target fragment file is determined from the plurality of fragment files, and the user message is stored in the target fragment file.

[0022] In some embodiments, if a historical message corresponding to the user message exists in the preset data structure, then replacing the second fragment file information corresponding to the historical message with the first fragment file information, and replacing the second offset information corresponding to the historical message with the first offset information, includes:

[0023] If a historical message corresponding to the user message exists in the preset data structure, query the first node corresponding to the historical message in the first data structure, and obtain the second index information corresponding to the historical message from the first node;

[0024] Based on the second index information, a second node corresponding to the historical message is determined in the third data structure; wherein, the second node is used to store the second fragment file information and the second offset information corresponding to the historical message;

[0025] The first fragment file information replaces the second fragment file information in the second node, and the first offset information replaces the second offset information in the second node.

[0026] In some embodiments, storing the user message in a preset data structure further includes:

[0027] If there is no historical message corresponding to the user message in the preset data structure, establish the first index information corresponding to the user message and record the writing time information of the user message;

[0028] In the first data structure, a third node corresponding to the user message is created; in the second data structure, a fourth node corresponding to the user message is created; and in the third data structure, a fifth node corresponding to the user message is created.

[0029] The first index information is stored in the third node, the write time information of the user message is stored in the fourth node, and the first fragment file information and the first offset information are stored in the fifth node.

[0030] In some embodiments, retrieving user messages within the preset data structure as target messages based on a delay time includes:

[0031] If the delay time is a preset invalid value, then the user messages are sequentially determined as target messages from the preset data structure according to the order corresponding to the write time information of each user message;

[0032] If the delay time is not a preset invalid value, the user message within the delay time is obtained from the preset data structure as the target message based on the write time information of each user message.

[0033] In some embodiments, the method uses at least one engine, each engine being configured with a corresponding preset data structure;

[0034] The step of storing the user message in a preset data structure includes:

[0035] Obtain the pre-set filter conditions;

[0036] Based on the filtering conditions, the target engine corresponding to the user message is determined among the various engines;

[0037] The user messages are stored based on a preset data structure corresponding to the target engine.

[0038] To achieve the above objectives, another aspect of this application provides a message processing apparatus, the apparatus comprising:

[0039] The message receiving module is used to receive user messages;

[0040] A storage module is used to store the user messages in a preset data structure;

[0041] The message acquisition module is used to acquire user messages within the preset delay time from the preset data structure as target messages based on the preset delay time.

[0042] The processing module is used to process the user messages.

[0043] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the message processing method described above.

[0044] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the message processing method described above.

[0045] The solution in this application embodiment receives user messages; stores user messages in a preset data structure; obtains user messages within the preset data structure as target messages based on a delay time; and processes the target messages. This facilitates message delay processing, prevents traffic surges, improves the comprehensiveness and integrity of message processing, enhances message processing efficiency and stability, and improves system efficiency. Attached Figure Description

[0046] Figure 1 This is a flowchart of the message processing method provided in the embodiments of this application;

[0047] Figure 2 This is another flowchart of the message processing method provided in the embodiments of this application;

[0048] Figure 3 This is a flowchart of the message continuity verification steps of the message processing method provided in this application embodiment;

[0049] Figure 4 This is a flowchart of the allocation engine steps of the message processing method provided in the embodiments of this application;

[0050] Figure 5 This is a flowchart of the rule creation steps of the message processing method provided in this application embodiment;

[0051] Figure 6 This is a flowchart of the message consumption steps of the message processing method provided in the embodiments of this application;

[0052] Figure 7 This is a flowchart illustrating the application of the message processing method provided in this embodiment to the TIL engine;

[0053] Figure 8 This is a schematic diagram illustrating the message storage method in the fragmented file provided in an embodiment of this application;

[0054] Figure 9 This is a schematic diagram of TI L fission provided in an embodiment of this application;

[0055] Figure 10 This is a schematic diagram of the data merging process provided in an embodiment of this application;

[0056] Figure 11 This is a schematic diagram of the newly added node provided in the embodiments of this application;

[0057] Figure 12 This is a schematic diagram of the deletion node provided in an embodiment of this application;

[0058] Figure 13 This is a schematic diagram of the structure of the message processing device provided in the embodiments of this application;

[0059] Figure 14 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0061] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0062] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0064] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0065] 1) The TI L (Time-Indexed Linked List Engine) is a custom engine configured with the TI L data structure. Here, T stands for time, I for index, and L for the linked list storing messages. This data structure combines the features of a binary search tree (BST) and a doubly linked list for efficient message storage, retrieval, and management. This engine can read and write data (such as user messages) based on this data structure.

[0066] 2) A BST (Binary Search Tree) is a data structure where each node has at most two child nodes. For each node in the tree, the value of all items in its left subtree is less than the value of the node, and the value of all items in its right subtree is greater than the value of the node.

[0067] 3) A doubly linked list is a linked data structure consisting of a series of nodes. Each node contains a data field and two pointer fields. These pointers point to the previous node (the predecessor node) and the next node (the successor node), respectively.

[0068] 4) ZooKeeper is used to manage coordination tasks in large distributed systems.

[0069] 5) etcd is a distributed key-value store system used to store data in distributed systems and ensure strong data consistency.

[0070] In related technologies, when a system processes messages, there may be traffic surges. The system needs to process messages in batches within a short period of time. During traffic surges, the system capacity decreases, data processing efficiency is low, and the system may become overloaded.

[0071] In summary, the technical problems existing in the relevant technologies need to be improved.

[0072] In view of this, embodiments of this application provide a message processing method, apparatus, electronic device, and storage medium. This solution receives user messages; stores user messages in a preset data structure; obtains user messages within the preset data structure as target messages based on a delay time; and processes the target messages. This facilitates message delay processing, prevents traffic surges, improves the comprehensiveness and integrity of message processing, enhances message processing efficiency and stability, and improves system efficiency.

[0073] The message processing method provided in this application relates to the field of computer technology. The message processing method provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the message processing method, but is not limited to the above forms.

[0074] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0075] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0076] Figure 1 This is an optional flowchart of the message processing method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S104.

[0077] Step S101: Receive user message.

[0078] This application does not impose specific limitations on the application scenarios involved in user messages, which can be flexibly selected based on actual processing needs.

[0079] For example, in some embodiments, user messages can be document messages, such as when a user uploads a document-type message through a platform and shares and discusses these documents with others; in other embodiments, user messages can be text messages, such as when a user enters a text-type user message through a chat application or customer service system. Of course, it is understood that the types of user messages in the embodiments of this application are not limited to these.

[0080] For example, in some embodiments, the user message can be either a review message or a backup message. The user message can be used to instruct processing of user data associated with a user. For instance, when the user message is a review message, it instructs the user data to be reviewed, which may include sensitive word identification, typo identification, and key point identification. As another example, when the user message is a backup message, it instructs the user data to be backed up.

[0081] Step S102: Store the user message in a preset data structure.

[0082] In this step, upon obtaining a user message, the user message can be stored in a preset data structure. Here, the specific type of the preset data structure is not limited in this application. For example, the preset data structure can include, but is not limited to, at least one of arrays, linked lists, stacks, queues, and trees. It should be noted that in this embodiment, when storing the user message, some information associated with the user message can be stored synchronously to facilitate subsequent consumption or querying of the corresponding user message. Consuming the user message refers to data processing of the user message and / or the information associated with the user message. For example, the information associated with the user message can include the user message's index information, the user message's write time information (or the time information of receiving the user message), and the user message's storage parameter information, etc. This application does not limit this; the preset data structure can store the user message and the information associated with the user message.

[0083] For example, in some embodiments, the preset data structure may include a first data structure, a second data structure, a third data structure, and a fragment file. The first data structure may be used to store index information corresponding to the user message; the second data structure may be used to store the write time information of the user message; the third data structure may be used to store fragment file information and offset information corresponding to the user message; the fragment file may be used to store the data content of the user message; and the third data structure is associated with the first and second data structures. In this embodiment, the first data structure may be a linked list, and the second and third data structures may be trees (such as binary trees). Of course, the specific types can be selected according to actual needs, and this application does not limit this.

[0084] It is understood that in this embodiment, different data structures can be used to store user messages and related information separately. This allows for quick retrieval of relevant results during subsequent data consumption or querying. For example, when querying user messages, the existence of the corresponding user message can be quickly determined using the index information in the first data structure; when querying the write time of a user message (e.g., processing user messages with a delay time), the write time information in the second data structure can provide the result. Furthermore, the first, second, and third data structures, along with the fragmented file, can be configured into a TIL engine.

[0085] The number of fragment files in this application embodiment can be multiple. In some embodiments, when storing user messages into fragment files, the first fragment file information and the first offset information corresponding to the user messages can be recorded. Here, the first fragment file information can be used to characterize which fragment file the user messages are stored in, and the first offset information can be used to characterize the specific storage location of the user messages in the fragment files.

[0086] Optionally, user messages are stored in a fragment file, including:

[0087] Based on a pre-defined hash function, the user message is hashed to obtain the corresponding hash value;

[0088] Based on the hash value, the target fragment file is determined from multiple fragment files, and the user message is stored in the target fragment file.

[0089] In this embodiment, when storing user messages into fragment files, firstly, a suitable hash function can be selected as a pre-defined hash function. Common hash functions include MD5, SHA-1, and SHA-256, and this application does not limit this choice. Then, the pre-defined hash function can be used to perform hash calculations on the user messages to obtain the corresponding hash values. For each fragment file, a storage area can be pre-allocated to it. For example, a virtual ring space can be established, and the fragment files can be evenly distributed to some points in the ring space. Then, the calculated hash values ​​can be mapped to the ring space, and the fragment file corresponding to the point closest to it in a specified direction in the ring space can be determined. This fragment file is then identified as the target fragment file, and the user messages can be stored in the target fragment file.

[0090] In this embodiment of the application, a distributed file system or object storage service can be used to store fragmented files in order to improve scalability and fault tolerance.

[0091] Optionally, in this embodiment of the application, the message processing method can use multiple engines. As described above, each engine can be configured with a corresponding preset data structure. The data structures corresponding to different engines can be the same or different. When storing user messages, relevant filtering conditions can be preset; then, the target engine corresponding to the user message can be determined according to the filtering conditions, thereby storing the user message in the target engine. It is understood that setting multiple engines and assigning a target engine to each user message helps reduce server pressure and improve the concurrent processing capability of user messages.

[0092] In some embodiments, for certain types of user messages, there may be multiple versions. For example, if a user uploads an edited document message, only the newly uploaded user message needs to be stored each time it is transmitted, requiring merging and updating of previously uploaded user messages. Therefore, in this embodiment, when storing user messages, a query can be performed in a preset data structure to check if there are any historical messages corresponding to the user message. Here, historical messages refer to user messages uploaded at historical nodes. If historical messages exist, it means that the user message and historical messages need to be merged. Specifically, in this embodiment, the data content of historical messages can be updated in the preset data structure, while retaining the write time information of historical messages. In this way, the updating and merging of newly written messages can be performed quickly, while retaining the initial write time information of the same group of user messages, thereby improving the efficiency of message processing.

[0093] Step S103: Based on the delay time, retrieve the user message within the delay time from the preset data structure as the target message.

[0094] In some embodiments, the delay time is obtained.

[0095] Specifically, the delay time is the time for message processing to be delayed. The delay time can be specified by the user or automatically generated by the system.

[0096] Optionally, the delay time can be a delay timestamp.

[0097] It should be noted that when the user specifies a delay time, the delay time is set to the user-specified value. If the user does not specify a delay time, the system automatically sets the delay time to a preset invalid value. For example, this preset invalid value can be a numerical value without time meaning, such as -1, or other characters. This application does not impose any restrictions on this. When the delay time is a preset invalid value, it means that no message delay processing is performed on the user message.

[0098] In one embodiment, after determining the target message, the target message is first processed (consumed), and if successful, the relevant information of the target message is then deleted.

[0099] In another embodiment, regardless of whether the processing is successful or not, as long as processing (consumption) is performed, the relevant information of the target message is deleted.

[0100] Specifically, if the delay time is a preset invalid value, the sixth node is obtained through the root node of the third data structure. Based on the sixth node, the target message, the seventh node, and the eighth node are determined, and the seventh and eighth nodes are deleted. The sixth node is associated with the seventh and eighth nodes. The seventh node is the node corresponding to the sixth node in the first data structure, and the eighth node is the node corresponding to the sixth node in the second data structure. The sixth, seventh, and eighth nodes store data related to the target message.

[0101] Specifically, if the delay time is not a preset invalid value, the ninth node is obtained from the second data structure based on the delay time, the ninth node is deleted, the target message, the tenth node, and the eleventh node are determined based on the ninth node, and the tenth node and the eleventh node are deleted. The tenth node is the node corresponding to the ninth node in the third data structure, and the eleventh node is the node corresponding to the tenth node in the first data structure. The ninth, tenth, and eleventh nodes store data related to the target message.

[0102] Step S104: Process the target message.

[0103] Optionally, the target message can be read based on the fragment file information and offset information corresponding to the target message.

[0104] Furthermore, the target message is written into the consumption chain.

[0105] In step S104 of some embodiments, the target message is written to the linked list corresponding to the consumer, and the target message is recorded in the message encoding (ID) and node mapping (MAP).

[0106] Further, process (consume) the target message and delete the node corresponding to the target message from the consumption chain.

[0107] In some embodiments, once the target message has been processed, information related to the target message is deleted.

[0108] In some embodiments, if the target message has been processed, the node corresponding to the target message in the consumer chain is deleted.

[0109] After receiving a message, the consumer returns the message encoding, searches for the corresponding linked list node in the mapping based on the message encoding, and deletes the linked list node.

[0110] Furthermore, the linked list nodes are deleted and archived.

[0111] Steps S101 to S104 as shown in the embodiments of this application involve receiving user messages; storing user messages in a preset data structure; obtaining user messages within the delay time from the preset data structure as target messages based on the delay time; and processing the target messages. This facilitates message delay processing, prevents traffic surges, improves the comprehensiveness and integrity of message processing, enhances message processing efficiency and stability, and improves system efficiency.

[0112] Figure 2 This is an optional flowchart of the message processing method provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, steps S201 to S207.

[0113] Step S201: Store the user message in the fragment file and record the first fragment file information and the first offset information corresponding to the user message.

[0114] This application does not impose specific limitations on user messages, and they can be flexibly selected according to actual processing needs. For example, user messages can be document messages or text messages.

[0115] This application does not impose specific limitations on the method of distributing user messages, and can flexibly select the method according to actual processing needs. For example, the storage location of user messages can be determined by consistent hashing, or it can be determined by random selection.

[0116] Optionally, the storage location of user messages can be determined through consistent hashing.

[0117] In step S201 of some embodiments, the first fragment file information is determined by consistent hashing, and then the first offset information is determined according to the available storage location in the fragment file corresponding to the first fragment file information.

[0118] Distributed file systems or object storage services can be used to store fragmented files to improve scalability and fault tolerance.

[0119] In some embodiments, the system records the first fragment file information and the first offset information of the user message.

[0120] In this embodiment, user messages are obtained, stored in fragment files allocated by consistent hashing, and the first fragment file information and first offset information of the user messages are obtained. This facilitates the persistent storage of user messages, thereby enabling rapid location and retrieval of messages.

[0121] Step S202: Search for the historical message corresponding to the user message in the preset data structure.

[0122] Specifically, the first data structure stores the message index and can be used for message searching.

[0123] Optionally, the first data structure can implement multi-level conditional search through node nesting.

[0124] In some embodiments, a user message is searched in a first data structure to determine whether a historical message corresponding to the user message exists in the first data structure. Specifically, if the message index corresponding to the user message is found in the first data structure, it can be determined that the user message has been found.

[0125] In this embodiment, searching for user messages in the first data structure helps determine whether the corresponding historical messages exist, thus preparing for subsequent updates or the creation of new nodes.

[0126] Step S203: If the preset data structure contains historical messages corresponding to user messages, then obtain the node corresponding to the user message in the first data structure as the first node, determine the node corresponding to the third data structure as the second node based on the first node, and update the information stored in the second node based on the first fragment file information and the first offset information.

[0127] Specifically, the first node is the node corresponding to the historical message in the first data structure. The first node stores the second index information of the historical message. Based on the second index information, the node corresponding to the user message in the third data structure, i.e., the second node, can be determined. The second node stores the second fragment file information and the second offset information corresponding to the historical message. After determining the second node, the first fragment file information of the current user message can be used to replace the second fragment file information in the second node, and the first offset information of the current user message can be used to replace the second offset information in the second node. This eliminates the need to add or delete data in the fragment files; only the pointer position of the information needs to be adjusted to quickly replace the old and new messages. Moreover, by replacing the old and new messages, the old and new messages are merged. The message only needs to be processed once, instead of being processed repeatedly, improving message processing efficiency and saving computing power. Furthermore, it should be noted that the second data structure storing the write time information remains unchanged. That is, during the merging of the old and new messages, the write time information of the old message is retained, ensuring that the message can still be processed in a timely manner within the preset delay time.

[0128] Step S204: If there is no historical message corresponding to the user message in the preset data structure, add a third node in the preset data structure, add a fourth node in the second data structure, and add a fifth node in the third data structure according to the user message.

[0129] Specifically, the third node is the node corresponding to the user message in the first data structure, and the third node is used to store the first index information corresponding to the user message. The fourth node is the node corresponding to the user message in the second data structure, and the fourth node is used to store the write time information of the user message. The fifth node is used to associate the third node and the fourth node, and it is used to store the first fragment file information and the first offset information corresponding to the user message.

[0130] In some embodiments, if the first data structure does not contain historical messages corresponding to user messages, then nodes corresponding to user messages need to be created in the first data structure, the second data structure, and the third data structure, respectively.

[0131] It is understandable that a fifth node is added to the third data structure using the first fragment file information and the first offset information. The fifth node can be used to store the first fragment file information and the first offset information.

[0132] A fifth node is added to the end of the third data structure.

[0133] In this embodiment, if the preset data structure does not contain historical messages corresponding to user messages, a third node is added to the first data structure, a fourth node is added to the second data structure, and a fifth node is added to the third data structure based on the user messages. By adding and initializing new messages, the comprehensiveness and completeness of message processing can be improved.

[0134] Step S205: Obtain the delay time.

[0135] The delay time can be specified by the user or automatically generated by the system.

[0136] Optionally, the delay time can be a delay timestamp.

[0137] It should be noted that when the user specifies a delay time, the delay time is set to the user-specified value. If the user does not specify a delay time, the system automatically sets the delay time to a preset invalid value. For example, the preset invalid value can be a numerical value without time meaning, such as -1, or other characters. This application does not impose any restrictions on this. In this case, the preset invalid value for the delay time indicates that no message delay processing is performed on the user message.

[0138] In this embodiment, the delay time is obtained, which prepares for the subsequent implementation of message delay processing.

[0139] Step S206: If the delay time is a preset invalid value, obtain the sixth node through the root node of the third data structure, and determine and delete the seventh and eighth nodes based on the sixth node.

[0140] Specifically, the sixth node represents the node corresponding to the message (i.e., the target message) that is about to be consumed (equivalent to being processed). The sixth node is associated with the seventh and eighth nodes. The seventh node is the node corresponding to the sixth node in the first data structure, and the eighth node is the node corresponding to the sixth node in the second data structure. The default invalid value is a predetermined invalid delay time value. The third data structure is associated with the first and second data structures. The first data structure is used to store the message index, and the second data structure is used to store the message write time.

[0141] The default invalid value can be determined by the user or by the system.

[0142] Optionally, the sixth node is the node following the root node of the third data structure.

[0143] Understandably, the nodes in the third data structure are used to associate the nodes in the first data structure with the nodes in the second data structure. Specifically, the sixth node is associated with the seventh node and the eighth node.

[0144] Optionally, a garbage collection mechanism can be implemented to release resources that are no longer needed. Garbage collection (GC) is an automated memory management technique that tracks the allocation and usage of all objects and automatically releases the memory occupied by objects that are no longer in use. For example, in some programming languages ​​such as Java, C#, and Python, developers do not need to manually allocate and release memory; these tasks are handled by the built-in garbage collector. When an object is no longer pointed to by any valid reference, it becomes a garbage collection object and is thus released. This approach greatly simplifies memory management tasks and reduces the probability of memory leaks and other problems. In this embodiment, promptly deleting related user messages after they are consumed also achieves the effect of releasing resources that are no longer needed.

[0145] It is understandable that the time complexity of obtaining the sixth node through the root node of the third data structure, and determining and deleting the seventh and eighth nodes based on the sixth node is close to O(1).

[0146] In some embodiments, the third data structure uses a first linked list. Specifically, if the delay time is a preset invalid value, the user message is retrieved as the target message from the first linked list in the preset data structure.

[0147] In this embodiment, if the delay time is a preset invalid value, the sixth node is obtained through the root node of the third data structure, and the seventh and eighth nodes are determined and deleted based on the sixth node. This is beneficial for message consumption at the linked list level and improves message processing efficiency.

[0148] Step S207: If the delay time is not a preset invalid value, obtain the ninth node in the second data structure according to the delay time, delete the ninth node, and determine and delete the tenth and eleventh nodes according to the ninth node.

[0149] Specifically, the ninth node is the earliest generated node in the second data structure. The ninth node includes the write time, and the write time of the ninth node is earlier than the delay time. The tenth node is the node corresponding to the ninth node in the third data structure. The eleventh node is the node corresponding to the tenth node in the first data structure. The tenth node is associated with the ninth node and the eleventh node. The ninth node, the tenth node, and the eleventh node store data related to the target message. The ninth node, the tenth node, and the eleventh node are associated with the target message.

[0150] It is understandable that the target message can be obtained based on the ninth, tenth, and eleventh nodes, and the ninth, tenth, and eleventh nodes need to be deleted after the target message has been consumed.

[0151] In some embodiments, the earliest generated node in the second data structure is obtained. If the write time of the earliest generated node is earlier than the delay time, the earliest generated node is taken as the ninth node, the ninth node is deleted, and the tenth and eleventh nodes are determined and deleted based on the ninth node. Otherwise, no data is popped.

[0152] Optionally, the ninth node is the earliest message obtained by subsequent traversal of the second data structure.

[0153] It is understandable that the ninth node represents the node corresponding to the delayed consumption message (i.e., the target message). When the ninth, tenth, and eleventh nodes are deleted, the target message is consumed (i.e. processed).

[0154] In some embodiments, the second data structure adopts a second binary tree. Specifically, if the delay time is not a preset invalid value, the user message within the delay time is obtained from the second binary tree in the preset data structure as the target message.

[0155] In this embodiment, if the delay time is not a preset invalid value, the ninth node is obtained from the second data structure based on the delay time, the ninth node is deleted, and the tenth and eleventh nodes are determined and deleted based on the ninth node. This is beneficial for message delay processing, preventing traffic surges, and improving message processing efficiency and stability.

[0156] In other words, in this embodiment, the user can set a delay time to switch the consumption mode of user messages. If the delay time is not a preset invalid value, the user messages within the delay time are obtained from the preset data structure as target messages and consumed. If the delay time is a preset invalid value, consumption starts from the root node of the linked list to achieve the first-in-first-out (FIFO) technical effect.

[0157] Specifically, steps S201 to S203 as shown in the embodiments of this application involve obtaining a delay time; if the delay time is a preset invalid value, obtaining the sixth node through the root node of the third data structure, and determining and deleting the seventh and eighth nodes based on the sixth node; if the delay time is not a preset invalid value, obtaining the ninth node in the second data structure based on the delay time and deleting the ninth node. This facilitates message delay processing, prevents traffic surges, improves message processing efficiency and stability, and enhances system efficiency.

[0158] Please see Figure 3 In some embodiments, step S102 may include, but is not limited to, steps S301 to S306:

[0159] Step S301: Generate the current message sequence number for the user message and store the current message sequence number in the current fragment file.

[0160] Optionally, step S301 can be performed after step S101.

[0161] In step S301 of some embodiments, a current message sequence number is generated for the user message and stored in the current fragment file, and the engine is notified asynchronously for processing.

[0162] In this embodiment, a current message sequence number is generated for the user message and stored in the current fragment file, which prepares for subsequent comparison of historical message sequence numbers with the current message sequence number.

[0163] Step S302: Obtain the historical message sequence number.

[0164] In step S302 of some embodiments, the historical message sequence number is obtained through the engine.

[0165] Optionally, the sequence number of historical messages can be obtained from ZooKeeper, the etcd key-value store, or a file.

[0166] In this embodiment, the historical message sequence number is obtained to prepare for subsequent comparison between the historical message sequence number and the current message sequence number.

[0167] Step S303: Determine whether the historical message sequence number and the current message sequence number are consecutive.

[0168] In step S303 of some embodiments, the engine compares the historical message sequence number with the current message sequence number to determine whether the historical message sequence number and the current message sequence number are consecutive.

[0169] In this embodiment, determining whether the historical message sequence number is consecutive with the current message sequence number prepares for the subsequent determination of user messages, which helps to improve the orderliness and continuity of message processing.

[0170] Step S304: If the historical message sequence number is not consecutive with the current message sequence number, then obtain the first message sequence number that is consecutive with the historical message sequence number.

[0171] In step S304 of some embodiments, if the current message sequence number is 1003 and the historical message sequence number is 1001, then the historical message sequence number is not continuous with the current message sequence number, and the first message sequence number that is continuous with the historical message sequence number is determined to be 1002.

[0172] In this embodiment, if the historical message sequence number is not consecutive with the current message sequence number, the first message sequence number that is consecutive with the historical message sequence number is obtained, which prepares for the subsequent determination of user messages.

[0173] Step S305: Obtain the first message in the file layer according to the first message sequence number.

[0174] In step S305 of some embodiments, the first message content corresponding to the first message sequence number is obtained in the file layer by means of the first message sequence number.

[0175] Step S306: Treat the first message as a user message.

[0176] In some embodiments, the first message is treated as a user message. Further, the user message is searched within the first data structure.

[0177] Optionally, step S102 can be performed after step S306.

[0178] In this embodiment, the first message is treated as a user message, which helps to improve the orderliness and continuity of message processing.

[0179] Please see Figure 4 In some embodiments, step S102 may include, but is not limited to, steps S401 to S403:

[0180] Step S401: Obtain the filtering conditions.

[0181] Specifically, the filtering conditions are used to assign the TIL engine to the message.

[0182] Step S402: Assign a target engine to the user message based on the filtering conditions.

[0183] In step S402 of some embodiments, when consuming user messages, the corresponding target engine is selected according to the filtering conditions.

[0184] Understandably, when server load is high, the split TIL engine can be used to reduce server load. Furthermore, messages can be assigned to different TIL engines for processing based on filtering conditions, reducing data processing pressure.

[0185] Optionally, the core TI L engine can be split. If a user message is assigned to a new engine, the historical data in the target engine can be cleaned up and written to the new target engine.

[0186] Step S403: Store the user message in the preset data structure corresponding to the target engine.

[0187] In step S403 of some embodiments, user messages are stored in the target engine to prepare for subsequent determination of whether user messages exist.

[0188] Please see Figure 5 In some embodiments, the message processing method provided in this application further includes a rule creation step, which may include, but is not limited to, steps S501 to S504:

[0189] Step S501: Create the first conditional rule and obtain the filtering conditions.

[0190] Specifically, the first conditional rule is the conditional rule that the user wants to create, and the filtering condition is the existing conditional rule.

[0191] Step S502: Perform condition overlap calculation on the first condition rule and the filtering condition to obtain the calculation result.

[0192] In step S502 of some embodiments, the first conditional rule is compared with the currently running filter (i.e., the filter condition) to calculate the conditional overlap; if there is a conflict, the creation fails.

[0193] Step S503: If the calculation result shows that the first condition rule does not conflict with the filtering condition, add the first condition rule to the filtering condition.

[0194] The process involves adding the first condition rule to the filter conditions, initializing the filter (which is configured with filter conditions), and then, after writing the message to the fragment file, executing the filter conditions to determine the data engine TI L node.

[0195] Furthermore, the data from the default TI L (i.e., the target engine) is gradually migrated to the new TI L (i.e., the target engine) according to the conditions.

[0196] Step S504: If the calculation result shows that the first condition rule conflicts with the filtering condition, then the creation of the first condition rule fails.

[0197] Understandably, if the first condition rule conflicts with the filtering condition, the creation of the first condition rule will fail, and the filtering condition will not be updated. This avoids the situation where the same user message is assigned to different engines for processing, which would lead to duplicate processing.

[0198] Please see Figure 6 Step S104 may include, but is not limited to, steps S601 to S606:

[0199] Step S601: Generate a consumption pop log based on the fragment file information and offset information corresponding to the target message.

[0200] Specifically, the pop logs are consumed for backing up and restoring target messages.

[0201] In step S601 of some embodiments, if the target message is lost or reports an error, the target message is recovered by consuming the popped log.

[0202] In this embodiment, recording the consumption pop log is beneficial for recovering the target message and for preventing message loss or duplicate consumption.

[0203] Step S602: Write the target message into the consumer chain list.

[0204] In step S602 of some embodiments, the target message is written into the linked list CL corresponding to the consumer, and the target message is recorded in message encoding and node mapping.

[0205] In this embodiment, the target message is written into the consumption list. The use of the list (CL) and the map (map) can help to effectively manage and allocate system resources, avoiding resource waste and redundancy.

[0206] Step S603: Consume the target message.

[0207] Step S604: Delete the node corresponding to the target message in the consumer chain.

[0208] In some embodiments, if the target message has been processed, the node corresponding to the target message in the consumer chain is deleted.

[0209] After receiving a message, the consumer returns the message encoding, searches for the corresponding linked list node CL(m) in the mapping map based on the message encoding, and deletes the linked list node CL(m).

[0210] Furthermore, the linked list node CL(m) is deleted and archived.

[0211] In this embodiment, deleting the node corresponding to the target message in the consumption chain is beneficial for the message to be consumed and archived correctly, and helps to improve the consistency and integrity of the message.

[0212] Figure 7 This is an optional flowchart of the message processing method provided in this application embodiment applied to the TIL engine. In some embodiments, the first data structure of the preset data structure adopts a binary search tree, the second data structure adopts a second binary search tree, the first linked list adopts a first linked list, and the TIL engine includes a first binary search tree "I", a second binary search tree "T", and a first linked list "L". Figure 7 The method may include, but is not limited to, steps S701 to S712.

[0213] Step S701: Perform consistent hashing on message x and allocate fragment files.

[0214] In step S701 of some embodiments, message x is a user message.

[0215] The fragment files store the actual message data. Newly arrived messages are randomly assigned to a file using consistent hashing.

[0216] It is understandable that this is equivalent to storing the user data of the user message into the allocated fragment file, and obtaining the first fragment file information and the first offset information corresponding to the user message.

[0217] Furthermore, performing consistent hashing on message x and allocating fragmented files helps avoid excessively large files, facilitates expansion, and speeds up anomaly recovery.

[0218] All user message-related information is stored in TIL, while the user message data content is only stored in files. When excessive memory is written to and the machine's memory is insufficient, TIL cannot add nodes. If memory is not expanded, it means that all messages pushed in by PUSH data are temporarily stored in fragmented files and cannot be consumed.

[0219] Furthermore, to prevent fragmented file data from merging, for example, messages in fragmented files are stored in the following manner: Figure 8 As shown. The metadata includes message content and personalized attributes.

[0220] Optionally, a topic can be recorded in a single-machine information log, and this topic can be backed up to storage (zk & etcd & files) periodically. The topic is used to store messages.

[0221] For example, the TOPIC structure is as follows:

[0222]

[0223] Understandably, in order to reduce hardware failure rate and improve processing efficiency, the producer's messages (i.e., user messages) are stored in a fragmented manner using consistent hashing, and the file where the returned message is stored and the offset of the file where the message is located are used to generate message metadata.

[0224] Step S702: Generate a sequence number for the new message x data and record it in the file, and asynchronously notify the target engine TI L for processing.

[0225] In step S702 of some embodiments, the sequence number is the current message sequence number.

[0226] This is understandable; it's equivalent to generating a current message sequence number for a user message and storing that sequence number in the corresponding fragment file.

[0227] When the backend server is under heavy pressure, it is very important for the middleware (queue) to support degradation. This can be achieved through the split TIL.

[0228] In some embodiments, conditional rules are created and their conditional overlap is calculated with that of currently running filters; if there is a conflict, the creation fails.

[0229] Understandably, when new data arrives, the filter is initialized, and after the message is written to the fragment file, the filter conditions are used to determine the data engine TIL node.

[0230] Understandably, by default, historical data from TIL is gradually migrated to the new TIL based on certain conditions.

[0231] It should be noted that when a consumer (referring to the business that consumes user messages) consumes messages, it selects the corresponding TIL based on the conditions; otherwise, it defaults to consuming the main TIL engine.

[0232] It is worth noting that all fission only applies to the default TIL engine and cannot be fissioned again for TIL engines that have already been fissed. The solution is to merge the already fissed engines back into the default TIL engine and then fiss again.

[0233] For example, a TIL fission diagram is shown below. Figure 9 As shown.

[0234] In step S703, the target engine TI L determines the orderliness and continuity of the messages.

[0235] Optionally, the orderliness and continuity of messages can be verified by comparing the sequence number of the last written message (i.e., the historical message sequence number) with the current message sequence number.

[0236] In step S703 of some embodiments, after receiving a message production request from the client (C-end), an ordered code (i.e., the current message sequence number) is generated, and the message is written to a file using consistent hashing and returned to the C-end. The message is asynchronously pushed to the data core maintenance module for operation using a channel or process communication. After the core module receives the data, it ensures that the data and the previous sequence number are continuous. If not, the current sequence number information is returned to the file layer to retrieve the lost data.

[0237] This is understandable; it's equivalent to determining whether the sequence number of the historical message is consecutive to the sequence number of the current message. If they are not consecutive, then message x is retrieved again.

[0238] Step S704: Check if message x exists based on the I-tree index.

[0239] Specifically, the I-tree uses a BST tree to store the message index information. The I-tree is used for message overwriting, searching, or message splitting. The I-tree achieves multi-level conditional search by nesting multiple layers of BST nodes.

[0240] In some embodiments, a unique key is defined for the message, and data is selected and merged based on the index when writing data.

[0241] It is understandable that steps S704-S707 are equivalent to storing user messages in a preset data structure. Specifically, if the user message already exists, the data is merged; otherwise, new data is added.

[0242] For example, the node structure in an I-tree is as follows:

[0243]

[0244]

[0245] Step S705: If it exists, obtain the node information of L(x) based on I(x), and update the fragment file and offset of the latest message based on L(x).

[0246] Specifically, L is a doubly linked list. The linked list L uses FIFO time dimension for linear storage. The new record is located at the tail node. The operation complexity is O(1). The nodes in L record two sets of tree node information. When operating the message, the two sets of trees are operated and maintained. I(x) represents the first node and L(x) represents the second node.

[0247] In step S705 of some embodiments, updating the latest message fragment file and offset according to L(x) is the data merging process. An exemplary diagram of the data merging process is shown below. Figure 10 As shown.

[0248] Understandably, when L(x) is updated, the timestamp of the T-tree remains unchanged, being the timestamp of the last write.

[0249] It should be noted that because the messages are randomly assigned to a file, it is necessary to calculate the offset to obtain the actual storage location of the message in the file. After retrieving the message from that storage location, the messages are then merged.

[0250] Among them, fragmented files store incremental messages and do not delete or modify data. The principle of merging only requires that the metadata node L(x) points to the new data offset position.

[0251] Optionally, under certain conditions, consumption can be initiated from the head of the linked list to achieve a POP (First-In, First-Out) effect. It can be understood that operating on node L involves maintaining multiple BST trees, with each node containing the actual location of the message. All data (i.e., messages) is stored in the underlying tree nodes, and each underlying node (data node) maps to the linked list L.

[0252] For example, the node structure in linked list L is as follows:

[0253]

[0254] Step S706: If message x does not exist, add a new node T(x) on the T-tree, with the key being "time sequence number". Add a node L(x) to the end of the linked list and add a new node I(x) on the I-tree. The linked list L(x) is associated with the node information of the T(x) and L(x) trees.

[0255] Specifically, the T-tree stores the timestamps of messages for use in delayed search. The time sequence number is timestamp + message sequence number. T(x) represents the node in the second data structure that stores the write time information of the message. The message sequence number of T(x) is the current message sequence number.

[0256] Understandably, this is equivalent to adding data corresponding to user messages to a pre-defined data structure.

[0257] Optionally, the time sequence number can be a 10-bit sequence number or a 26-bit sequence number.

[0258] In step S706 of some embodiments, the complexity of adding node L(x) to the end of the linked list is O(1).

[0259] For example, a schematic diagram of adding a node in a T-tree, I-tree, and linked list is shown below. Figure 11 As shown in the image. Red nodes represent newly added nodes.

[0260] Step S707: The user defines the consumption delay time.

[0261] Step S708: When the timestamp is -1, it is downgraded to chain list consumption. Starting from the root node, POP returns the data x, and delete the T and I tree nodes according to L(x). The time complexity is close to O(1).

[0262] Specifically, the timestamp is the delay time, and the Root node is the root node of the linked list L.

[0263] For example, the default invalid value is -1.

[0264] Understandably, when the timestamp is -1, the POP return data x starting from the root node is equivalent to retrieving the user message within the delay time from a preset data structure as the target message, based on the delay time. The target message is determined based on whether the delay time equals a preset invalid value.

[0265] In this context, deleting T and I tree nodes according to L(x) is equivalent to deleting the data corresponding to the user message.

[0266] Step S709: When the timestamp is not -1, search for the earliest message T(x) in T according to the delay time (subsequent traversal), and then perform deletion operations on the associated nodes L(x) and I(x) in sequence according to T(x).

[0267] Specifically, T(x) represents the ninth node, L(x) represents the tenth node, and I(x) represents the eleventh node.

[0268] For example, a schematic diagram of deleting a node in a T-tree, I-tree, and linked list L is shown below. Figure 12 As shown in the image. The red nodes represent the deleted nodes.

[0269] Understandably, when the timestamp is not -1, searching for the earliest message T(x) from T based on the delay time (in subsequent traversals) is equivalent to retrieving the user message within the delay time from a preset data structure as the target message based on the delay time. The target message is determined based on whether the delay time equals a preset invalid value.

[0270] Specifically, based on the delay time set by the consumer, the earliest generated message T(x) in the T-tree is obtained. If the writing time of the earliest message in the T-tree is earlier than the delay time, the deletion operation is performed on the associated nodes L(x) and I(x) in sequence according to T(x); otherwise, no data is popped.

[0271] Specifically, performing deletion operations on the associated nodes L(x) and I(x) according to T(x) is equivalent to deleting the data corresponding to the user message.

[0272] Further processing of the target message includes deleting the data corresponding to the user message and subsequent steps S710-S712.

[0273] Step S710: Obtain the file fragment file and offset from L(x) and return them, and record the consumption pop log.

[0274] Step S711: Read the message from the file and write it into the consumer's corresponding linked list CL and message encoding and node mapping map.

[0275] In step S712, the consumer ACK returns the message encoding, obtains the linked list node CL(m) based on the encoding, deletes and archives the message, and then re-executes the POP process.

[0276] In this process, after the C-end defines the rules to pull data, the TI L engine writes the data to the consumer log file for backup after performing a POP operation on the data. After the C-end confirms (ACK, Acknowledgment) message, it records a successful consumption log.

[0277] Understandably, if data or service anomalies occur, data can be recovered based on logs, and the TIL data structure can also be reconstructed.

[0278] Please see Figure 13 This application also provides a message processing apparatus that can implement the above-described message processing method. The apparatus includes:

[0279] Message receiving module 1301 is used to receive user messages;

[0280] Storage module 1302 is used to store user messages in a preset data structure;

[0281] The message acquisition module 1303 is used to acquire user messages within the preset delay time from a preset data structure as target messages based on a preset delay time.

[0282] Processing module 1304 is used to process user messages.

[0283] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0284] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned message processing method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0285] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0286] Please see Figure 14 , Figure 14 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0287] The processor 1401 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0288] The memory 1402 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1402 and is called and executed by the processor 1401 using the message processing method of the embodiments of this application.

[0289] The input / output interface 1403 is used to implement information input and output;

[0290] The communication interface 1404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0291] Bus 1405 transmits information between various components of the device (e.g., processor 1401, memory 1402, input / output interface 1403, and communication interface 1404);

[0292] The processor 1401, memory 1402, input / output interface 1403 and communication interface 1404 are connected to each other within the device via bus 1405.

[0293] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described message processing method.

[0294] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0295] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0296] The message processing method, message processing device, electronic device, and storage medium provided in this application embodiment receive user messages; store user messages in a preset data structure; obtain user messages within the delay time from the preset data structure as target messages based on a delay time; and process the target messages. This facilitates message delay processing, prevents traffic surges, improves the comprehensiveness and integrity of message processing, enhances message processing efficiency and stability, and improves system efficiency.

[0297] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0298] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0299] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0300] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0301] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0302] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0303] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0304] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0305] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0306] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0307] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A message processing method characterized by, The method comprises the following steps: receiving a user message; storing the user message in a preset data structure; acquiring a user message within a delay time as a target message from the preset data structure based on the delay time; processing the target message.

2. The method of claim 1, wherein, The step of storing the user message in the preset data structure comprises the following steps: inquiring whether there is a historical message corresponding to the user message in the preset data structure; if there is a historical message corresponding to the user message in the preset data structure, updating data content of the historical message in the preset data structure according to the user message and keeping write time information of the historical message; if there is no historical message corresponding to the user message in the preset data structure, adding the user message in the preset data structure.

3. The method of claim 1, wherein, The preset data structure comprises a first data structure, a second data structure, a third data structure and a fragment file, the first data structure is used for storing index information corresponding to a user message, the second data structure is used for storing write time information of a user message, the third data structure is used for storing fragment file information and offset information corresponding to a user message, and the fragment file is used for storing data content of a user message, and the third data structure is associated with the first data structure and the second data structure.

4. The method of claim 3, wherein, The step of storing the user message in the preset data structure comprises the following steps: storing the user message in the fragment file and recording first fragment file information and first offset information corresponding to the user message; inquiring whether there is a historical message corresponding to the user message in the preset data structure; if there is a historical message corresponding to the user message in the preset data structure, replacing second fragment file information corresponding to the historical message with the first fragment file information and replacing second offset information corresponding to the historical message with the first offset information.

5. The method of claim 4, wherein, The number of the fragment files is multiple; the step of storing the user message in the fragment file comprises the following steps: hashing the user message based on a preset hash function to obtain corresponding hash value; determining a target fragment file from the multiple fragment files according to the hash value and storing the user message in the target fragment file.

6. The method of claim 4, wherein, The step of replacing second fragment file information corresponding to the historical message with the first fragment file information and replacing second offset information corresponding to the historical message with the first offset information if there is a historical message corresponding to the user message in the preset data structure comprises the following steps: inquiring a first node corresponding to the historical message in the first data structure if there is a historical message corresponding to the user message in the preset data structure, and acquiring second index information corresponding to the historical message from the first node. According to the second index information, a second node corresponding to the historical message is determined in the third data structure; wherein the second node is used to store the second fragment file information and the second offset information corresponding to the historical message; The second fragment file information in the second node is replaced by the first fragment file information, and the second offset information in the second node is replaced by the first offset information.

7. The method of claim 3, wherein, The method further comprises: If the historical message corresponding to the user message does not exist in the preset data structure, first index information corresponding to the user message is established, and write time information of the user message is recorded; A third node corresponding to the user message is newly created in the first data structure, a fourth node corresponding to the user message is newly created in the second data structure, and a fifth node corresponding to the user message is newly created in the third data structure; The first index information is stored in the third node, the write time information of the user message is stored in the fourth node, and the first fragment file information and the first offset information are stored in the fifth node.

8. The method of claim 1, wherein, The method further comprises: If the delay time is a preset invalid value, the user messages within the delay time are sequentially determined as target messages from the preset data structure according to the order corresponding to the write time information of each user message; If the delay time is not a preset invalid value, the user messages within the delay time are obtained as target messages from the preset data structure according to the write time information of each user message.

9. The method of claim 1, wherein, The method uses at least one engine, and each engine is configured with a corresponding preset data structure; The method further comprises: A filter condition is obtained; According to the filter condition, a target engine corresponding to the user message is determined in each engine; The user message is stored based on the preset data structure corresponding to the target engine.

10. A message processing device, characterized by The apparatus comprises: A message receiving module configured to receive a user message; A storage module configured to store the user message in a preset data structure; A message obtaining module configured to obtain, based on a preset delay time, user messages within the delay time as target messages from the preset data structure; A processing module configured to process the user message.

11. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, the computer-readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 9.

Images