Pseudo-stream instant messaging application and large language model interaction method and system

Abstract

The invention discloses a pseudo-streaming instant messaging application and large language model interaction method and system, and the method comprises the steps: obtaining a to-be-converted character stream, segmenting the to-be-converted character stream into a plurality of text units, and enabling the to-be-converted character stream to be a lightweight markup language-based streaming reply outputted by a large language model; analyzing and converting each text unit through a preset grammar analysis state machine to obtain a plurality of grammar blocks; judging each grammar block to obtain an output content corresponding to each grammar block; determining a lexical threshold value, and rendering each piece of output content according to the lexical threshold value to obtain a rendered message; and adding the rendering message into a target user task queue, and sending the rendering message in the target user task queue to a target user side corresponding to the instant messaging application in a delayed manner through a rate controller. The method not only can meet the requirement of response speed, but also can ensure that the reply format is highly consistent with the reply format before segmentation, and can be widely applied to the technical field of artificial intelligence.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to an interaction method and system between a pseudo-streaming instant messaging application and a large language model. Background Art

[0002] Currently, the large language model server usually supports streaming responses based on the one-way real-time communication protocol (Server-Sent Events, SSE) from the server to the client, while most instant messaging applications do not support streaming replies. Therefore, for large language model applications with instant messaging applications as the front end on the market (such as chatgpt-on-wechat developed by LinkAI), it is necessary to wait for the large language model to output all tokens before a complete reply can be displayed. Especially when the reply content is long, compared with large language model applications with web browsers as the front end, these methods bring a poor user experience.

[0003] In addition, traditional lightweight markup language (Markdown) parsing and rendering methods are only applicable to complete lightweight markup language (Markdown) source texts. Applying these methods to segmented lightweight markup language (Markdown) source texts will damage or drop nodes on the syntax tree, ultimately resulting in distorted rendering results. Summary of the Invention

[0004] To solve the above technical problems, the purpose of the present invention is to provide an interaction method and system between a pseudo-streaming instant messaging application and a large language model, which can improve the user experience and enhance the rendering accuracy.

[0005] To achieve the above purpose, one aspect of the embodiments of the present application proposes an interaction method between a pseudo-streaming instant messaging application and a large language model, including the following steps:

[0006] Obtain a character stream to be converted, and segment the character stream to be converted into multiple text units, where the character stream to be converted is a streaming reply based on a lightweight markup language output by a large language model;

[0007] Parse and convert each of the text units through a preset syntax analysis state machine to obtain multiple syntax blocks;

[0008] Judge each of the syntax blocks to obtain the output content corresponding to each of the syntax blocks;

[0009] Determine a token threshold, and render each of the output contents according to the token threshold to obtain a rendered message;

[0010] Add the rendering message to the target user task queue, and delay the sending of the rendering message in the target user task queue to the target user terminal corresponding to the instant messaging application through a rate controller.

[0011] In some embodiments, the syntax blocks include reserved characters, fallback syntax blocks, list syntax blocks, code syntax blocks, and table syntax blocks. Parsing and converting each of the text units through a preset syntax analysis state machine to obtain a plurality of syntax blocks specifically includes:

[0012] Input each of the text units into the syntax analysis state machine;

[0013] Determine the regular expressions corresponding to the list syntax block, the code syntax block, and the table syntax block;

[0014] Match each of the text units with the list syntax block, the code syntax block, and the table syntax block according to the regular expressions to obtain a matching result;

[0015] Convert each of the text units into the corresponding syntax block according to the matching result.

[0016] In some embodiments, judging each of the syntax blocks to obtain the output content corresponding to each of the syntax blocks specifically includes:

[0017] Perform a compliance judgment on the text units in each of the syntax blocks to obtain non-compliant words, and filter the non-compliant words;

[0018] Perform an integrity judgment on each of the filtered syntax blocks to obtain an integrity judgment result;

[0019] Extract the output content corresponding to each of the syntax blocks according to the integrity judgment result.

[0020] In some embodiments, the syntax blocks include fallback syntax blocks, list syntax blocks, code syntax blocks, and table syntax blocks, the rendering message includes rich text messages and file messages. Rendering each of the output contents according to the token threshold to obtain a rendering message specifically includes:

[0021] Calculate the cumulative number of words corresponding to each of the output contents, and then judge whether the cumulative number of words exceeds the token threshold;

[0022] When the cumulative word count exceeds the token threshold, render the output content corresponding to the list syntax block and the code syntax block as the rich text message, render the output content corresponding to the table syntax block as the file message, render the output content corresponding to the fallback syntax block as the rich text message or the file message, and then merge adjacent rich text messages;

[0023] Reset each of the rendered syntax blocks.

[0024] In some embodiments, rendering the output content corresponding to the fallback syntax block as the rich text message or the file message specifically includes:

[0025] Render the output content corresponding to the fallback syntax block as a source code block;

[0026] Through a lightweight markup language parsing engine, convert the source code block into semantic fragments, where the semantic fragments include image semantic fragments and multi-type semantic fragments;

[0027] Obtain the resource address attribute of the image semantic fragment, and convert the resource address attribute into the file message;

[0028] Convert the multi-type semantic fragments into the rich text message.

[0029] In some embodiments, adding the rendered message to the target user task queue, and through a rate controller, delaying the sending of the rendered message in the target user task queue to the target user side corresponding to the instant messaging application specifically includes:

[0030] Add the rendered message to the target user task queue corresponding to the target user side;

[0031] Determine the message sending interval, and through the rate controller, delay the sending of the rendered message to the target user side according to the message sending interval.

[0032] In some embodiments, the interaction method further includes:

[0033] Judge the sending status of the rendered message, where the sending status includes the sending-in-progress status and the sending-completed status;

[0034] When the sending status is the sending-in-progress status, send status indication information to the target user side;

[0035] When the sending status is the sending-completed status, withdraw or update the sending status indication information.

[0036] To achieve the above object, on the other hand, an embodiment of the present application proposes an interaction system between a pseudo-streaming instant messaging application and a large language model, including:

[0037] A splitter module, configured to obtain a character stream to be converted, and convert the character stream to be converted into multiple text units, where the character stream to be converted is a streaming reply based on a lightweight markup language output by a large language model;

[0038] A syntax analysis state machine module, configured to parse and convert each of the text units through a preset syntax analysis state machine to obtain multiple syntax blocks;

[0039] A syntax block analysis module, configured to judge each of the syntax blocks to obtain the output content corresponding to each of the syntax blocks;

[0040] A syntax block rendering module, configured to determine a token threshold, and render each of the output contents according to the token threshold to obtain a rendered message;

[0041] A message sending module, configured to add the rendered message to a target user task queue, and delay the sending of the rendered message in the target user task queue to a target user end corresponding to the instant messaging application through a rate controller.

[0042] To achieve the above object, on the other hand, an embodiment of the present application proposes an electronic device, where the electronic device includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory. When the program is executed by the processor, it implements the interaction method between the pseudo-streaming instant messaging application and the large language model as described above.

[0043] To achieve the above object, on the other hand, an embodiment of the present application proposes a storage medium, where the storage medium is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the interaction method between the pseudo-streaming instant messaging application and the large language model as described above.

[0044] The beneficial effects of the present invention are as follows: For the method and system for the interaction between the pseudo-streaming instant messaging application and the large language model of the present invention, first, the character stream to be converted in the reply of the large language model is segmented into multiple text units, and then each text unit is parsed and converted through a preset syntax analysis state machine to obtain multiple syntax blocks. Next, each syntax block is judged to obtain the output content corresponding to each syntax block. Then, each output content is rendered according to the token threshold to obtain a rendered message. Finally, the rendered message is added to the target user task queue, and the rendered message in the target user task queue is delayed and sent to the target user end corresponding to the instant messaging application through a rate controller. The present invention segments the streaming reply output by the large language model, parses and transfers the syntax blocks, extracts the output content of the syntax blocks, can convert the streaming reply of the large language model into a pseudo-streaming message in real time and accurately, and renders the output content into a rendered message adapted to the instant messaging application according to the token threshold, so as to continuously feedback the reply content and reply status to the user, can fully adapt to the scenario of interacting with the long context model, and can meet the requirements of response speed and ensure that the reply format is highly consistent with that before segmentation. Description of the Drawings

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduces the drawings required to be used in the embodiments of the present invention. It should be understood that the drawings introduced below are only for conveniently and clearly expressing some embodiments of the technical solutions in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 Schematic diagram of the source text of the lightweight markup language provided by an embodiment of the present invention;

[0047] Figure 2 Schematic diagram of the rendering result of the traditional rendering method provided by an embodiment of the present invention;

[0048] Figure 3 Flowchart of the steps of the method for the interaction between the pseudo-streaming instant messaging application and the large language model provided by an embodiment of the present invention;

[0049] Figure 4 Flowchart of the steps of step S102 provided by an embodiment of the present invention;

[0050] Figure 5 Flowchart of the steps of step S103 provided by an embodiment of the present invention;

[0051] Figure 6 Flowchart of the steps of step S104 provided by an embodiment of the present invention;

[0052] Figure 7The flowchart of the steps for rendering the fallback syntax block provided by an embodiment of the present invention;

[0053] Figure 8 The flowchart of the steps for step S105 provided by an embodiment of the present invention;

[0054] Figure 9 The flowchart of the steps for message sending status indication provided by an embodiment of the present invention;

[0055] Figure 10 The schematic diagram of the processing flow of the interaction method between the pseudo-streaming instant messaging application and the large language model provided by an embodiment of the present invention;

[0056] Figure 11 The schematic diagram of the structure of the interaction system between the pseudo-streaming instant messaging application and the large language model provided by an embodiment of the present invention;

[0057] Figure 12 The schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0058] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the embodiments of the present application. They are only examples of devices and methods that are consistent with some aspects of the embodiments of the present application detailed in the appended claims.

[0059] It can be understood that the terms "first", "second", etc. used in the present application can be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be called the second information, and similarly, the second information can also be called the first information. Depending on the context, the words "if", "when" as used herein can be interpreted as "when...", "while...", or "in response to determining".

[0060] The terms "at least one", "a plurality", "each", "any one", etc. used in the present application, at least one includes one, two or more, a plurality includes two or more, each refers to each of the corresponding plurality, and any one refers to any one of the plurality.

[0061] Before elaborating on the embodiments of the present application, some nouns and terms involved in the embodiments of the present application are first explained, and the nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

[0062] LLM (Large Language Model): A machine learning model mainly based on the transformer neural network, used for natural language processing tasks mainly focused on language generation.

[0063] IM (Instant Messaging): A computer-mediated synchronous communication method used for real-time message transmission between two or more terminals over computer networks such as the Internet.

[0064] Markdown: A lightweight markup language that allows writing documents using easy-to-read and write text and can be converted into various formats of documents.

[0065] Currently, the large language model server usually supports streaming responses based on the server-to-client unidirectional real-time communication protocol (Server-Sent Events, SSE), while most instant messaging applications do not support streaming replies. Therefore, for large language model applications with instant messaging applications as the front end on the market (such as chatgpt-on-wechat developed by LinkAI), it is necessary to wait for the large language model to output all tokens before a complete reply can be displayed. Especially when the reply content is long, compared with large language model applications with web browsers as the front end, these methods provide a poor user experience.

[0066] To reduce the response time and avoid the message length limit, the streaming reply in the lightweight markup language (Markdown) format of the large language model needs to be sent to the user in multiple segments. However, traditional lightweight markup language (Markdown) parsing and rendering methods are only applicable to complete lightweight markup language (Markdown) source texts. Applying these methods to segmented lightweight markup language (Markdown) source texts will damage or shed nodes on the syntax tree, ultimately resulting in a distorted rendering result. For example Figure 1 For the Markdown text example shown, if the text is directly segmented and processed using a traditional Markdown parsing engine (such as remarkjs), the possible rendering result is as shown in Figure 2 shown, and this result has a damaged format, leading to a poor user reading experience.

[0067] To this end, an embodiment of the present invention proposes an interaction method between a pseudo-streaming instant messaging application and a large language model. First, the character stream to be converted in the reply of the large language model is segmented into multiple text units, and then each text unit is parsed and converted through a preset syntax analysis state machine to obtain multiple syntax blocks. Next, each syntax block is judged to obtain the output content corresponding to each syntax block. Then, each output content is rendered according to the token threshold to obtain a rendered message. Finally, the rendered message is added to the target user task queue, and the rendered message in the target user task queue is delayed and sent to the target user side corresponding to the instant messaging application through a rate controller. The present invention segments, parses and transfers syntax blocks, and extracts the output content of syntax blocks from the streaming reply of the large language model, can convert the streaming reply of the large language model into a pseudo-streaming message in real time and accurately, and render the output content into a rendered message adapted to the instant messaging application according to the token threshold, so as to continuously feedback the reply content and reply status to the user, can fully adapt to the scenario of interacting with a long context model, and can not only meet the requirement of response speed, but also ensure that the reply format is highly consistent with that before segmentation. This interaction method can be applied to scenarios such as instant messaging application interaction for RAG knowledge Q&A, instant messaging application Q&A interaction for Text2SQL data intelligent analysis, instant messaging application Q&A interaction for Copilot class Agent intelligent body applications, etc., but is not limited thereto.

[0068] Referring to Figure 3 , Figure 3 FIG. is a flowchart of the steps of an interaction method between a pseudo-streaming instant messaging application and a large language model provided by an embodiment of the present invention. An embodiment of the present invention proposes an interaction method between a pseudo-streaming instant messaging application and a large language model, and the method includes steps S101 to S105:

[0069] S101. Obtain a character stream to be converted, and segment the character stream to be converted into multiple text units. The character stream to be converted is a streaming reply based on a lightweight markup language output by the large language model;

[0070] In some optional embodiments, first read in the character stream to be converted, convert the character stream of the lightweight markup language (Markdown) replied by the large language model into multiple text units, count the total number of words in each text unit, and cache the converted messages in a queue. Among them, a text unit refers to a character sequence obtained by segmenting through punctuation marks, semantic analysis or a statistical model, including but not limited to sentences, clauses or phrases. In the embodiment of the present invention, the character stream to be converted is pre-segmented with sentences as the basic unit.

[0071] S102. Parse and convert each text unit through a preset syntax analysis state machine to obtain multiple syntax blocks;

[0072] Specifically, the segmented text units are sequentially input into the syntax analysis state machine. The syntax analysis state machine parses the determinable syntax blocks according to the current state and the input text units, identifies the transitions between different syntax blocks, and converts each segment of text units into corresponding syntax blocks.

[0073] Refer to Figure 4 , Figure 4 , which is the step flow chart of step S102 provided by an embodiment of the present invention. Further, as an optional implementation manner, the syntax blocks include reserved characters, fallback syntax blocks, list syntax blocks, code syntax blocks, and table syntax blocks. The step of parsing and converting each text unit through a preset syntax analysis state machine to obtain multiple syntax blocks can be specifically further divided into the following steps S1021 to S1024:

[0074] S1021. Input each text unit into the syntax analysis state machine;

[0075] S1022. Determine the regular expressions corresponding to the list syntax block, code syntax block, and table syntax block;

[0076] S1023. According to the regular expressions, match each text unit with the list syntax block, code syntax block, and table syntax block to obtain a matching result;

[0077] S1024. Convert each text unit into a corresponding syntax block according to the matching result.

[0078] In some optional embodiments, the syntax analysis state machine includes two basic states and at least three extended states. The basic states are respectively reserved characters and fallback syntax blocks, and the extended states include other syntax blocks such as code syntax blocks, table syntax blocks, and list syntax blocks. Each syntax block is set with 3 regular expressions for state transition, namely start regular, hit regular, and end regular. Exemplarily, the start regular of the list syntax block is set as " / ^([0-9]+\.|-) / ", the hit regular is set as " / ^([0-9]+(\.?)?|-?)$ / ", and the end regular is set as " / \n\n / ". When the syntax analysis state machine reads in a text unit, it sequentially checks the regular expressions of each syntax block except the fallback syntax block. If the start regular is matched, the text unit is converted into the corresponding syntax block; if the end regular is matched, it exits the corresponding syntax block and transfers to the reserved character state; if the hit regular is matched, the text unit is added to the reserved characters; otherwise, the reserved characters are converted into the fallback syntax block.

[0079] S103. Judge each syntax block to obtain the output content corresponding to each syntax block;

[0080] Specifically, after converting the text unit into the corresponding grammar block, judge the integrity and compliance of the grammar block, and extract the output content corresponding to each grammar block.

[0081] Refer to Figure 5 , Figure 5 FIG. 186 is a flowchart of step S103 provided by an embodiment of the present invention. Further, as an optional implementation manner, the step of judging each grammar block to obtain the output content corresponding to each grammar block can be specifically divided into the following steps S1031 to S1033:

[0082] S1031. Judge the compliance of the text units in each grammar block to obtain non-compliant words, and filter the non-compliant words;

[0083] S1032. Judge the integrity of each filtered grammar block to obtain an integrity judgment result;

[0084] S1033. Extract the output content corresponding to each grammar block according to the integrity judgment result.

[0085] In some optional embodiments, according to the compliance standard of the instant messaging application, judge the compliance of the text units in each grammar block, identify non-compliant words, and then filter them to prevent these words from appearing in the final output content. As the basic component of the grammar block, the integrity of the text unit affects the semantic expression of the entire grammar block. Judge the integrity of the text unit to ensure the integrity of the output content. Through the analysis and judgment of the above steps, extract the grammar block and the high-quality, compliant and complete output content within the grammar block.

[0086] Exemplarily, the list grammar block uses any sentence as the output content, and records the list item index and the character index within the item after the current segmentation; the table grammar block uses the complete table as the output content; the code grammar block uses the complete code line as the output content, records the code language and the code block identifier, and records the line number reached after the current segmentation.

[0087] S104. Determine the token threshold, and render each output content according to the token threshold to obtain a rendering message;

[0088] Specifically, the token threshold is determined comprehensively according to factors such as the specific application scenario, system performance, and expected output effect. According to the set token threshold, render the output content extracted from all grammar blocks into a format adapted to the instant messaging application, which is convenient for subsequent storage, transmission or further processing in the form of a file.

[0089] Refer to Figure 6 , Figure 6The flowchart of step S104 provided by an embodiment of the present invention. Further as an optional implementation manner, the syntax blocks include fallback syntax blocks, list syntax blocks, code syntax blocks, and table syntax blocks, and the rendering messages include rich text messages and file messages. The step of rendering each output content according to the token threshold to obtain the rendering message can be further divided into the following steps S1041 to S1043:

[0090] S1041. Calculate the cumulative word count corresponding to each output content, and then determine whether the cumulative word count exceeds the token threshold;

[0091] S1042. When the cumulative word count exceeds the token threshold, render the output content corresponding to the list syntax block and the code syntax block as rich text messages, render the output content corresponding to the table syntax block as file messages, and render the output content corresponding to the fallback syntax block as rich text messages or file messages, and then merge adjacent rich text messages;

[0092] S1043. Reset each rendered syntax block.

[0093] In some optional embodiments, the cumulative word count of the output content is calculated by a word counter, and when the cumulative count exceeds the preset token threshold, rendering is triggered, and the output content in all syntax blocks is rendered as rich text messages or file messages. After the rendering is completed, a reset operation is performed on the syntax blocks so that the system can continue to process new input data. Among them, the token threshold is used to control the average length of the segments, and a balance is achieved between the integrity of the rendered segments and the response speed. It can be set according to specific application scenarios, system performance, and expected output effects, etc. For example, it can be set to fixed values such as 150 words, 180 words, and 210 words. Under the setting of the fixed value of 150 words, the average length of the message is about 160 words per message, and the average sending rate is about 2.8 seconds per message.

[0094] Refer to Figure 7 , Figure 7 The flowchart of the step of rendering the fallback syntax block provided by an embodiment of the present invention. Further as an optional implementation manner, the step of rendering the output content corresponding to the fallback syntax block as rich text messages or file messages can be further divided into the following steps S10421 to S10424:

[0095] S10421. Render the output content corresponding to the fallback syntax block as a source code block;

[0096] S10422. Through a lightweight markup language parsing engine, convert the source code block into semantic fragments, and the semantic fragments include image semantic fragments and multi-type semantic fragments;

[0097] S10423. Obtain the resource address attribute of the image semantic fragment and convert the resource address attribute into a file message;

[0098] S10424. Convert multi-type semantic fragments into rich text messages.

[0099] In some alternative embodiments, render the list syntax block and the code syntax block to obtain a text block, and then convert the text block into a rich text message; render the table syntax block to obtain a binary image block, and then convert the binary image block into a file message; render the fallback syntax block to obtain a source code block, and further convert it into a semantic fragment (span) through the lightweight markup language parsing engine mistletoe. After the src attribute (i.e., the resource address attribute) of the image semantic fragment (span) is obtained, it is converted into a file message, and other types of semantic fragments (i.e., multi-type semantic fragments) are converted into rich text messages. Finally, adjacent rich text messages are merged into one rich text message to reduce the total number of messages and the system processing overhead. Moreover, adjacent rich text is usually semantically related, and merging them can maintain this relevance.

[0100] Further, the list syntax block rendering recursively renders the list starting from the index position and adds item numbers to each list item; the table syntax block rendering displays the tabulation of python-tabulate on the image through the image processing tool PIL (Python Imaging Library); the code syntax block rendering renders the code starting from the line number index position, uses the <font color＝"comment">< / font> tag to display line numbers and highlight comments, and uses <font color＝"warning">< / font> tag to highlight keywords and <font color＝"info">< / font> to highlight strings.

[0101] S105. Add the rendered message to the target user task queue, and through the rate controller, delay the sending of the rendered message in the target user task queue to the target user end corresponding to the instant messaging application.

[0102] Specifically, to solve the problem that the instant messaging application does not guarantee the message receiving order of the user end, send the rendered message to the target user task queue of the target user end, merge the messages that can be sent together, and send the rendered message to the target user end in order through the rate controller to reduce message disorder.

[0103] Refer to Figure 8 , Figure 8The figure shows a flowchart of step S105 provided by an embodiment of the present invention. As an optional further implementation, the step of adding a rendering message to the target user task queue and delaying the sending of the rendering message in the target user task queue to the target user side corresponding to the instant messaging application through a rate controller can be further divided into the following steps S1051 and S1052:

[0104] S1051: Add the rendering message to the target user task queue corresponding to the target user side;

[0105] S1052: Determine the message sending interval, and delay the sending of the rendering message to the target user side according to the message sending interval through the rate controller.

[0106] In some optional embodiments, a task queue and a task consumption thread are bound to each user, so that the task consumption thread loops and waits for tasks. The rendering message to be sent is placed in the task queue of the corresponding user to trigger task consumption. The task consumption thread sends a message to the instant messaging application through an HTTP POST request, and records the start time and end time of the request through the rate controller. When the task consumption thread sends a message next time, it synchronously calculates through the rate controller whether the difference between the current time and the start time and end time of the previous request exceeds the preset message sending interval. If it exceeds, it sends directly; if it is insufficient, it waits and then sends, so as to ensure that the target user side of the instant messaging application receives messages in sequence under normal network conditions. Among them, the message sending interval can be set according to requirements such as the rate limit of the instant messaging application, network environment indicators, user side device performance, and message content characteristics. Exemplarily, the start time interval between each message sending task and the previous task is set to be more than 0.5 seconds, and the end time interval between each message sending task and the previous task is set to be more than 0.2 seconds.

[0107] Refer to Figure 9 , Figure 9 The figure shows a flowchart of message sending status indication provided by an embodiment of the present invention. As an optional further implementation, the interaction method may further include the following steps S106 to S108:

[0108] S106: Judge the sending status of the rendering message. The sending status includes the sending status and the sent status;

[0109] S107: When the sending status is the sending status, send status indication information to the target user side;

[0110] S108: When the sending status is the sent status, withdraw or update the sending status indication information.

[0111] Specifically, during the process of sending a rendering message, a progress indicator is used to indicate two states: waiting for segmentation and all segments having been sent, so as to prompt the user whether the reply is completed currently, and solve the problem that the user cannot confirm the reply status due to the unfixed sending time of the segments.

[0112] In some alternative embodiments, the progress indicator can use the image sending and message recall method. When starting to reply, a dynamic picture is sent to the target user terminal and the message identification number is recorded. When ending the reply, the message is recalled to implement the prompt of the pseudo-streaming reply status to the user; it can also use the card sending and card updating method. When starting to reply, a card containing progress information is sent to the target user terminal, and the card content is dynamically updated when the message sending status changes, so as to intuitively display the current reply progress status to the user.

[0113] In summary, the processing flow of the pseudo-streaming instant messaging application and the interaction method with the large language model according to the embodiments of the present invention is as Figure 10 shown:

[0114] The first step is character stream input. The lightweight markup language (Markdown) character stream replied by the large language model is converted into text units (such as sentences) and written into the syntax analysis state machine.

[0115] The second step is syntax block parsing. First, the sentences are parsed and combined into other syntax blocks such as reserved characters, fallback syntax blocks, list syntax blocks, and code syntax blocks. Then, the integrity of the sentences is judged to ensure the integrity of the output content. Finally, the compliance of the sentences is judged to filter out all non-compliant words in the syntax blocks. Through the parsing and judgment of the above steps, the syntax blocks and the output content within the syntax blocks are extracted.

[0116] The third step is syntax block rendering. When the cumulative word count of the output content exceeds the token threshold, the output content within all syntax blocks is rendered as a rich text message or a file message, and then the syntax blocks are reset.

[0117] The fourth step is message queue output. The rendering message is sent to the task queue, and the messages that can be sent together are merged.

[0118] Applying the interaction method according to the embodiments of the present invention to the Figure 1 Markdown text example in, the rendering result obtained is as Figure 11 shown, and it can be seen from Figure 11 that the interaction method proposed by the embodiments of the present invention can not damage the reply format of the large language model, and its output reply format is highly consistent with that before segmentation.

[0119] The above describes the interaction method between the pseudo-streaming instant messaging application and the large language model of the embodiments of the present invention. It can be recognized that compared with the existing lightweight markup language parsing and rendering methods, the embodiments of the present invention have the following advantages:

[0120] 1. Pseudo-streaming output is achieved through intelligent segmentation, and the human-computer interaction effect is good. During the generation of responses by the large language model in the instant messaging application in the embodiments of the present invention, through the processes of clause splitting, syntax block recognition and transfer, and extractable output content of syntax blocks for the character stream to be converted, the streaming response of the large language model can be efficiently and accurately converted into pseudo-streaming messages, so as to continuously feedback the response content and response status to the user, significantly improving the response speed, especially suitable for scenarios of large models with long contexts and chained calls to large models;

[0121] 2. Rich text output in various styles, high readability of messages, and wide application scenarios. Through the lightweight markup language rendering implemented in the embodiments of the present invention, rich text messages adapted to the user side of the instant messaging application can be generated;

[0122] 3. Using message queues and task queues, large-concurrency user interactions can be stably supported. In the embodiments of the present invention, messages are sent through a delay task queue, and the probability of out-of-order messages can be greatly reduced without affecting the response speed.

[0123] Referring to Figure 11 , the embodiments of the present invention also provide an interaction system between a pseudo-streaming instant messaging application and a large language model, including:

[0124] A splitter module, configured to obtain a character stream to be converted and convert the character stream to be converted into multiple text units, where the character stream to be converted is a streaming response based on a lightweight markup language output by the large language model;

[0125] A syntax analysis state machine module, configured to parse and convert each text unit through a preset syntax analysis state machine to obtain multiple syntax blocks;

[0126] A syntax block analysis module, configured to judge each syntax block to obtain the output content corresponding to each syntax block;

[0127] A syntax block rendering module, configured to determine a token threshold and render each output content according to the token threshold to obtain a rendered message;

[0128] A message sending module, configured to add the rendered message to the target user task queue and delay the sending of the rendered message in the target user task queue to the target user side corresponding to the instant messaging application through a rate controller.

[0129] The content in the embodiments of the method for the interaction between the above-mentioned pseudo-streaming instant messaging application and the large language model is applicable to the embodiments of the system for the interaction between the pseudo-streaming instant messaging application and the large language model. The functions specifically implemented in the embodiments of the system for the interaction between the pseudo-streaming instant messaging application and the large language model are the same as those in the embodiments of the method for the interaction between the pseudo-streaming instant messaging application and the large language model, and the beneficial effects achieved are also the same as those in the embodiments of the method for the interaction between the pseudo-streaming instant messaging application and the large language model.

[0130] An embodiment of the present invention further provides an electronic device, which includes: a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing the connection and communication between the processor and the memory. When the program is executed by the processor, it realizes the method for the interaction between the pseudo-streaming instant messaging application and the large language model. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0131] As Figure 12 shown is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. Referring to Figure 12 , an embodiment of the present invention provides an electronic device, including:

[0132] A processor 1001, which can be implemented in ways such as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention;

[0133] A memory 1002, which can be implemented in forms such as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1002 and are called by the processor 1001 to execute the method for the interaction between the pseudo-streaming instant messaging application and the large language model of the embodiments of the present invention;

[0134] An input / output interface 1003, which is used to realize information input and output;

[0135] A communication interface 1004, which is used to implement communication and interaction between this device and other devices. It can achieve communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0136] A bus 1005, which transmits information between various components of the device (such as a processor 1001, a memory 1002, an input / output interface 1003, and a communication interface 1004);

[0137] Among them, the processor 1001, the memory 1002, the input / output interface 1003, and the communication interface 1004 achieve communication connections with each other inside the device through the bus 1005.

[0138] An embodiment of the present invention also provides a storage medium. The storage medium is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned interaction method between the pseudo-streaming instant communication application and the large language model.

[0139] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include a high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0140] An embodiment of the present invention also discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to enable the computer device to execute Figure 1 the method shown.

[0141] In some alternative embodiments, the functions / operations recited in the block diagrams may not occur in the order presented in the operational illustrations. For example, depending on the functions / operations involved, two blocks shown in succession may actually be executed substantially simultaneously or the blocks may sometimes be executed in the reverse order. Additionally, the embodiments presented and described in the flowcharts of the present invention are provided by way of example for the purpose of providing a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and in which sub-operations described as part of a larger operation are performed independently.

[0142] Furthermore, although the present invention has been described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the above-described functions and / or features may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It should also be understood that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the modules will be understood within the ordinary skill of an engineer. Accordingly, those of ordinary skill in the art can implement the present invention as set forth in the claims without undue experimentation. It should also be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0143] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0144] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definitional sequence list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in conjunction with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or more wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the above program can be printed, because the above program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0146] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0147] In the above description of this specification, the description with reference to terms such as "one embodiment / example", "another embodiment / example", or "certain embodiments / examples" etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0148] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

[0149] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for the interaction between a pseudo-streaming instant messaging application and a large language model, characterized in that, It includes the following steps: Obtain the character stream to be converted, and split the character stream to be converted into multiple text units. The character stream to be converted is a streaming reply based on a lightweight markup language output by a large language model; Parse and convert each of the text units through a preset syntax analysis state machine to obtain multiple syntax blocks; Judge each of the syntax blocks to obtain the output content corresponding to each of the syntax blocks; Determine a token threshold, and render each of the output contents according to the token threshold to obtain a rendering message; Add the rendering message to the target user task queue, and through a rate controller, delay the sending of the rendering message in the target user task queue to the target user side corresponding to the instant messaging application.

2. The interactive method between a pseudo-streaming instant messaging application and a large language model according to claim 1, wherein The syntax blocks include reserved characters, fallback syntax blocks, list syntax blocks, code syntax blocks, and table syntax blocks. The step of parsing and converting each of the text units through a preset syntax analysis state machine to obtain multiple syntax blocks specifically includes: Input each of the text units into the syntax analysis state machine; Determine the regular expressions corresponding to the list syntax block, the code syntax block, and the table syntax block; According to the regular expressions, match each of the text units with the list syntax block, the code syntax block, and the table syntax block to obtain a matching result; Convert each of the text units into the corresponding syntax block according to the matching result.

3. A method for interacting between a pseudo-streaming instant messaging application and a large language model according to claim 1, characterized in that, The step of judging each of the syntax blocks to obtain the output content corresponding to each of the syntax blocks specifically includes: Perform a compliance judgment on the text units in each of the syntax blocks to obtain non-compliant words, and filter the non-compliant words; Perform an integrity judgment on each of the filtered syntax blocks to obtain an integrity judgment result; Extract the output content corresponding to each of the syntax blocks according to the integrity judgment result.

4. A method for interacting between a pseudo-streaming instant messaging application and a large language model according to claim 1, characterized in that, The syntax blocks include fallback syntax blocks, list syntax blocks, code syntax blocks, and table syntax blocks. The rendering message includes a rich text message and a file message. The step of rendering each of the output contents according to the token threshold to obtain a rendering message specifically includes: Calculate the cumulative number of tokens corresponding to each of the output contents, and then judge whether the cumulative number of tokens exceeds the token threshold; When the cumulative number of tokens exceeds the token threshold, render the output content corresponding to the list syntax block and the code syntax block as the rich text message, render the output content corresponding to the table syntax block as the file message, render the output content corresponding to the fallback syntax block as the rich text message or the file message, and then merge adjacent rich text messages; Reset each of the rendered syntax blocks.

5. A method for interacting between a pseudo-streaming instant messaging application and a large language model according to claim 4, characterized in that The step of rendering the output content corresponding to the fallback syntax block as the rich text message or the file message specifically includes: Render the output content corresponding to the fallback syntax block as a source code block; Through a lightweight markup language parsing engine, convert the source code block into semantic fragments, and the semantic fragments include image semantic fragments and multi-type semantic fragments; Obtain the resource address attribute of the image semantic segment, and convert the resource address attribute into the file message; Convert the multi-type semantic segment into the rich text message.

6. A method for interacting between a pseudo-streaming instant messaging application and a large language model according to claim 1, characterized in that The step of adding the rendering message to the target user task queue and delaying the sending of the rendering message in the target user task queue to the target user side corresponding to the instant messaging application through a rate controller specifically includes: Add the rendering message to the target user task queue corresponding to the target user side; Determine the message sending interval, and delay the sending of the rendering message to the target user side according to the message sending interval through the rate controller.

7. A method for interacting between a pseudo-streaming instant messaging application and a large language model according to any one of claims 1 to 6, characterized in that, The interaction method further includes: Judge the sending status of the rendering message, where the sending status includes the sending-in-progress status and the sending-completed status; When the sending status is the sending-in-progress status, send status indication information to the target user side; When the sending status is the sending-completed status, withdraw or update the sending status indication information.

8. An interactive system between a pseudo-streaming instant messaging application and a large language model, characterized in that, It includes: A splitter module, configured to obtain a character stream to be converted, and convert the character stream to be converted into multiple text units, where the character stream to be converted is a streaming reply based on a lightweight markup language output by a large language model; A syntax analysis state machine module, configured to parse and convert each of the text units through a preset syntax analysis state machine to obtain multiple syntax blocks; A syntax block analysis module, configured to judge each of the syntax blocks to obtain the output content corresponding to each of the syntax blocks; A syntax block rendering module, configured to determine a token threshold, and render each of the output contents according to the token threshold to obtain a rendering message; A message sending module, configured to add the rendering message to a target user task queue, and delay the sending of the rendering message in the target user task queue to the target user side corresponding to the instant messaging application through a rate controller.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing the connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the interaction method between the pseudo-streaming instant messaging application and the large language model according to any one of claims 1 to 7 are implemented.

10. A storage medium, the storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the interaction method between the pseudo-streaming instant messaging application and the large language model according to any one of claims 1 to 7.

Images