Thought chain text optimization method, device, storage medium and program product

Abstract

The specification provides a thinking chain text optimization method, device, storage medium and program product. Streaming thinking chain text generated by a generative language model for a user question is obtained, the streaming thinking chain text reflecting a thinking process of the generative language model for the user question; the streaming thinking chain text is split to obtain a plurality of original text blocks; for each original text block in the plurality of original text blocks, at least based on a preset text optimization strategy and content of the original text block, a prompt word corresponding to the original text block is constructed, and the original text block is optimized by a text optimization model guided by the prompt word to obtain an optimized text block corresponding to the original text block; and the plurality of original text blocks are output in the order of the generative language model outputting the plurality of original text blocks, and the optimized text blocks corresponding to the plurality of original text blocks are output in a streaming manner.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a method, device, storage medium and program product for optimizing thought chain text. Background Technology

[0002] Generative language models, such as Large Language Models (LLMs), are widely used in various scenarios, including intelligent question answering and logical reasoning. Chain of Thought (CoT) has become a core bridge connecting user questions and the model's final formal answer. CoT is a concrete representation of the model's step-by-step reasoning process in response to a user question. Its core value lies in revealing the generation logic of the formal answer to the user, helping them understand the rationality and credibility of the formal answer, thereby improving the transparency of human-computer interaction and user acceptance of the results.

[0003] However, current generative language models generally suffer from problems such as poor readability, low information density, potential leakage of internal tool information, and exposure of underlying prompting specifications. Furthermore, the presentation format and style of the thought chains are fixed, preventing users from adjusting them based on their needs. Considering that the quality of the thought chains directly impacts users' understanding and trust in the formal answers, it is necessary to optimize the content of the thought chains output by generative language models before presenting them to users. Summary of the Invention

[0004] In view of this, one or more embodiments of this specification provide a method, device, storage medium, and program product for optimizing thought chain text. It can segment the streaming thought chain text output by a generative language model, optimize the segmented original text blocks using a text optimization model, and then stream the optimized text blocks. This allows for optimization of thought chain text based on user needs and reduces the overall time spent displaying the first word on the front end.

[0005] According to a first aspect of one or more embodiments of this specification, a method for optimizing thought chain text is proposed, the method comprising: Obtain the streaming thought chain text generated by the generative language model in response to the user's question, wherein the streaming thought chain text reflects the thinking process of the generative language model in response to the user's question; The streaming thought chain text is segmented to obtain multiple original text blocks; For each of the plurality of original text blocks, at least based on a preset text optimization strategy and the content of the original text block, a prompt word corresponding to the original text block is constructed, and the prompt word guides the text optimization model to optimize the original text block to obtain the optimized text block corresponding to the original text block. The optimized text blocks corresponding to each of the multiple original text blocks are streamed according to the order in which the generative language model outputs the multiple original text blocks.

[0006] According to a second aspect of the embodiments of this specification, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; Wherein, when the processor executes the executable instructions, it is used to implement the method described in the first aspect.

[0007] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect.

[0008] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] As can be seen from the above embodiments, after obtaining the streaming thought chain text output by the generative language model, this specification can segment the streaming thought chain text to obtain multiple original text blocks. For each original text block, a prompt word corresponding to the original text block can be constructed based on a preset text optimization strategy and the content of the original text block. The prompt word guides the text optimization model to optimize the original text block, resulting in an optimized text block. Then, following the order in which the generative language model outputs multiple original text blocks, the optimized text blocks corresponding to each of the multiple original text blocks are streamed out to display the thought chain text optimized according to the text optimization strategy to the user. By segmenting the streamed thought chain text into text blocks and then optimizing the text blocks, segmentation and block-by-block optimization can be performed in real time without waiting for the complete thought chain to be generated, reducing the user's waiting time. For each original text block, prompt words are constructed in conjunction with preset text optimization strategies. This guides the text optimization model to rewrite and optimize the original thought chain content output by the generative language model according to user needs, thereby improving the quality of the thought chain text finally presented to the user, enhancing user satisfaction and trust in the model's output results, and optimizing the human-computer interaction experience. Attached Figure Description

[0010] Figure 1This is a schematic diagram illustrating an application scenario provided by an exemplary embodiment.

[0011] Figure 2 This is a schematic diagram of a thought chain text optimization method provided in an exemplary embodiment.

[0012] Figure 3 This is a flowchart of a thought chain text optimization method provided in an exemplary embodiment.

[0013] Figure 4 This is a schematic diagram illustrating asynchronous execution of text segmentation and text optimization as provided in an exemplary embodiment.

[0014] Figure 5 This is an exemplary embodiment illustrating the additional TTFT and intermediate waiting time in scenarios where text optimization is not performed on the thought chain, text segmentation and text optimization are performed synchronously, text segmentation and text optimization are performed asynchronously, and text optimization is performed in parallel.

[0015] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0017] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0018] Generative language models (such as large language models) are widely used in various scenarios such as intelligent question answering and logical reasoning. When given a user-input question, the generative language model outputs two parts during reasoning: a "thought chain" and a "formal answer." The "thought chain" text is placed between fixed thought chain tags, reflecting the model's thought process in organizing the formal answer. Currently, the "thought chains" output by generative language models generally suffer from poor readability, low information density, potential leakage of internal tool information, and exposure of underlying prompting specifications. Furthermore, the style and format of the thought chains output by generative language models are fixed and cannot be adjusted based on user needs, lacking flexibility. Therefore, it is necessary to optimize the thought chain text output by generative language models to improve user satisfaction.

[0019] Since thought chain text is a byproduct of the output of a generative language model, an easy approach is to directly train and optimize the generative language model to improve the quality of the output thought chain text. However, this approach may lead to a decrease in the quality of the formal answers output by the generative language model. Furthermore, since generative language models used in fields such as intelligent answering are often language models with a particularly large number of parameters, adopting this approach will also greatly increase the training cost.

[0020] Of course, to avoid the above problems, the thought chain text can be optimized through post-processing without modifying the internal thought chain text of the generative language model. For example, an additional processing chain can be set up to optimize the thought chain text output by the generative language model. However, optimizing the thought chain text output by the generative language model will inevitably prolong the latency from when the generative language model is invoked to when the first token is displayed to the user (i.e., TTFT, Time-To-First-Token), affecting the streaming efficiency.

[0021] Considering that when displaying the thought chain text output by the generative language model to the user, the thought chain text is collected in a streaming manner and pushed to the client in real time, collecting all the original thought chain content and then optimizing it uniformly would yield the best optimization effect because it can be optimized in combination with the entire context of the thought chain text. However, this would result in a longer blocking time (i.e., a longer delay from the generative language model being invoked to the display of the first word unit to the user), leading to a poor user experience. Based on this, the embodiments of this specification provide a thought chain text optimization method. A bypass link can be added after the generative language model to segment the thought chain text streamed by the generative language model into multiple original text blocks. Then, each original text block is input into a text optimization model, which optimizes each original text block according to a preset text optimization strategy to obtain the corresponding optimized text blocks. Then, the optimized text blocks corresponding to each of the multiple original text blocks can be streamed and displayed to the user in the order in which the generative language model outputs the multiple original text blocks. The entire processing flow can achieve a parallel pipeline effect of "local segmentation-optimization-streaming". By segmenting the streamed thought chain text into text blocks and then optimizing the text blocks, it can segment and optimize block by block in real time without waiting for the complete thought chain to be generated, reducing the user's waiting time. At the same time, the text optimization model can be used to rewrite and optimize the original thought chain content output by the generative language model based on preset text optimization strategies. This allows the form and style of the optimized thought chain text to be adjusted according to user needs, making it more flexible and improving the quality of the final thought chain text presented to the user, as well as user satisfaction.

[0022] The thought chain text optimization method described in this specification can be executed by various electronic devices, such as mobile phones, tablets, laptops, physical servers, cloud servers, server clusters, etc. This specification does not impose any limitations on the embodiments.

[0023] like Figure 1 The diagram shown is an application scenario illustration of an exemplary embodiment of this description. Figure 1 This is a schematic diagram of the architecture of an intelligent question-answering service system provided in an exemplary embodiment. Figure 1 As shown, the intelligent question-and-answer service system may include a server 11, a network 12, and several electronic devices, such as a PC (Personal Computer) 13, a mobile phone 14, etc.

[0024] Server 11 can be a physical server containing an independent host, or it can be a virtual server hosted in a host cluster. During operation, server 11 can run server-side programs for a certain application to implement the relevant functions of that application. For example, when server 11 runs a program for an intelligent question-and-answer service, it can be implemented as a corresponding intelligent question-and-answer service platform.

[0025] PC13 and mobile phone14 are just some of the types of electronic devices that users can use. In reality, users can obviously also use electronic devices such as tablets, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smartwatches, etc.), etc., and one or more embodiments in this specification do not limit this. During operation, the electronic device can run a client-side program of an application to achieve the relevant functions of that application. For example, when the electronic device runs a smart question-and-answer service program, it can act as a client for that smart question-and-answer service. The aforementioned smart question-and-answer service client application can be launched and run on the electronic device. This client-side program can be a native application installed on the electronic device, or it can be a mini-program, quick app, or other similar form. Of course, when using web technologies such as HTML5 or similar, the relevant functions can be achieved through a page displayed by a browser. This browser can be a standalone browser application or a browser module embedded in some applications.

[0026] As for the network 12 that enables interaction between electronic devices such as PC13 and mobile phone 14 and server 11, communication can be achieved using either wired or wireless networks, depending on the communication methods supported by the respective electronic devices. This specification does not impose any restrictions on this. For example, PC13 can support both wired and wireless communication, so it can use either wired or wireless networks as needed. Mobile phone 14 typically only supports wireless communication, so it can use a wireless network for communication.

[0027] Server 11 can deploy a generative language model and a text optimization model. Users can submit questions via clients on PC 13 and mobile phones 14. After receiving a user question, the server on server 11 can invoke the generative language model to reason about the question, obtaining a streaming thought chain text and a formal answer. After obtaining the streaming thought chain text, the server can segment it into multiple raw text blocks, then input each raw text block into the text optimization model to obtain corresponding optimized text blocks. These optimized text blocks are then streamed sequentially to the client, allowing the client to stream and display the optimized text blocks to the user. Once all raw text blocks have been optimized, the server can collect the formal answer output by the generative language model and stream it to the client for display.

[0028] The generative language model and text optimization model in the embodiments of this specification can be any large open-source language model, or a specialized model that has been fine-tuned. For example, because generative language models need to be used for reasoning on various user questions, the tasks they handle are more diverse and complex. Therefore, a language model with a more complex structure and a larger number of parameters can be selected to improve the accuracy of the output results. On the other hand, the main task of the text optimization model is to optimize thought chain text, which is a simpler and more straightforward task. Therefore, a lightweight language model with a simpler structure and fewer parameters can be selected.

[0029] The following combination Figure 2 and Figure 3 This specification introduces the thought chain text optimization method according to embodiments. Figure 2 A schematic diagram illustrating the overall processing flow of the thought chain text optimization method according to an embodiment of this specification is shown. Figure 3 A flowchart of the thought chain text optimization method is shown. (For example...) Figure 3 As shown, the method may include the following steps: S302. Obtain the streaming thought chain text generated by the generative language model for the user's question, wherein the streaming thought chain text reflects the thinking process of the generative language model for the user's question. In step S302, after receiving the user's input question, such as... Figure 2 As shown, a generative language model can generate both streaming thought chain text and formal answers to user questions. The streaming thought chain text reflects the generative language model's thought process in response to the user question, fully presenting the model's reasoning path (such as question decomposition, conditional analysis, logical deduction, and conclusion prediction), serving as a crucial bridge connecting the "user question" and the "formal answer." The thought chain text output by the generative language model is streaming data, meaning it outputs the thought chain text token by token. To reduce the time-to-the-first-to-the-user (TTFT) latency, streaming thought chain text can be obtained from the generative language model in real-time and incrementally. Here, "streaming" means that during the generation of the thought chain, each incremental text segment (such as a paragraph or a complete logical unit) generated by the generative language model is transmitted to the text processing link in real-time, without waiting for the generative language model to output all thought chain text before optimization. By adopting streaming processing, the entire optimization process can be synchronized with the generation rhythm of the generative language model, avoiding delays caused by data accumulation.

[0030] S304. The streaming thought chain text is segmented to obtain multiple original text blocks; In step S304, the streaming thought chain text can be segmented to obtain multiple original text blocks arranged in the output order of the generative language model. For example, each time an incremental text segment is received from the generative language model, it can be determined whether the segmentation conditions are met based on preset text segmentation rules. If they are met, the text is segmented into independent original text blocks, and all original text blocks strictly follow the sequential order of the thought chain output by the generative language model (i.e., the thought chain segment generated earlier corresponds to the original text block segmented earlier). The text segmentation rules can be flexibly set based on the needs of the actual application scenario. For example, the streaming thought chain text can be segmented according to a fixed number of characters. For example, with 100 characters, the text is segmented into an original text block for every 100 characters output by the generative language model. Alternatively, the streaming thought chain text can be segmented according to fixed punctuation marks (such as commas and periods). For example, with a period, punctuation mark detection can be performed on the content output by the generative language model. After detecting a period, the entire preceding sentence can be treated as an original text block. Alternatively, the streaming thought chain text can be segmented based on paragraph markers, for example, each paragraph can be divided into a block of original text.

[0031] Of course, when setting text segmentation rules, it's important to balance semantic integrity, processing efficiency, and latency. This means ensuring the original text blocks resulting from the segmentation are neither too few nor too many characters, while maintaining semantic coherence within the context. For example, too few characters might require multiple calls to the text optimization model, leading to poor optimization results and wasted model resources due to the limited text content and lack of context. Too many characters could result in poor latency when displaying the first token to the user, impacting user experience. Furthermore, incorrect segmentation can lead to incomplete semantic information in the original text blocks, causing the text optimization model to incorrectly complete completion and answer tasks, adding redundant content.

[0032] S306. For each of the plurality of original text blocks, at least based on a preset text optimization strategy and the content of the original text block, a prompt word corresponding to the original text block is constructed, and the prompt word guides the text optimization model to optimize the original text block to obtain the optimized text block corresponding to the original text block. In step S306, for each original text block obtained from the segmentation, such as Figure 2As shown, based on a preset text optimization strategy and the specific content of the current original text block, prompt words can be constructed. These prompt words are then input into the text optimization model to guide it in optimizing the original text block and generating an optimized text block corresponding to the original text block. The text optimization strategy can include various rules for optimizing text, such as compression rules for compressing the original text block, rules on the style and format to which the original text block should be rewritten, and structured rules, etc. The optimization of each original text block can be initiated independently, and the text optimization rules can also be adjusted differently based on the specific content of each text block.

[0033] S308. Stream the optimized text blocks corresponding to each of the multiple original text blocks according to the order in which the generative language model outputs the multiple original text blocks.

[0034] In step S308, after each original text block is optimized to obtain an optimized text block, there is no need to wait for the optimization results of other original text blocks. The optimized text blocks can be pushed to the user's front end in real time (such as interface display, voice broadcast, etc.) according to the "order in which the generative language model outputs the original text blocks". The entire output process is executed in parallel with the generation of subsequent optimized text blocks (for example, when the first optimized text block is output, the second block may still be being optimized), forming a streaming delivery rhythm of "optimizing one block and outputting one block". At the same time, the output process must ensure that the optimized text blocks are not lost or out of order, and strictly match the logical progression relationship of the original inference of the generative language model.

[0035] "Real-time streaming output" continues the core advantages of streaming interaction, allowing users to obtain results without waiting for all thought processes to be optimized, significantly reducing "end-to-end first display latency" and avoiding a decline in user experience caused by long waiting times.

[0036] In one or more exemplary embodiments, the number of characters in each original text block obtained from the segmentation does not exceed a preset character limit, which is determined based on the expected first-display latency (TTFT). The TTFT is the time delay from calling the generative language model to answer the user's question to displaying the first word element output by the generative language model to the user. Each original text block obtained from the segmentation of the streaming thought chain text is subject to the constraint that "the number of characters does not exceed the preset character limit," and this limit can be set based on the expected target of the "first-display latency." The "first-display latency" refers to the entire time elapsed from the start of processing the user's question by calling the generative language model to the display of the first word element (which can be the first character, word, or phrase in the thought chain) to the user through the front-end interface (such as an interactive window or app page), and is a key indicator for measuring the real-time performance of streaming interaction. Since the number of characters in the original text block directly affects the time consumption of subsequent processing links (the more characters, the higher the computational cost for the text optimization model to process the block, thus extending the total time from "receiving the original text block" to "outputting the optimized text block"), when setting the upper limit of the number of characters, we can first clarify the user's acceptable first display latency threshold (such as 1 second or 1.5 seconds), and then combine the output rate of the generative language model, the processing speed of the text optimization model, the front-end rendering time and other link parameters to deduce that "the total time from receiving the first streaming thought chain segment, splitting and generating the original text block, completing optimization and front-end display" does not exceed the maximum number of characters with the expected latency. This is used as the upper limit of the number of characters in the original text block, and finally achieves the precise constraint that "the number of characters in each original text block is controlled within the expected first display latency".

[0037] By "deriving the character limit based on the expected initial display latency," the abstract "real-time requirement" is transformed into a quantifiable and executable text block character limit. This avoids the accumulation of processing time due to excessively long text blocks, ensuring that the initial display latency remains stable within an acceptable range for users. This fundamentally solves the problem of "slow initial response and impact on user experience" in streaming interaction. Furthermore, when different scenarios have different requirements for initial display latency (e.g., shorter latency for real-time customer service scenarios, and more relaxed latency for deep inference scenarios), the expected latency target can be adjusted to flexibly adapt to the character limit without refactoring the segmentation logic, improving the versatility and practicality of the solution.

[0038] When segmenting a streaming thought chain, a reasonable streaming text chunking strategy is crucial. Overly long chunks lead to excessively long intervals between original text blocks, resulting in output blocking and a degraded user experience. Overly short chunks increase the number of calls to the text optimization model and communication overhead. Furthermore, when the original text block to be optimized is too short, the optimization space is limited, easily leading to a situation where the difference between local and global optimization effects is too large. In addition, segmenting according to a fixed number of characters, without considering the semantic coherence and completeness of the text blocks, will also prevent the text optimization model from acquiring complete semantic information, resulting in poor optimization performance. Based on the applicant's analysis and research, a text chunking strategy based on "paragraph markers + character count limits" is proposed. For example, in one or more exemplary embodiments, when segmenting the thought chain text to obtain streaming original text blocks, the streaming thought chain text output by the generative language model can be continuously received. Paragraph marker detection is performed on the received streaming thought chain text, and the number of characters in the currently received but un-segmented streaming thought chain text is counted. A currently received, unblocked stream of thought chain text can be treated as a single raw text block if any of the following conditions are met.

[0039] (1) If a paragraph mark is detected in the received streaming thought chain text, and the number of words in the currently received and unblocked streaming thought chain text is not less than the preset minimum number of words, the currently received and unblocked streaming thought chain text can be used as a raw text block.

[0040] (2) If the number of characters in the currently received and undivided streaming thought chain text reaches the preset character limit, the currently received and undivided streaming thought chain text can be used as a raw text block.

[0041] For example, during the streaming output of the thought chain text in the generative language model, incremental text data can be continuously received. During this process, two operations are performed simultaneously: First, the received text is subjected to real-time detection of paragraph markers (such as line breaks, paragraph separator labels, etc.) to capture the natural semantic segmentation boundaries in the thought chain. Second, the number of words in the currently received but not yet segmented text segments (i.e., "unsegmented text") is counted to keep track of its length changes in real time. When the unsegmented text meets any of the following triggering conditions, it is immediately identified as an independent original text block. Condition (1): If a paragraph marker is detected, and the number of words in the unsegmented text has reached or exceeded the preset lower limit (this lower limit is used to avoid segmenting text blocks that are too short and semantically incomplete), then the segmentation is completed with the paragraph marker as the boundary to ensure that the original text block conforms to the natural semantic segmentation and has a basic information carrying capacity. Condition (2) If the number of words in the unblocked text has reached the preset word limit (this limit is used to avoid the text block being too long and causing subsequent optimization and output delay), the unblocked text will be immediately split even if the paragraph mark has not been detected, so as to avoid the decrease in real-time performance caused by waiting for the paragraph mark.

[0042] By combining paragraph marker detection and character limit conditions, text is prioritized for segmentation based on natural semantic boundaries. This maximizes the preservation of the logical integrity of the thought chain and avoids semantic breaks caused by fixed character limit segmentation (such as splitting complete reasoning paragraphs). It ensures that the original text blocks have a coherent reasoning context, laying the foundation for maintaining logical consistency during subsequent optimization. The "character limit" fallback condition effectively controls the length of text blocks, preventing the accumulation of unsegmented text due to excessively long paragraphs in the thought chain (such as long sentences without punctuation or continuous reasoning fragments). This avoids excessively long subsequent optimization processing time, affecting the real-time performance of streaming output, and ensuring controllable "first display latency." Simultaneously, it avoids excessively short text, which would limit optimization space and consume resources due to multiple calls to the text optimization model. Furthermore, the entire segmentation process does not require waiting for complete text, perfectly adapting to the streaming reception rhythm and achieving dynamic processing of "receiving a segment, detecting a segment, and segmenting upon reaching the target." This satisfies the real-time requirements of streaming interaction while balancing "semantic integrity" and "processing efficiency" through dual conditions.

[0043] In some embodiments, each original text block obtained by segmentation may include one or more complete paragraphs in the thought text, and the number of words meets the preset word count requirement, such as not less than the preset lower word count limit and not more than the preset upper word count limit, so as to ensure that the word count meets the requirements and that the semantics are coherent.

[0044] In one or more exemplary embodiments, to improve processing efficiency, the steps of segmenting the streaming thought chain text and optimizing multiple original text blocks can be performed asynchronously. For example, when segmenting the streaming thought chain text, after each segmentation yields an original text block, the currently segmented original text block can be stored in a preset buffer pool. When optimizing the original text blocks, the currently optimized original text blocks can be retrieved sequentially from the buffer pool. For each currently optimized original text block, a prompt word is constructed based at least on a preset text optimization strategy and the content of the original text block. The prompt word guides the text optimization model to optimize the original text block.

[0045] For example, such as Figure 4 As shown, the steps of segmenting the streaming thought chain text and optimizing multiple original text blocks can be executed by two different modules (e.g., different processing nodes, different threads, different processes, or different microservices). The text segmentation module responsible for text segmentation can receive the streaming thought chain text output by the generative language model. When it determines that the current text meets the text segmentation rules (such as paragraph marks + word count limits), it switches the text. After each original text block is segmented, it is stored in a preset buffer pool according to the generation sequence. The buffer pool can take the form of a memory queue, shared cache, etc., and supports sequential access. Then, it can directly return to continue processing the segmentation of the next segment of streaming text. The text optimization module, responsible for text optimization, starts and runs independently of the text segmentation module. It sequentially retrieves raw text blocks to be optimized from the buffer pool using a sequential reading mechanism (strictly adhering to the order in which text blocks are stored in the buffer pool to ensure consistency with the logical order of the model's output thought chain). For each raw text block to be optimized, it constructs a unique prompt word by combining preset text optimization strategies (such as compression, style unification, and logical streamlining), the specific content of the raw text block, and previously optimized historical text blocks and their corresponding optimization results. This prompt word is then input into the text optimization model to guide it in performing targeted optimization and generating optimized text blocks. Throughout this process, the "text segmentation - storage to buffer pool" action of the text segmentation module and the "retrieve raw text blocks from buffer pool - optimization" action of the text optimization module are executed asynchronously, without direct dependency. Text segmentation will not stall due to slow processing speed of the text optimization module, and text optimization does not need to wait for the text segmentation module to generate all text blocks. The two achieve asynchronous data transfer and decoupling through the buffer pool.

[0046] Asynchronous decoupling eliminates the bidirectional blocking of traditional serial processing, where "text segmentation waits for text optimization, and text optimization waits for text segmentation." For example, the text segmentation module can immediately segment the second text block after storing the first, while the text optimization module processes the first simultaneously, allowing segmentation and optimization to proceed in parallel. This significantly reduces the total time from receiving to completing the streaming thought process. Furthermore, a buffer pool can temporarily store segmented original text blocks. Even if the text optimization module is briefly busy, the text segmentation module can continue processing newly received streaming text, preventing interruptions in streaming text reception due to optimization delays and ensuring that the "receive-segmentation" process does not slow down the initial front-end display latency. Moreover, resources can be flexibly allocated based on the computational load differences between text segmentation and text optimization (e.g., allocating more computing power to the computationally intensive optimization module and more bandwidth to the I / O-intensive segmentation module), avoiding single resource bottlenecks that limit overall process efficiency and thus improving resource utilization.

[0047] In one or more exemplary embodiments, the operation of segmenting the streaming thought chain text is performed by a first thread, and the operation of optimizing multiple original text blocks is performed by at least one second thread different from the first thread.

[0048] For example, the segmentation operation of the streaming thought chain text can be assigned to a first thread. This first thread continuously receives streaming data output from the generative language model. Once it acquires incremental text, it immediately performs segmentation according to preset segmentation rules (such as paragraph marks + word count constraints), generating raw text blocks and quickly storing them in a preset buffer pool. The optimization processing of the raw text blocks is handled by at least one second thread independent of the first thread. Each second thread extracts the raw text blocks to be optimized from the buffer pool through a sequential reading mechanism (strictly following the order in which raw text blocks are stored in the buffer pool to ensure that the reasoning logic is not disordered). For each raw text block, it can combine preset text optimization strategies with the text block content to construct prompt words, and then guide the text optimization model to complete the optimization and generate optimized text blocks. The optimization of raw text blocks can be performed serially or in parallel. For example, in a serial execution scenario, the optimization of the current raw text block can be completed before the optimization of the next raw text block, that is, only one raw text block is optimized at a time, thus requiring only one second thread to be called. In parallel execution scenarios, N (N is an integer greater than or equal to 2) original text blocks can be obtained from the buffer pool simultaneously. Then, the text optimization model can be used to optimize the N original text blocks in parallel. In this scenario, N second threads can be called, and each second thread is used to perform the optimization operation of one original text block.

[0049] In one or more exemplary embodiments, the current raw text block to be optimized may include multiple blocks, and the text optimization model can be used to optimize multiple current raw text blocks to be optimized in parallel, wherein the amount of parallel processing of the raw text blocks processed in parallel by the text optimization model is determined based on the backlog of raw text blocks in the buffer pool, wherein the amount of parallel processing is positively correlated with the backlog.

[0050] For example, when multiple raw text blocks to be optimized accumulate in the buffer pool, the text optimization model can initiate parallel processing mode, simultaneously performing optimization operations on multiple currently unoptimized raw text blocks. The number of raw text blocks processed in parallel by the text optimization model (i.e., the parallel processing volume) can be dynamically adjusted based on the backlog of raw text blocks in the buffer pool (referring to the total number of raw text blocks currently stored in the buffer pool but not yet optimized), and the two are positively correlated. For instance, the system can monitor the backlog in the buffer pool in real time; if the backlog is low, a lower parallel processing volume is applied. If the backlog is large, the parallel processing volume is increased to avoid excessive backlog of raw text blocks.

[0051] The above methods can quickly digest the backlog in the buffer pool and prevent blockage of the upstream segmentation process. When the buffer pool experiences a temporary lag in processing speed due to optimization, resulting in a backlog of text blocks, the synchronous increase in parallel processing can quickly reduce the backlog and prevent the segmentation module from being unable to store newly segmented text blocks due to a full buffer pool, thus ensuring the continuous and smooth operation of the entire "segmentation-optimization" chain.

[0052] In one or more exemplary embodiments, the number of characters in the original text block is not less than a preset lower limit, which can be determined based on the backlog of the original text block in the buffer pool; wherein the lower limit is positively correlated with the backlog.

[0053] The segmentation of the original text block must meet the basic constraint of "the number of characters is not less than the preset lower limit" to avoid semantic breaks caused by excessively short text blocks (such as containing only scattered words or incomplete short sentences), ensuring that subsequent optimization can be based on coherent reasoning fragments. In some scenarios, this lower limit can be dynamically adjusted based on the backlog of original text blocks in the buffer pool (i.e., the total number of text blocks stored in the buffer pool but not yet optimized), and the two are positively correlated. For example, when the backlog is low, the lower limit is set to a lower value, and the segmentation can more flexibly capture short and complete semantic units, ensuring semantic integrity while avoiding excessively long text blocks that affect real-time optimization efficiency. When the backlog in the buffer pool is high, the lower limit can be adjusted upwards to reduce the total number of text blocks to be optimized in the buffer pool and avoid excessive accumulation.

[0054] The above approach can ensure semantic integrity while flexibly adapting processing efficiency, alleviating buffer pool congestion, and avoiding an imbalance between the pace of upstream segmentation and downstream optimization.

[0055] When constructing prompt words based on a preset text optimization strategy and the content of the original text block, "historical optimization examples" can be introduced to ensure optimization continuity and improve optimization results. Therefore, in one or more exemplary embodiments, when constructing prompt words for the original text block based on a preset text optimization strategy and the content of the original text block, for each original text block to be optimized, the corresponding historical text block and its corresponding optimization result can be obtained; wherein, the historical text block is output from the generative language model before the original text block, and the historical text block has already undergone optimization processing. Historical optimization examples can be constructed using each historical text block and its corresponding optimization result, and prompt words can be constructed based on the content of the original text block, the text optimization strategy, and the historical optimization examples. These prompt words guide the text optimization model to optimize the original text block.

[0056] For example, for the current original text block to be optimized, we can obtain its corresponding "historical text block" and the optimization result of that historical text block. Here, a historical text block refers to an original text block that was output from the generative language model before the current original text block to be optimized and has already undergone optimization. Then, each historical text block can be associated with its corresponding optimization result to construct a "historical optimization example." For instance, the original text of all the historical text blocks that have been optimized before the current text block can be concatenated in output order to obtain a piece of original text. Then, the optimized text blocks corresponding to each historical text block can be concatenated in order to obtain a piece of optimized text. Thus, we can obtain the mapping relationship of "original content → optimization result" and the specific optimization logic. When constructing prompt words, it does not rely solely on the content of the current original text block and abstract text optimization strategies. Instead, it incorporates historical optimization examples as "concrete references" into the prompt words, forming a composite guide of "current block content + optimization strategy + historical optimization examples." This guides the text optimization model to both follow the preset strategy and refer to the optimization methods of historical text blocks (such as the simplification scale and sentence adjustment logic of similar reasoning fragments) when processing the current block, thereby generating optimized text blocks that match the previous optimization results.

[0057] By employing the above methods, the problem of "isolated block optimization" can be reduced. Historical optimization examples provide a unified optimization standard for the text optimization model, avoiding stylistic inconsistencies (e.g., a concise statement in one block followed by redundant expansion) or logical disjointness (e.g., a reasoning premise emphasized in one block is mistakenly deleted in another). This ensures that the entire thought chain maintains a consistent style and coherent reasoning logic after optimization. Furthermore, historical optimization examples transform abstract optimization strategies into concrete "operational cases," reducing the model's misunderstanding of the strategies and preventing over- or under-optimization, thus significantly improving optimization effectiveness.

[0058] For example, when calling an optimization text model to optimize the original text to be optimized, each call can assemble an incremental prompt. This prompt uses variables such as the original thought process, the historical optimization thought process, the current thought process text, and controllable requirements (e.g., formatting requirements, compression requirements, style, etc.) to form a dynamic prompt. This prompt guides the text optimization model to complete the task of continuing the current original text block into the existing output, referencing historical optimization results. In other words, each optimization input is the current original text block, and the output is the optimized current text block. The current input and output are concatenated with the historical input and output to form an optimization example with historical context for reference.

[0059] In one or more exemplary embodiments, the text optimization strategy is input by the user through interactive components in the interactive interface after the user enters a question. For example, the determination of the text optimization strategy can introduce a user-initiated participation mechanism. After the user completes the input of the question in the interactive interface (such as an intelligent question-answering platform or a model interaction terminal), the interface will simultaneously display interactive components for customizing the text optimization strategy. These interactive components can be designed in various forms according to the needs of the scenario. For example, the user can select the optimization style (such as "popular expression", "preserve technical terms", "academic and rigorous style") through a drop-down menu, select optimization rules (such as "compress redundant text", "split long sentences", "extract reasoning nodes", "add structured tags") through a checkbox, or fill in custom optimization requirements (such as "emphasize the formula derivation process" and "delete repeated premise descriptions") through a text input box. After the user completes the interactive operation according to their own reading needs of the thought chain text, the system can collect these interactive input information in real time, integrate and generate targeted text optimization strategies, and use them as the core basis for subsequently constructing prompt words and guiding the text optimization model to process the original text blocks.

[0060] The above approach enables personalized customization of text optimization strategies, breaking the limitations of the traditional fixed text optimization strategy's "one-size-fits-all" approach. This allows the optimized thought chain text to accurately match the different needs of different users (such as users with different knowledge backgrounds and usage scenarios having different requirements for the style and level of detail in the thought chain), significantly improving user satisfaction with the optimization results.

[0061] In one or more exemplary embodiments, the text optimization strategy includes one or more of the following: the style of the optimized text, the format requirements of the optimized text, and the compression rules of the optimized text.

[0062] This can be achieved by setting differentiated style directions based on user needs or application scenarios. For example, there can be a "popular style" for general users (simplifying technical terms and explaining reasoning logic in everyday language) and a "professional style" for professional fields (retaining technical terms and maintaining academic / industry rigor). The optimized text format should focus on the structured presentation of the text, such as setting "paragraph division rules" (splitting paragraphs according to reasoning steps), "labeling requirements" (adding structured labels to premises, derivation processes, and conclusions), and "long sentence processing guidelines" (breaking complex long sentences into groups of shorter sentences to reduce reading difficulty). The optimized text compression rules can aim to increase information density, clearly defining "retained content" (core reasoning premises, key derivation logic, and conclusion predictions) and "removed content" (repeated premise statements, irrelevant transitional statements, and redundant examples). Compression levels can also be set (e.g., "light compression" removes only obvious redundancy, while "heavy compression" retains only the reasoning framework). In practical applications, you can choose a single dimension (such as enabling only compression rules if you only need to compress redundancy) or combine multiple dimensions (such as enabling "professional style + structured format + medium compression" at the same time) to form a complete optimization strategy, depending on your needs.

[0063] Multi-dimensional text optimization strategies can cover the core optimization needs of the thought chain, solving the problem of "disorganized style" (style dimension), improving the defect of "loose structure" (format dimension), and eliminating the drawback of "information redundancy" (compression dimension), thus comprehensively improving the readability and practical value of the optimized text.

[0064] In one or more exemplary embodiments, the text optimization model can optimize multiple original text blocks obtained by segmentation in any of the following ways: (1) optimizing multiple original text blocks serially; (2) optimizing multiple original text blocks in each group in parallel, wherein the multiple original text blocks are divided into multiple groups, each group including several original text blocks continuously output by the generative language model.

[0065] In this context, serial processing refers to the process where the optimization, collection, and streaming of the previous original text block are completed before the processing of the next original text block begins. The additional intermediate waiting time is the TTFT (Time-To-Flight Transition) of each original text block fed into the text optimization model. During serial processing, since each original text block can use the optimization results of the previous complete content as an example reference, semantic coherence can be maintained, resulting in better optimization effects. However, because each original text block must wait for the optimization of the previous original text block to be completed before it can begin optimization, the intermediate waiting time is relatively long, leading to lower efficiency.

[0066] Parallel processing means that multiple raw text blocks can be fed into the text optimization model in parallel, forming a parallel pipeline effect. Parallel processing can shorten the intermediate waiting time and improve processing efficiency. However, each raw text block cannot be used as an example reference based on the optimization results of the previous complete content. Its optimization effect will be lower than that of serial optimization. Furthermore, the larger the amount of parallel processing, the higher the processing efficiency, but the worse the optimization effect will be.

[0067] like Figure 5 As shown, the TFTT duration and additional intermediate waiting time are illustrated under different scenarios, including no optimization of the thought chain text, serial execution of text chunking and text optimization, asynchronous execution of text chunking and text optimization, and parallel execution of text optimization.

[0068] In scenarios where the thought chain text is not optimized, the thought chain text output by the generative language model can be directly collected and streamed.

[0069] In scenarios where text chunking and text optimization are performed sequentially, the streaming thought chain text output by the generative language model can be first segmented into chunks, then optimized, and then the next chunk can be collected and optimized. Compared to a scenario without optimization, this adds TTFT (Time-To-Through Time) and intermediate waiting time. Because text chunks must be optimized strictly in sequence, an additional TTFT is added; this additional TTFT represents the text chunking time and the time the text optimization model spends optimizing the chunks. Similarly, the additional intermediate waiting time is the text chunking time and the time the text optimization model spends optimizing the chunks.

[0070] For example, in the sequential processing of text segmentation and text optimization, we can first collect the streaming thought chain text output by the generative language model. After confirming that it conforms to the text segmentation rules, we can segment the streaming thought chain text to obtain the original text block 1. Then, based on the original text block 1 and the text optimization strategy, we can construct a prompt word 1 and input this prompt word 1 into the text optimization model to obtain the optimized text block 1 corresponding to the original text block 1. Then, we can push the optimized text block 1 to the client to display it to the user. Next, we can collect the original text block 2, construct historical optimization examples based on the original text block 1 and optimized text block 1, and then construct prompt word 2 based on the historical optimization examples, the original text block 2, and the text optimization strategy. We can input this prompt word 2 into the text optimization model to obtain the optimized text block 2 corresponding to the original text block 2, and then push the optimized text block 2 to the client. Then, we can concatenate the original text block 2 and optimized text block 2 into the historical input and output to form "original text block 1 + original text block 2" and "optimized text block 1 + optimized text block 2" to construct new historical optimization examples as example references for the next original text block. In serial processing, each original text block can be used as an example reference based on the optimization result of the preceding complete content, thus maintaining semantic coherence and achieving better optimization results. However, since each original text block must wait for the previous original text block to be optimized before it can begin optimization, the intermediate waiting time is relatively long, resulting in low efficiency.

[0071] Asynchronous execution of text segmentation and text optimization refers to decoupling the two steps of text segmentation and text optimization. This means that text segmentation and text optimization can be performed by different modules, and the segmented text blocks can be stored in a buffer pool. For example, the text segmentation module can first collect the streaming thought chain text output by the generative language model. After determining that it conforms to the text segmentation rules, it can segment the streaming thought chain text to obtain the original text block 1, which is then stored in the buffer pool. Further text segmentation can then be performed. The text optimization module can retrieve the segmented original text block 1 from the buffer pool and optimize it to obtain the optimized text block 1. Because text segmentation and text optimization are performed asynchronously, while optimizing the original text block 1, text segmentation can be performed simultaneously to obtain the original text block 2. This method can shorten the additional intermediate waiting time. For example, as... Figure 5 As shown, in this scenario, the additional intermediate waiting time can be shortened, which is the time for the text optimization model to optimize the text block.

[0072] Furthermore, to further shorten the intermediate waiting time and improve processing efficiency, the optimization process for the original text blocks can also be executed in parallel. That is, each time the text optimization model is called, a batch (i.e., several) of the original text blocks can be input. For example, during the parallel optimization of the original text blocks, multiple original text blocks can be input into the optimization model in parallel to improve efficiency. For instance, taking a parallel processing volume of 2 (i.e., optimizing 2 original text blocks in parallel each time) as an example, the streaming thought chain text output by the generative language model can be segmented to obtain original text block 1 and original text block 2. For each original text block, prompt words can be constructed based on the text optimization strategy and input into the text optimization model so that the text optimization model can optimize original text block 1 and original text block 2 in parallel to obtain optimized text block 1 and optimized text block 2. Then, we can obtain the original text block 3 and original text block 4 obtained from the segmentation. For each original text block, we can construct historical optimization examples based on "original text block 1 + original text block 2" and "optimized text block 1 + optimized text block 2". Then, based on the content of each original text block, the text optimization strategy, and the historical optimization examples, we can construct prompt words to obtain optimized text block 3 and optimized text block 4. Then, we can concatenate "original text block 3, original text block 4" and "optimized text block 3, optimized text block 4" into the historical input and output respectively to form new historical optimization examples, which will serve as example references for the next batch of original text blocks.

[0073] During parallel processing, the historical optimization examples of the same batch of original text blocks are the same, that is, they are constructed using the same batch of historical contexts. The historical optimization examples of different batches of original text blocks are different, that is, they are constructed using different batches of historical contexts.

[0074] The thought chain text optimization method provided in the embodiments of this specification allows for controllable optimization of thought chain text. For example, the text optimization strategy can be defined by the user according to their needs; the synchronization mode in the text optimization process (e.g., whether text segmentation and text optimization are executed synchronously or asynchronously, whether the text optimization steps are executed serially or in parallel) can be defined by the user based on their needs; and the parallel processing volume (i.e., the number of text blocks processed in parallel during the text optimization process) can also be defined by the user. Users can define the above indicators based on their needs for optimization effect, processing efficiency, and the throughput of the generative language model, thereby achieving autonomous control over the thought chain text optimization process.

[0075] Figure 6 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 6As shown, device 400 mainly consists of a communication interface 402, a user interface 404, a processor 406, and a data storage 408. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 410. The communication interface 402 enables device 400 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 402 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 402 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 402 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 402 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces.

[0076] User interface 404 includes receiving user input and providing output to the user. Therefore, user interface 404 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 404 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 404 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 400 may support remote access from other devices via communication interface 402 or another physical interface (not shown). User interface 404 may be configured to receive user input, the position and movement of which may be indicated by an indicator or cursor described herein. User interface 404 may also be configured as a display device for rendering or displaying text fragments.

[0077] Processor 406 may contain one or more general-purpose processors and / or special-purpose processors.

[0078] Data storage 408 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 406. Data storage 408 may include removable and non-removable components.

[0079] Processor 406 is capable of executing program instructions 418 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 408 to perform the various functions described herein. Data storage 408 may comprise a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 400, enable device 400 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Processor 406 executing program instructions 418 may result in processor 406 using data 412.

[0080] For example, program instructions 418 may include an operating system 422 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 400 and one or more applications 420 (e.g., a browser, social application, or game application). Similarly, data 412 may include operating system data 416 and application data 414. Operating system data 416 is primarily accessible to the operating system 422, while application data 414 is primarily accessible to one or more applications 420. Application data 414 may reside in a file system visible or hidden from the user of device 400.

[0081] Application 420 can communicate with operating system 422 through one or more application programming interfaces (APIs). These APIs help application 420 read and / or write application data 414, transmit or receive information via communication interface 402, receive or display information on user interface 404, etc.

[0082] In some terminology, application 420 may be simply referred to as "app". Furthermore, application 420 can be downloaded to device 400 through one or more online app stores or app markets. However, applications can also be installed on device 400 in other ways, such as through a web browser or a physical interface on device 400 (e.g., a USB port).

[0083] The MindChain Text Optimization Device can be applied to, for example... Figure 6 The device shown is used to implement the technical solution of this specification. The device may include: The acquisition module is used to acquire the streaming thought chain text generated by the generative language model in response to the user's question. The streaming thought chain text reflects the thinking process of the generative language model in response to the user's question. The segmentation module is used to segment the streaming thought chain text to obtain multiple original text blocks; The optimization module is used to construct a prompt word corresponding to each of the plurality of original text blocks based at least on a preset text optimization strategy and the content of the original text block, and guide the text optimization model to optimize the original text block through the prompt word to obtain the optimized text block corresponding to the original text block. The output module is used to stream the optimized text blocks corresponding to each of the multiple original text blocks in the order in which the generative language model outputs the multiple original text blocks.

[0084] In one or more exemplary embodiments, the number of characters in each original text block obtained by segmentation does not exceed a preset upper limit on the number of characters, the upper limit on the number of characters being determined based on the expected first display delay, the first display delay being the delay from calling the generative language model to answer the user's question to displaying the first word of the generative language model output to the user.

[0085] In one or more exemplary embodiments, the segmentation module is used to segment the thought chain text to obtain streaming original text blocks, specifically for: The system continuously receives the streaming thought chain text output by the generative language model, performs paragraph mark detection on the received streaming thought chain text, and counts the number of words in the currently received and unblocked streaming thought chain text. The currently received and unblocked streaming thought chain text will be treated as a single original text block if any of the following conditions are met: Paragraph markers are identified in the received streaming thought chain text, and the number of words in the currently received and unblocked streaming thought chain text is not less than the preset minimum word count; or The number of characters in the currently received and unblocked streaming thought chain text has reached the preset character limit.

[0086] In one or more exemplary embodiments, the segmentation module is used to segment the streaming thought chain text to obtain multiple original text blocks, specifically for: The streaming thought chain text is segmented, and after each segmentation to obtain an original text block, the currently segmented original text block is stored in a preset buffer pool. The optimization module is used to construct a prompt word corresponding to each of the plurality of original text blocks, based at least on a preset text optimization strategy and the content of the original text block, and guide the text optimization model to optimize the original text block through the prompt word. Specifically, it is used for: According to the storage order of the plurality of original text blocks, the current original text block to be optimized is obtained from the buffer pool in sequence. For each current original text block to be optimized, a prompt word for the original text block is constructed based on at least the preset text optimization strategy and the content of the original text block. The prompt word guides the text optimization model to optimize the original text block.

[0087] In one or more exemplary embodiments, the operation of segmenting the streaming thought chain text is performed by a first thread, and the operation of optimizing the plurality of original text blocks is performed by at least one second thread different from the first thread.

[0088] In one or more exemplary embodiments, the current original text blocks to be optimized include a plurality of them, and the text optimization model is used to optimize the plurality of current original text blocks to be optimized in parallel. The amount of parallel processing of the original text blocks processed in parallel by the text optimization model is determined based on the backlog of original text blocks in the buffer pool, wherein the amount of parallel processing is positively correlated with the backlog.

[0089] In one or more exemplary embodiments, the number of characters in the original text block is not less than a preset lower limit, which is determined based on the backlog of original text blocks in the buffer pool; wherein the lower limit is positively correlated with the backlog.

[0090] In one or more exemplary embodiments, the text optimization model optimizes the plurality of original text blocks in any of the following ways: Optimize the multiple original text blocks sequentially; or The optimization process is performed in parallel on multiple original text blocks in each group, wherein the multiple original text blocks are divided into multiple groups, and each group includes several original text blocks continuously output by the generative language model.

[0091] In one or more exemplary embodiments, the optimization module is used to construct prompt words for each original text block, at least based on a preset text optimization strategy and the content of the original text block, specifically for: For each original text block to be optimized, obtain the corresponding historical text block and the optimization result corresponding to the historical text block; wherein, the historical text block is output from the generative language model before the original text block, and the historical text block has already completed the optimization process; Construct historical optimization examples using each historical text block and the corresponding optimization results; Based on the content of the original text block, the text optimization strategy, and the historical optimization examples, prompt words are constructed to guide the text optimization model to optimize the original text block.

[0092] In one or more exemplary embodiments, the text optimization strategy is input by the user through an interactive component in the interactive interface after the user inputs the user question; and / or The text optimization strategy includes one or more of the following: the style of the optimized text, the format requirements of the optimized text, and the compression rules of the optimized text.

[0093] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0094] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0095] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0096] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0097] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0098] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0099] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0100] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0101] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0102] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0103] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A method for optimizing thought chain text, characterized in that, The method includes: Obtain the streaming thought chain text generated by the generative language model in response to the user's question, wherein the streaming thought chain text reflects the thinking process of the generative language model in response to the user's question; The streaming thought chain text is segmented to obtain multiple original text blocks; For each of the plurality of original text blocks, at least based on a preset text optimization strategy and the content of the original text block, a prompt word corresponding to the original text block is constructed, and the prompt word guides the text optimization model to optimize the original text block to obtain the optimized text block corresponding to the original text block. The optimized text blocks corresponding to each of the multiple original text blocks are streamed according to the order in which the generative language model outputs the multiple original text blocks.

2. According to the method of claim 1, the number of characters in each original text block obtained by segmentation does not exceed a preset upper limit of the number of characters, the upper limit of the number of characters is determined based on the expected first display delay, the first display delay is the delay from calling the generative language model to answer the user question to displaying the first word unit output by the generative language model to the user.

3. The method according to claim 1, wherein segmenting the thought chain text to obtain streaming original text blocks includes: The system continuously receives the streaming thought chain text output by the generative language model, performs paragraph mark detection on the received streaming thought chain text, and counts the number of words in the currently received and unblocked streaming thought chain text. The currently received and unblocked streaming thought chain text will be treated as a single original text block if any of the following conditions are met: Paragraph markers are identified in the received streaming thought chain text, and the number of words in the currently received and unblocked streaming thought chain text is not less than the preset minimum word count. or The number of characters in the currently received and unblocked streaming thought chain text has reached the preset character limit.

4. The method according to claim 1, wherein the step of segmenting the streaming thought chain text to obtain multiple original text blocks includes: The streaming thought chain text is segmented, and after each segmentation to obtain an original text block, the currently segmented original text block is stored in a preset buffer pool. For each of the plurality of original text blocks, at least based on a preset text optimization strategy and the content of the original text block, a corresponding prompt word is constructed for that original text block, and the prompt word guides the text optimization model to optimize the original text block, including: According to the storage order of the plurality of original text blocks, the current original text block to be optimized is obtained from the buffer pool in sequence. For each current original text block to be optimized, a prompt word for the original text block is constructed based on at least the preset text optimization strategy and the content of the original text block. The prompt word guides the text optimization model to optimize the original text block.

5. The method according to claim 4, wherein the operation of segmenting the streaming thought chain text is performed by a first thread, and the operation of optimizing the plurality of original text blocks is performed by at least one second thread different from the first thread.

6. The method according to claim 4, wherein the current original text blocks to be optimized include multiple blocks, the text optimization model is used to optimize the multiple current original text blocks to be optimized in parallel, and the parallel processing amount of the original text blocks processed in parallel by the text optimization model is determined based on the backlog of original text blocks in the buffer pool, wherein, The amount of parallel processing is positively correlated with the amount of backlog.

7. The method according to claim 4, wherein the number of characters in the original text block is not less than a preset lower limit, and the lower limit is determined based on the backlog of original text blocks in the buffer pool; wherein, The lower limit of the word count is positively correlated with the backlog amount.

8. The method according to any one of claims 1-7, wherein the text optimization model optimizes the plurality of original text blocks in any of the following ways: Optimize the multiple original text blocks sequentially; or The optimization process is performed in parallel on multiple original text blocks within each group, where... The multiple original text blocks are divided into multiple groups, and each group includes several original text blocks continuously output by the generative language model.

9. The method according to any one of claims 1-7, wherein constructing prompt words for each original text block based at least on a preset text optimization strategy and the content of the original text block includes: For each original text block to be optimized, obtain the corresponding historical text block and the optimization result corresponding to the historical text block; wherein, the historical text block is output from the generative language model before the original text block, and the historical text block has already completed the optimization process; Construct historical optimization examples using each historical text block and the corresponding optimization results; Based on the content of the original text block, the text optimization strategy, and the historical optimization examples, prompt words are constructed to guide the text optimization model to optimize the original text block.

10. The method according to claim 1, wherein the text optimization strategy is input by the user through an interactive component in the interactive interface after the user inputs the user question; and / or The text optimization strategy includes one or more of the following: the style of the optimized text, the format requirements of the optimized text, and the compression rules of the optimized text.

11. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-10 by executing the executable instructions.

12. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-10.

13. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-10.

Images