A front-end rendering optimization method based on streaming output
By introducing message components, stream message stacks, and message ending signs in the GPT streaming environment, the unstable problem of streaming output is solved, and the correct display of the Markdown format is ensured through the text distribution and analysis of the text text of the Markdown text. Stability of streaming output and improvement of user experience.
Patent Information
- Application Number
- CN202411321769.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-01-28
- Estimated Expiration
- 2044-09-23
AI Technical Summary
The instability and lag problems of streaming output in the prior art, as well as the correct display of Markdown document format in streaming output.
By creating the GPT streaming output environment, including message components, stream message stacks, and message ending signs to ensure the order of stream message. At the same time, identify the special characters and structures of Markdown, divide the text and add a segmentation mark to ensure the integrity of the Markdown syntax in the flow transmission process.
Improve the stability of the streaming output, avoid the loss or disorder of the message, ensure the correct analysis and rendering of the Markdown style, and improve the user experience and response speed.
Smart Images
Figure CN118869803B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electronic digital data processing, in particular to the technical field of user interface implementation, and specifically to a front-end rendering optimization method based on streaming output. Background Art
[0002] Currently, generative pre-trained models such as GPT are widely used, and data processing and analysis, as well as real-time interactive experience have become key issues. Streaming output technology, as a bridge connecting AI models and user interfaces, plays an important role in real-time data transmission and display. However, under the complex and changeable network environment and large-scale data processing requirements, traditional streaming output technology faces many challenges in terms of stability and efficiency; front-end rendering, as the core technology of the display layer, directly affects the user experience. In AI services, front-end rendering not only needs to process real-time data streams from models such as GPT, but also needs to ensure the correct display of data formats, which is crucial to improving the overall service response speed and user satisfaction.
[0003] When processing large-scale data, the GPT model relies on a stable server environment and high-speed network connection. However, factors such as server load fluctuations, network delays or interruptions can cause freezes or delays in the streaming output process, affecting the real-time and continuity of the data. The amount of data returned by the GPT model varies significantly due to query complexity and the model's internal processing mechanism. This results in uneven density of data flow during streaming, which is sometimes sparse and sometimes dense, further exacerbating the instability of the output.
[0004] After receiving streaming data, the front end needs to parse and render Markdown documents in real time. Markdown documents rely on characters in a specific format to define the style, and these characters must appear in pairs to be correctly parsed. However, in the streaming scenario, since the data is transmitted step by step, it is easy to cause character fragments to be transmitted separately, resulting in the Markdown style not being rendered correctly, and the formatting being chaotic or wrong.
[0005] Therefore, it is necessary to improve the front-end rendering method based on streaming output in the prior art to solve the above problems. Summary of the invention
[0006] The present invention overcomes the shortcomings of the prior art and provides a front-end rendering optimization method based on streaming output, aiming to solve the instability and jamming problems of streaming output in the prior art, as well as the problem of correct display of markdown document format in streaming output.
[0007] To achieve the above object, the technical solution adopted by the present invention is: a front-end rendering optimization method based on streaming output, comprising:
[0008] S1. Create a GPT streaming output environment, including: message components, streaming message stack and message end flag;
[0009] S2, GPT pushes streaming messages to the message component;
[0010] S3, the message component stores the streaming message data into the streaming message stack;
[0011] S4. When the content of the streaming message stack is added, the message component processes the data in the message stack;
[0012] S5. After GPT pushes the streaming message, it notifies the message component, which records the message end mark and cleans up the data in the message stack.
[0013] S6. When there is no data in the streaming message stack and the message end mark appears, the streaming message output ends.
[0014] In a preferred embodiment of the present invention, in step S2, the GPT server segments the generated text in a streaming output manner, and pushes the generated text to the front-end message component one by one through the GPT push method. The text segmentation includes:
[0015] S21. Identify special characters and structures of markdown;
[0016] S22, determine the segmentation point according to the special characters and structure of markdown;
[0017] S23, segmenting the text according to the segmentation point and adding segmentation marks;
[0018] S24: Push the segmented text segments.
[0019] In a preferred embodiment of the present invention, special characters and structures in markdown are recognized, including: titles, lists, code blocks, links, pictures, and emphasized text.
[0020] In a preferred embodiment of the present invention, the segmentation points are set at positions that do not affect the markdown syntax and rendering effects, including spaces, punctuation marks, and line breaks.
[0021] In a preferred embodiment of the present invention, after determining the segmentation points, the text generated by GPT is segmented into multiple segments; for each segment, when it contains the beginning part of the special characters or structure of markdown but does not contain the ending part, a start segmentation marker is added at the end of the segment; when the segment contains the ending part but does not contain the beginning part, an end segmentation marker is added at the beginning of the segment.
[0022] In a preferred embodiment of the present invention, in step S4, the segments segmented in S2 are parsed before processing the data in the message stack:
[0023] S41, checking whether the retrieved text segment contains a segmentation mark;
[0024] S42, the text fragment contains a start segmentation tag, and a new Markdown object is created to indicate the beginning of the structure;
[0025] S43: The text fragment contains an end segmentation marker, and the corresponding Markdown object is searched and updated to indicate the end of the structure.
[0026] In a preferred embodiment of the present invention, in step S4, the step of processing the data in the message stack includes:
[0027] S44, check whether the extracted word is a line break character, if so, clear all properties of the current markdown object and continue to process the next word.
[0028] S45. Check whether the extracted word is a special character of markdown:
[0029] A. When the extracted word is a special character of markdown, further determine whether the character is a package character, and perform corresponding processing according to the current state of the markdown object:
[0030] If the same package character already exists, delete the attribute of the character in the markdown object and update the display container;
[0031] If there is no same wrapping character, create an attribute of the character type in the markdown object, and create a corresponding div element to insert into the display container;
[0032] B. If the extracted word is not a special character of markdown, it is treated as a normal character and inserted according to the current state of the markdown object:
[0033] When there is any attribute in the markdown object, insert the current normal character into the corresponding div element;
[0034] When no attributes exist in the markdown object, ordinary characters are directly inserted into the display container.
[0035] In a preferred embodiment of the present invention, in step S3, the processing process of the streaming message stack is monitored and logged, and the log record includes: the processing time of the streaming message, the streaming message rendering time and the streaming message error information.
[0036] In a preferred embodiment of the present invention, a buffer queue is created as a storage space for temporary storage; when the GPT pushes streaming messages faster than the message component processes them, the GPT pushes the streaming messages to the buffer queue, and the message component then takes out and processes the data from the buffer queue.
[0037] The present invention provides a front-end rendering system based on streaming output, comprising:
[0038] The message component module is used to receive streaming messages pushed by GPT, is responsible for the interaction between the front end and the GPT server, and obtains the content generated by GPT in real time through API or WebSocket;
[0039] The streaming message stack module is used to temporarily store the received streaming message data. As a temporary storage area, this module can ensure the order and integrity of the message and avoid message loss or confusion;
[0040] The message end mark module is used to mark whether the streaming message has been completely received. This module determines when to stop waiting for the arrival of new messages and start the final processing and rendering;
[0041] The text segmentation module is used to segment the text generated by GPT. The module first recognizes the special characters and structure of markdown, then determines the segmentation points, divides the text into multiple segments, and adds segmentation markers;
[0042] The text parsing module is used to parse the text generated by GPT, parse the text fragments, and create the corresponding HTML structure according to the markdown syntax;
[0043] The push method selection module is used to select the method of GPT push messages. According to the network environment and front-end requirements, WebSocket or HTTP long polling is selected to receive streaming messages from the GPT server. When the network environment is stable, WebSocket is preferred; when the network environment is unstable, a fallback mechanism is set to automatically switch to HTTP long polling;
[0044] Buffer queue module, used as a storage space for temporary storage;
[0045] The error handling module is used to handle network errors or data format errors that occur during streaming output;
[0046] The rendering module is used to render the parsed markdown text to the front-end page and display the text content to the user according to the parsed HTML structure.
[0047] The present invention solves the defects existing in the background technology and has the following beneficial effects:
[0048] (1) The present invention creates a GPT streaming output environment, including a message component, a streaming message stack, and a message end flag, thereby ensuring the sequence and integrity of the streaming message; the text generated by GPT is pushed in a streaming manner, and the message component receives and stores it in the streaming message stack in real time, effectively avoiding the problem of message loss or confusion, thereby improving the stability of the streaming output.
[0049] (2) The present invention performs text segmentation in step S2 and combines it with the push judgment of the GPT push method. The text generated by GPT is reasonably cut into multiple segments and segmentation marks are added to ensure that the streaming message will not cause network congestion or delay due to the excessive size of a single message during transmission, thereby improving the stability of the streaming output. Then, the corresponding GPT push method including WebSocket and HTTP long polling is automatically selected according to the network environment. The combination of the two optimizes the text processing speed and increases the transmission rate.
[0050] (3) In step S2 of the present invention, special characters and structures of Markdown, such as titles, lists, code blocks, etc., are specially considered when segmenting text, ensuring that the segmentation points are selected at positions that do not affect the Markdown syntax and rendering effects. By adding segmentation marks, the present invention ensures the integrity of the Markdown syntax during streaming transmission. Combined with the parsing of the segmentation marks in step S4, the Markdown style can be correctly parsed and rendered even in the streaming transmission scenario, avoiding format confusion or errors.
[0051] (4) The present invention introduces a buffer queue as a temporary storage space, which effectively solves the problem of untimely front-end processing when the GPT push speed is too fast. When the message processing speed cannot keep up with the push speed, the buffer queue can cache redundant messages to avoid front-end rendering jams. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.
[0053] Figure 1 is a flow chart of a preferred embodiment of the present invention;
[0054] Figure 2 It is a schematic diagram of a preferred embodiment of the present invention;
[0055] Figure 3It is a logic diagram of data processing in a message stack of a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.
[0058] like Figure 1 As shown, a front-end rendering optimization method based on streaming output includes the following steps:
[0059] S1. Create a GPT streaming output environment, including: message components, streaming message stack and message end flag;
[0060] S2, GPT pushes streaming messages to the message component;
[0061] S3, the message component stores the streaming message data into the streaming message stack;
[0062] S4. When the content of the streaming message stack is added, the message component processes the data in the message stack;
[0063] S5. After GPT pushes the streaming message, it notifies the message component, which records the message end mark and cleans up the data in the message stack.
[0064] S6. When there is no data in the streaming message stack and the message end mark appears, the streaming message output ends.
[0065] GPT streaming output means that when GPT generates text or messages, it does not return the entire response at once, but returns the results step by step in a streaming manner; GPT streaming output allows the front end to continue receiving and rendering subsequent text while the user sees the initial response, thereby improving the user experience and reducing waiting time.
[0066] Specifically, in step S1, the message component is the interface for the front end to interact with GPT, and is responsible for receiving streaming messages pushed by GPT through API or WebSocket. The message component ensures that the front end can obtain the content generated by GPT in real time and prepare for subsequent rendering and display.
[0067] The streaming message stack is a temporary storage area used to cache streaming message data received from GPT. Due to factors such as network delays or GPT processing time, messages may not arrive continuously. The streaming message stack can ensure the order and integrity of messages and avoid message loss or confusion. The streaming message stack enables the front end to start rendering part of the content before the message is fully received.
[0068] The message end flag is used to indicate whether the streaming message pushed by GPT has completely ended, and determines when to stop waiting for the arrival of new messages and start the final processing and rendering.
[0069] In summary, the message component is used to receive streaming messages pushed by GPT; the streaming message stack is used to temporarily store the received streaming message data; and the message end flag is used to indicate whether the streaming message has been completely received.
[0070] Specifically, in step S2, the GPT server divides the generated text into segments in a streaming output manner, and pushes them one by one to the front-end message component through the GPT push method.
[0071] Text segmentation includes:
[0072] S21. Identify special characters and structures of markdown;
[0073] S22, determine the segmentation point according to the special characters and structure of markdown;
[0074] S23, segmenting the text according to the segmentation point and adding segmentation marks;
[0075] S24, pushing the segmented text segments;
[0076] Specifically, in S21, before segmenting the text, it is necessary to first identify the special characters and structures in the markdown, including: titles, lists, code blocks, links, images, and emphasized text;
[0077] Specifically in S22, the segmentation point is set at a position that does not affect the markdown syntax and rendering effect, including: a space, a punctuation mark, or a line break; and ensures that the segmented text fragments still maintain the integrity of the markdown syntax.
[0078] Specifically, in S23, after determining the segmentation point, the text generated by GPT is segmented into multiple segments; for each segment, when it contains the special characters of markdown or the beginning of the structure but does not contain the end, a start segmentation marker is added at the end of the segment to indicate the existence of subsequent segments and the need to continue rendering; when the segment contains the end but not the beginning, an end segmentation marker also needs to be added at the beginning of the segment.
[0079] GPT push methods include: WebSocket and HTTP long polling. GPT push method determination:
[0080] When the network environment is stable and the front end needs to get the response of the GPT server immediately, use the WebSocket method;
[0081] When the network environment is unstable, WebSocket is preferred. When the WebSocket connection fails, a fallback mechanism is set to automatically switch to HTTP long polling.
[0082] When the network becomes stable again, fall back from HTTP long polling to WebSocket.
[0083] WebSocket is a protocol for full-duplex communication over a single TCP connection. In a WebSocket connection, messages can be pushed from the GPT server to the front-end messaging component in real time.
[0084] In HTTP long polling, the front end sends a request to the GPT server and waits for the GPT server to respond. When the GPT server does not have immediately available data, it keeps the connection open until there is data to send. When the GPT server has data, it immediately sends a response and the front end immediately sends another request.
[0085] Specifically, in step S3, after receiving the streaming messages, the message component will store them in the streaming message stack in the order of receipt for subsequent processing and rendering.
[0086] like Figure 2 As shown in the figure, the streaming message stack storage uses a first-in-first-out data structure to manage streaming message data. In this structure, each new message fragment is added to the end of the data structure, and when processing or rendering the message, the message fragments are taken out from the front of the data structure in sequence. This design ensures the order of the messages, because the earliest received message fragment is at the front of the data structure, and the latest received message fragment is at the end of the data structure.
[0087] Specifically, in step S3, the processing process of the streaming message stack is monitored and logged, and the log records include: the processing time of the streaming message, the streaming message rendering time and the streaming message error information; so as to timely discover and solve problems
[0088] When the streaming message stack is full, the streaming message stack is processed by methods such as discarding old messages, compressing messages, or waiting for stack space to be released.
[0089] like Figure 3As shown, specifically, in step S4, the segments segmented by S2 are parsed before processing the data in the message stack:
[0090] S41, checking whether the retrieved text segment contains a segmentation mark;
[0091] S42, the text fragment contains a start segmentation tag, and a new Markdown object is created to indicate the beginning of the structure;
[0092] S43: The text fragment contains an end segmentation marker, and the corresponding Markdown object is searched and updated to indicate the end of the structure.
[0093] Specifically, in step S4, the step of processing the data in the message stack includes:
[0094] S44, check whether the extracted word is a line break character, if so, clear all properties of the current markdown object and continue to process the next word.
[0095] S45. Check whether the extracted word is a special character of markdown:
[0096] A. When the extracted word is a special character of markdown, further determine whether the character is a package character, and perform corresponding processing according to the current state of the markdown object:
[0097] If the same package character already exists, delete the attribute of the character in the markdown object and update the display container;
[0098] If there is no same wrapping character, create an attribute of the character type in the markdown object, and create a corresponding div element to insert into the display container;
[0099] B. If the extracted word is not a special character of markdown, it is treated as a normal character and inserted according to the current state of the markdown object:
[0100] When there is any attribute in the markdown object, insert the current normal character into the corresponding div element;
[0101] When no attributes exist in the markdown object, ordinary characters are directly inserted into the display container.
[0102] Specifically, in step S4, for a message in markdown format, string processing techniques such as regular expressions are used to parse and render the markdown style.
[0103] Specifically, in step S5, the data in the message stack is cleaned up, including: the message stack, the markdown object, etc. Embodiment 2
[0104] This embodiment is further limited on the basis of the first embodiment.
[0105] Create a buffer queue as a storage space for temporary storage;
[0106] When the GPT message push speed is greater than the message component processing speed, GPT pushes streaming messages to the buffer queue, and then the message component takes out and processes the data from the buffer queue at a stable rate, thereby avoiding front-end rendering jams. Embodiment 3
[0107] This embodiment adds error handling logic based on the first or second embodiment. When a network error or a data format error occurs during the streaming output process, the system can automatically retry or prompt the user to reload the data. Embodiment 4
[0108] This embodiment, based on the first embodiment, the second embodiment or the third embodiment, proposes a front-end rendering system based on streaming output, including:
[0109] The message component module is used to receive streaming messages pushed by GPT, is responsible for the interaction between the front end and the GPT server, and obtains the content generated by GPT in real time through API or WebSocket;
[0110] The streaming message stack module is used to temporarily store the received streaming message data. As a temporary storage area, this module can ensure the order and integrity of the message and avoid message loss or confusion;
[0111] The message end mark module is used to mark whether the streaming message has been completely received. This module determines when to stop waiting for the arrival of new messages and start the final processing and rendering;
[0112] The text segmentation module is used to segment the text generated by GPT. The module first recognizes the special characters and structure of markdown, then determines the segmentation points, divides the text into multiple segments, and adds segmentation markers;
[0113] The text parsing module is used to parse the text generated by GPT, parse the text fragments, and create the corresponding HTML structure according to the markdown syntax;
[0114] The push method selection module is used to select the method of GPT push messages. According to the network environment and front-end requirements, WebSocket or HTTP long polling is selected to receive streaming messages from the GPT server. When the network environment is stable, WebSocket is preferred; when the network environment is unstable, a fallback mechanism is set to automatically switch to HTTP long polling;
[0115] Buffer queue module, used as a storage space for temporary storage;
[0116] The error handling module is used to handle network errors or data format errors that occur during streaming output;
[0117] The rendering module is used to render the parsed markdown text to the front-end page and display the text content to the user according to the parsed HTML structure.
[0118] The above is based on the ideal embodiment of the present invention. Through the above description, relevant personnel can make various changes and modifications without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the content in the specification, and the technical scope must be determined according to the scope of the claims.
Claims
1. A front-end rendering method based on streaming output, characterized in that: Includes steps: S1. Create a GPT streaming output environment, including: message components, streaming message stack and message end flag; S2. GPT pushes streaming messages to the message component; S3, the message component stores the streaming message data into the streaming message stack; S4. When the content of the streaming message stack is added, the message component processes the data in the message stack; S5. After GPT pushes the streaming message, it notifies the message component, which records the message end mark and cleans up the data in the message stack. S6: When there is no data in the streaming message stack and the message end mark appears, the streaming message output ends; In step S2, the GPT server segments the generated text in a streaming output manner and pushes the texts to the front-end message component one by one through the GPT push method. The text segmentation includes: S21. Identify special characters and structures of markdown; S22, determine the segmentation point according to the special characters and structure of markdown; S23, segmenting the text according to the segmentation point and adding segmentation marks; S24, pushing the segmented text segments; After determining the segmentation points, the text generated by GPT is segmented into multiple segments. For each segment, when it contains the special characters or the beginning of the structure of markdown but not the end, a start segmentation marker is added at the end of the segment. When the segment contains the end but not the beginning, an end segmentation marker is added at the beginning of the segment. In step S4, the segments split in S2 are parsed before processing the data in the message stack: S41, checking whether the retrieved text segment contains a segmentation mark; S42, the text segment contains a start segmentation marker, and a new markdown object is created to indicate the start of the markdown object; S43, the text segment includes an end segmentation marker, searching and updating a corresponding markdown object to indicate the end of the markdown object; In step S4, the steps of processing the data in the message stack include: S44, checking whether the extracted word is a line break character, if so, clearing all attributes of the current markdown object and continuing to process the next word; S45. Check whether the extracted word is a special character of markdown: A. When the extracted word is a special character of markdown, further determine whether the character is a package character, and perform corresponding processing according to the current state of the markdown object: If the same package character already exists, delete the attribute of the character in the markdown object and update the display container; If there is no same wrapping character, create an attribute of the character type in the markdown object, and create a corresponding div element to insert into the display container; B. If the extracted word is not a special character of markdown, it is treated as a normal character and inserted according to the current state of the markdown object: When there is any attribute in the markdown object, insert the current normal character into the corresponding div element; When no attributes exist in the markdown object, ordinary characters are directly inserted into the display container.
2. The front-end rendering method based on streaming output according to claim 1, characterized in that: Recognize special characters and structures in markdown, including: titles, lists, code blocks, links, images, and emphasized text.
3. The front-end rendering method based on streaming output according to claim 1, characterized in that: The split points are set at locations that do not affect the markdown syntax and rendering effects, including spaces, punctuation marks, and line breaks.
4. The front-end rendering method based on streaming output according to claim 1, characterized in that: In step S4, the processing process of the streaming message stack is monitored and logged, and the log record includes: the processing time of the streaming message, the streaming message rendering time and the streaming message error information.
5. The front-end rendering method based on streaming output according to claim 1, characterized in that: Create a buffer queue as a storage space for temporary storage; when the speed at which GPT pushes streaming messages is greater than the processing speed of the message component, GPT pushes streaming messages to the buffer queue, and then the message component takes out and processes the data from the buffer queue.
6. A front-end rendering system based on streaming output, wherein the system is based on a front-end rendering method based on streaming output according to any one of claims 1 to 5 when running, characterized in that: include: The message component module is used to receive streaming messages pushed by GPT, is responsible for the interaction between the front end and the GPT server, and obtains the content generated by GPT in real time through API or WebSocket; The streaming message stack module is used to temporarily store the received streaming message data. As a temporary storage area, it can ensure the order and integrity of the messages and avoid message loss or confusion. The message end flag module is used to mark whether the streaming message has been completely received, and determines when to stop waiting for the arrival of new messages and start the final processing and rendering; The text segmentation module is used to segment the text generated by GPT. It first recognizes the special characters and structures of markdown, then determines the segmentation points, divides the text into multiple segments, and adds segmentation marks; The text parsing module is used to parse the text generated by GPT, parse the text fragments, and create the corresponding HTML structure according to the markdown syntax; The push method selection module is used to select the method of GPT push messages. According to the network environment and front-end requirements, WebSocket or HTTP long polling is selected to receive streaming messages from the GPT server. When the network environment is stable, WebSocket is preferred; when the network environment is unstable, a fallback mechanism is set to automatically switch to HTTP long polling; A buffer queue module is used as a storage space for temporarily storing streaming messages; The error handling module is used to handle network errors or data format errors that occur during streaming output; The rendering module is used to render the parsed markdown text to the front-end page and display the text content to the user according to the parsed HTML structure.