A data output method, device, equipment and storage medium based on large model

Abstract

This application discloses a method, apparatus, device, and storage medium for data output based on a large model, relating to the field of computer science. The method includes: establishing a target connection with a server using a preset streaming interface to receive real-time model output data of a target large model transmitted by the server in streaming data form; storing the real-time model output data in a preset buffer queue and monitoring in real-time whether the target connection is interrupted; if the target connection is not interrupted, determining whether the current queue length of the preset buffer queue is greater than a preset queue length threshold, and determining whether to adjust the data output rate based on the queue length determination result to obtain the current target data output rate; calling a target data output function and extracting the target model output data from the preset buffer queue based on the current target data output rate and outputting it. This application solves the problem of instability in data reception and display to ensure smooth data display.

Description

Technical Field

[0001] This invention relates to the field of computers, and in particular to a data output method, apparatus, device, and storage medium based on a large model. Background Technology

[0002] When processing the results of large backend model calls, if the frontend outputs the data obtained through the call in real-time response mode, it can provide a good experience when the model processing is fast. However, when the output of large models is slow or there are network communication delays, it can cause the frontend page rendering to lag. For example, the frontend page may lag after a sudden surge of text, followed by a large output, resulting in an unsmooth visual experience for the user. Traditional pagination or lazy loading methods fetch and output a fixed amount of data at a time, which can alleviate the frontend page rendering lag caused by changes in data volume in real-time response mode to some extent, but the effect is limited.

[0003] Therefore, how to solve the instability problem of data reception and display to ensure smooth data display is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a data output method, apparatus, device, and storage medium based on a large model, applied to the front end, which can solve the instability problem of data reception and display to ensure smooth data display. The specific solution is as follows:

[0005] Firstly, this application discloses a data output method based on a large model, applied to a front-end, including:

[0006] Establish a target connection with the server using a preset streaming interface, and receive the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection;

[0007] The real-time model output data is stored in a preset cache queue, and the target connection is monitored in real time for interruption.

[0008] If the target connection is not interrupted, it is determined whether the current queue length of the preset cache queue is greater than the preset queue length threshold, and based on the corresponding queue length determination result, it is determined whether to adjust the data output rate of the front end to obtain the current target data output rate.

[0009] The target data output function is called and the corresponding target model output data is extracted from the preset cache queue based on the current target data output rate, and the target model output data is output.

[0010] Optionally, before receiving the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection, the method further includes:

[0011] Determine whether the preset cache queue contains model output data obtained through the previous connection;

[0012] If the preset cache queue contains model output data obtained through the previous connection, then the target response interrupt function is used to clear all model output data in the preset cache queue to interrupt the output.

[0013] Optionally, storing the real-time model output data in a preset cache queue includes:

[0014] The real-time model output data is parsed into corresponding parsed data based on the target form using the target event processing function, and the parsed data is stored in the preset cache queue.

[0015] Optionally, before parsing the real-time model output data into corresponding parsed data based on the target form using the target event processing function, the method further includes:

[0016] Check whether the target parameters of the target connection are preset parameter values;

[0017] If the target parameter is a preset parameter value, then clear all data in the current preset cache queue and check whether there are any errors in the real-time model output data;

[0018] If the real-time model output data contains errors, then the real-time model output data is discarded.

[0019] Optionally, the data output method based on the large model further includes:

[0020] If the target connection is interrupted, or a preset cleanup instruction is received, the target cleanup function is called to clear all data in the current preset cache queue, thereby terminating the action of outputting the target model output data.

[0021] Optionally, the data output method based on the large model further includes:

[0022] Obtain the target data output rate adjustment instruction, and adjust the current target data output rate accordingly based on the target data output rate adjustment instruction to obtain a new current target data output rate.

[0023] Optionally, determining whether to adjust the data output rate of the front end based on the corresponding queue length judgment result includes:

[0024] If the current queue length of the preset cache queue is greater than or equal to the preset queue length threshold, then the data output rate is increased based on the target time, which is the time required to output all model output data in the preset cache queue, determined based on the data output rate and the queue length.

[0025] Secondly, this application discloses a data output device based on a large model, applied to a front end, comprising:

[0026] The data receiving module is used to establish a target connection with the server using a preset streaming interface, and to receive the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection.

[0027] The connection monitoring module is used to store the real-time model output data in a preset cache queue and monitor in real time whether the target connection is interrupted.

[0028] The first output rate adjustment module is used to determine whether the current queue length of the preset buffer queue is greater than the preset queue length threshold if the target connection is not interrupted, and to determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result, so as to obtain the current target data output rate.

[0029] The target data output module is used to call the target data output function and extract the corresponding target model output data from the preset cache queue based on the current target data output rate, and output the target model output data.

[0030] Thirdly, this application discloses an electronic device, including:

[0031] Memory, used to store computer programs;

[0032] A processor is used to execute the computer program to implement the aforementioned data output method based on a large model.

[0033] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned data output method based on a large model.

[0034] In this application, the front-end establishes a target connection with the server using a preset streaming interface, and receives real-time model output data of the target large model transmitted by the server in streaming data form through the preset streaming interface via the target connection; the real-time model output data is stored in a preset cache queue, and the front-end monitors in real time whether the target connection is interrupted; if the target connection is not interrupted, it is determined whether the current queue length of the preset cache queue is greater than a preset queue length threshold, and based on the corresponding queue length determination result, it is determined whether to adjust the data output rate of the front-end to obtain the current target data output rate; the target data output function is called and the corresponding target model output data is extracted from the preset cache queue based on the current target data output rate, and the target model output data is output. As can be seen, this application controls the output of real-time model output data obtained from the target large model by introducing a caching mechanism and a preset cache queue. The front-end application no longer directly displays the data obtained from the back-end large model in real time, but temporarily stores the real-time model output data in the preset cache queue. The front-end can dynamically adjust the data output rate according to the queue length of the preset cache queue, thereby extracting the corresponding target model output data from the preset cache queue based on the current target data output rate for output. This solves the problem of instability in data reception and display, and ensures that the data is smoothly displayed by the front-end. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0036] Figure 1 This is a flowchart of a data output method based on a large model disclosed in this application;

[0037] Figure 2 This is a schematic diagram of a system architecture disclosed in this application;

[0038] Figure 3 This application discloses a flowchart of a streaming data processing procedure.

[0039] Figure 4 This is a schematic diagram of a user-adjustable streaming data output rate process disclosed in this application.

[0040] Figure 5 This is a schematic diagram of a data output device based on a large model disclosed in this application;

[0041] Figure 6This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] When processing the results of large backend model calls, if the frontend outputs the data obtained through the call in real-time response mode, it can provide a good experience when the model processing is fast. However, when the output of large models is slow or there are network communication delays, it can cause frontend page rendering stutters. For example, the frontend page may stutter after a sudden surge of text, followed by a large output, resulting in an unsmooth visual experience for the user. Traditional pagination or lazy loading methods only fetch and output a fixed amount of data at a time, which can alleviate the frontend page rendering stutters caused by changes in data volume in real-time response mode to some extent, but the effect is limited. To solve the above technical problems, this application discloses a data output method based on large models, which can solve the instability problem of data reception and display, and ensure smooth data display.

[0044] See Figure 1 As shown, this embodiment of the invention discloses a data output method based on a large model, applied to a front end, including:

[0045] Step S11: Establish a target connection with the server using a preset streaming interface, and receive the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection.

[0046] like Figure 2As shown, in this embodiment, after establishing a target connection with the server using a preset streaming interface, the real-time model output data of the target large model transmitted by the server in streaming data form can be received through the target connection using the preset streaming interface, thus realizing the streaming data reception function. The preset streaming interface can be an EventSource API (EventSource Application Programming Interface, a lightweight application programming interface) or a WebSocket API (WebSocket Application Programming Interface, an application programming interface for establishing bidirectional communication connections between clients and servers). It is understood that in this embodiment, the target large model can be a large model located on the server, or a large model that only forwards data through the server. To avoid outputting model output data obtained through the previous connection as model output data obtained in the current target connection, this embodiment further includes, before receiving the real-time model output data: determining whether model output data obtained through the previous connection exists in the preset cache queue; if model output data obtained through the previous connection exists in the preset cache queue, then using the target response interrupt function to clear all model output data in the preset cache queue to interrupt the output.

[0047] In this embodiment, the interface used when creating a connection can be `fetchEventSource`, and the parameter `signal: controller.value.signal` can be added to monitor the connection status, where `controller.value = newAbortController()`. If concurrent events need to be handled, that is, if it is found that the previous connection has not been closed when creating the current target connection, `controller.value.abort()` can be used to cancel the response to the previous connection, achieving the effect of termination. The `EventSource` object provides several event handlers to monitor the connection status, such as `open`, `error`, and `close`. These event handlers are added to handle connection opening, error occurrence, and connection closing, such as triggering `close` when events are concurrent, triggering `open` each time a connection is successfully established, and triggering `onresult` each time data is received normally.

[0048] In this embodiment, the preset cache queue is a pre-initialized cache queue. Specifically, this cache queue can be a variable named `analysisSuggestionsBuffer`, an array type defined in the local scope using the `let` keyword. This allows the cache queue to implement FIFO (First Input First Output) characteristics using convenient methods such as `push` (adding an element to the end of the array) and `shift` (removing an element from the beginning of the array) to store subsequent real-time model output data received from the server. Furthermore, because the `let` keyword is used when defining the cache queue, the variable in the local scope will be automatically destroyed when the request is canceled or the page is closed, thus releasing the previously occupied memory space. This is particularly important for long-running pages or applications, preventing performance degradation or crashes caused by memory leaks.

[0049] Step S12: Save the real-time model output data in a preset cache queue, and monitor in real time whether the target connection is interrupted.

[0050] like Figure 3 As shown in this embodiment, the specific process of storing real-time model output data in a preset cache queue may include: using a target event handling function to parse the real-time model output data into corresponding parsed data based on the target format, and storing the parsed data in the preset cache queue. Before using the target event handling function to parse the real-time model output data into corresponding parsed data based on the target format, it is also necessary to check whether the target parameter of the target connection is a preset parameter value; if the target parameter is a preset parameter value, then all data in the current preset cache queue is cleared, and the real-time model output data is checked for errors; if the real-time model output data has errors, then the real-time model output data is discarded.

[0051] In one specific implementation, a message event (or other custom event) can be listened to, and the real-time model output data can be parsed into corresponding parsed data in the target event handling function. This parsed data is in target format, that is, data suitable for storage in a preset cache queue. Before storing the real-time model output data into the preset cache queue, it is necessary to check whether the controller.value.signal.aborted state of the target connection is true. If it is true, all values ​​in the preset cache queue are cleared in time to achieve the stopping effect. At the same time, it is also necessary to check whether there are errors in the real-time model output data. If there are errors, they are thrown in time. When the close signal is detected, the target connection can be closed using eventSource.close() to terminate the function. For normal real-time model output data, the push method is used to store the parsed data corresponding to these real-time model output data into the analysisSuggestionsBuffer that was created at the beginning. During this period, the data in the preset cache queue can be processed to obtain the data required by the front end. In addition, if the amount of data in the preset cache queue continues to grow without effective processing, it may lead to memory leaks or performance degradation. Therefore, it is necessary to judge in real time whether the length of analysisSuggestionsBuffer is too long to decide whether to clean up the data. By introducing a caching queue mechanism, the front-end application can effectively handle large amounts of data output from large models, avoiding performance bottlenecks caused by receiving large amounts of data in a short period of time. This not only reduces the computational burden on the front-end but also improves data processing efficiency and response speed, ensuring smooth application operation.

[0052] Step S13: If the target connection is not interrupted, determine whether the current queue length of the preset cache queue is greater than the preset queue length threshold, and determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result, so as to obtain the current target data output rate.

[0053] In this embodiment, after the real-time model output data is received and stored in the preset buffer queue `analysisSuggestionsBuffer`, it can be output smoothly using a timer. Specifically, a timer is set using the `setInterval` or `setTimeout` function, which triggers a callback function at specified time intervals of 50ms (or other values). In this callback function, data is retrieved from the `analysisSuggestionsBuffer` and output. Each time data is output, it is necessary to determine whether the target connection of the streaming interface has not been terminated and whether the current `analysisSuggestionsBuffer` still contains data. If the target connection has not been terminated and the current `analysisSuggestionsBuffer` still contains data, the timer is triggered.

[0054] Understandably, if the receiving rate of real-time model output data is much higher than the front-end data output rate, the length of the preset buffer queue may grow rapidly and consume a large amount of memory. To avoid this, consider pausing the reception of new real-time model output data when the queue length reaches a preset queue length threshold, or increasing the data output rate. Since the back-end output cannot be interrupted by the front-end, only increasing the data output rate is considered. Simultaneously, stop the timer when the preset buffer queue is empty to avoid unnecessary resource waste.

[0055] In one specific implementation, if the target connection is not interrupted, it is determined whether the current preset cache queue length is greater than a preset queue length threshold. Based on the queue length determination result, it is determined whether to adjust the front-end data output rate to obtain the current target data output rate. If the target connection is interrupted, or a preset cleanup instruction is received, the target cleanup function is called to clear all data in the current preset cache queue, thereby terminating the output of target model output data. If the target connection is not interrupted, and the current preset cache queue length is less than the preset queue length threshold but greater than 0, the front-end data output rate is not adjusted. When determining whether to adjust the front-end data output rate based on the corresponding queue length determination result, if the current preset cache queue length is greater than or equal to the preset queue length threshold, the data output rate is increased based on a target time, which is the time required to output all model output data in the preset cache queue, determined based on the front-end data output rate and the queue length.

[0056] Specifically, if the preset cache queue can store 1000 model output data entries, and a timer is triggered every 50ms, the current data output rate is 200 entries / second. This means that if the target connection is not interrupted and the preset cache queue has sufficient data, each timer trigger will output 10 model output data entries. If the target connection is not interrupted and the number of model output data entries in the preset cache queue is less than 10, then all model output data entries in the preset cache queue will be output. When the preset cache queue length reaches the preset queue length threshold (e.g., the preset queue length threshold is 600 entries, and the current preset cache queue length is 800 entries), the target time is 4 seconds. If the target time is set to be greater than 3 seconds, the front-end data output rate will be increased to obtain the current target data output rate, meaning the current target data output rate should be no less than 267 entries / second.

[0057] Furthermore, considering the output rate of the target large model's data, the front-end data output rate can be adjusted based on the target large model's data output rate, the front-end data output rate, and the queue length. For example, if the model data output rate is 300 records / second, and other parameters remain unchanged, the current target time is 4 seconds. If the data output rate is not adjusted, after all model output data in the current preset cache queue has been output, i.e., after 4 seconds, the preset cache queue will contain 1200 model output data records, exceeding the storage capacity of the preset cache queue. To solve this problem, an adjustable queue length threshold can be introduced. In this case, the adjusted current target data output rate can be determined based on the target queue length function L. The expression for the target queue length function L is as follows:

[0058] ;

[0059] in, For the target large model's model data output rate, To adjust the pre-set queue length of the cache queue, The adjusted current target data output rate, The target time. That is to say, when... Output the model output data in the current preset cache queue. After a few seconds, the preset cache queue length is L. Setting L not to exceed the adjustable queue length threshold allows us to obtain the adjusted range of the current target data output rate. The upper limit of the adjustable queue length threshold can be set to 1000, and the lower limit can be set to 0. This ensures that the preset cache queue always stores model output data, and the amount of model output data will not exceed the maximum capacity of the preset cache queue. This threshold range can be adjusted according to user needs. For example, if the adjustable queue length threshold is set to 700, then based on L, we can determine "800 + 300 * 4 - *4≤700”, thus obtaining ≥325. That is to say, if the data output rate is adjusted to 325 data points / s, the number of model output data points in the preset cache queue will be equal to 700 data points after 4 seconds.

[0060] In this embodiment, by employing a strategy of dynamically adjusting the data output speed, the data display becomes smoother and more continuous, avoiding the discomfort caused by sudden changes in data output speed. Users can enjoy a more seamless information flow, greatly enhancing the user experience.

[0061] Step S14: Call the target data output function and extract the corresponding target model output data from the preset cache queue based on the current target data output rate, and output the target model output data.

[0062] In this embodiment, target model data is retrieved from a preset cache queue and output by calling the target data output function. The number of target model data retrieved each time is related to the current target data output rate. Specifically, if a timer is triggered every 50ms and the current data output rate is 200 data entries / second, that is, if the target connection is not interrupted and the amount of data in the preset cache queue is sufficient, 10 model output data entries will be output each time the timer is triggered; if the target connection is not interrupted and the number of model output data entries in the preset cache queue is less than 10, then all model output data entries in the preset cache queue will be output.

[0063] In this embodiment, see Figure 4As shown, to enhance the user experience when using the streaming data output function, user interaction feedback can be introduced to obtain the target data output rate adjustment command. Based on this command, the current target data output rate can be adjusted accordingly to obtain a new target data output rate. In other words, an intuitive interface control (such as a slider or button group) can be designed in the interactive interface, allowing users to manually adjust the data output speed and stop output. By listening to user interaction events, the target data output rate adjustment command is obtained, and a global variable (such as `userOutputSpeed`) is updated to store the new speed value, such as 500 data entries / second. In one specific implementation, a new timer trigger time is determined based on this global variable, but the amount of data output by each timer is not adjusted. For example, before the adjustment, the timer triggers once every 50ms, 20 times per second, outputting 10 data entries each time, with a data output rate of 200 data entries / second; after the user adjustment, the timer triggers once every 20ms, 50 times per second, outputting 10 data entries each time, resulting in a current target data output rate of 500 data entries / second. In another specific implementation, the number of data points output by each timer is adjusted based on this global variable, but the timer trigger time remains unchanged. For example, after adjustment, the timer triggers once every 50ms, triggering 20 times per second and outputting 50 data points each time. The adjusted target data output rate is then 500 data points / s. Alternatively, both the timer trigger time and the number of data points output by each timer can be adjusted simultaneously. For instance, adjusting the timer trigger time to 40ms, triggering 25 times per second, and outputting 20 data points each time, still results in a target data output rate of 500 data points / s.

[0064] It is understandable that various abnormal situations may be encountered during the data output process, such as network errors and data format errors. In this embodiment, appropriate exception handling logic can be added to ensure application stability and user experience: for example, the `onClose` and `onErrorr` methods of the `fetchEventSource` function correspond to the handling logic for function termination and exception catching, respectively. The `controller.value.abort()` method stops the API call from triggering the `fetchEventSource`'s `onClose` function, thus stopping the timer or setting the API abort value to `true`. Under normal circumstances, when data output is no longer needed (e.g., when the page is closed or the user cancels the operation), the `clearInterval` function can be called to stop the timer and clean up other related resources, ensuring the cleanup and destruction of timer-related resources to avoid memory leaks or other potential problems.

[0065] As can be seen, this application controls the output of real-time model output data obtained from the target large model by introducing a caching mechanism and a preset cache queue. The front-end application no longer directly displays the data obtained from the back-end large model in real time; instead, it temporarily stores the real-time model output data in the preset cache queue. The front-end can dynamically adjust the data output rate according to the queue length of the preset cache queue, thereby retrieving the corresponding target model output data from the preset cache queue based on the current target data output rate for output. This solves the instability problem of data reception and display, ensuring smooth data display by the front-end. Furthermore, the introduction of the cache queue and queue control mechanism reduces the front-end application's direct dependence on the back-end model processing speed, lowering the complexity of system maintenance. Simultaneously, automated data management and exception handling logic simplify the work of operations and maintenance personnel, reducing long-term maintenance costs.

[0066] See Figure 5 As shown, this application discloses a data output device based on a large model, applied to a front end, comprising:

[0067] Data receiving module 11 is used to establish a target connection with the server using a preset streaming interface, and to receive the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection.

[0068] The connection monitoring module 12 is used to store the real-time model output data in a preset cache queue and monitor whether the target connection is interrupted in real time.

[0069] The first output rate adjustment module 13 is used to determine whether the current queue length of the preset buffer queue is greater than the preset queue length threshold if the target connection is not interrupted, and to determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result, so as to obtain the current target data output rate.

[0070] The target data output module 14 is used to call the target data output function and extract the corresponding target model output data from the preset cache queue based on the current target data output rate, and output the target model output data.

[0071] As can be seen, this application controls the output of real-time model output data obtained from the target large model by introducing a caching mechanism and a preset cache queue. The front-end application no longer directly displays the data obtained from the back-end large model in real time, but temporarily stores the real-time model output data in the preset cache queue. The front-end can dynamically adjust the data output rate according to the queue length of the preset cache queue, thereby extracting the corresponding target model output data from the preset cache queue based on the current target data output rate for output. This solves the problem of instability in data reception and display, and ensures that the data is smoothly displayed by the front-end.

[0072] Optionally, the device may further include:

[0073] The condition judgment module is used to determine whether there is model output data obtained through the previous connection in the preset cache queue;

[0074] The data clearing unit is used to clear all model output data in the preset cache queue to interrupt output if model output data obtained through the previous connection exists in the preset cache queue using the target response interrupt function.

[0075] Optionally, the connection monitoring module 12 may specifically include:

[0076] The parsing data storage unit is used to parse the real-time model output data into corresponding parsing data based on the target form using the target event processing function, and to store the parsing data in the preset cache queue.

[0077] Optionally, the device may further include:

[0078] The target parameter checking module is used to check whether the target parameters of the target connection are preset parameter values;

[0079] The data inspection module is used to clear all data in the current preset cache queue and check whether there are errors in the real-time model output data if the target parameter is a preset parameter value.

[0080] The data discarding module is used to discard the real-time model output data if there is an error in the output data.

[0081] Optionally, the device may further include:

[0082] The output termination module is used to call the target cleanup function to clear all data in the current preset cache queue if the target connection is interrupted or a preset cleanup instruction is obtained, so as to terminate the action of outputting the target model output data.

[0083] Optionally, the device may further include:

[0084] The second output rate adjustment module is used to acquire the target data output rate adjustment instruction and adjust the current target data output rate accordingly based on the target data output rate adjustment instruction to obtain a new current target data output rate.

[0085] Optionally, the first output rate adjustment module 13 may specifically include:

[0086] The output rate increasing unit is used to increase the data output rate based on a target time if the current queue length of the preset cache queue is greater than or equal to a preset queue length threshold. The target time is the time required to output all model output data in the preset cache queue, determined based on the data output rate and the queue length.

[0087] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0088] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the large-model-based data output method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0089] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0090] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include an operating system 221, computer programs 222, etc., and the storage method can be temporary storage or permanent storage.

[0091] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the large-model-based data output method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0092] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned data output method based on a large model. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0093] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0094] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0095] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0096] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0097] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data output method based on a large model, characterized in that, Applied to front-end, including: Establish a target connection with the server using a preset streaming interface, and receive the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection; The real-time model output data is stored in a preset cache queue, and the target connection is monitored in real time for interruption. If the target connection is not interrupted, it is determined whether the current queue length of the preset cache queue is greater than the preset queue length threshold, and based on the corresponding queue length determination result, it is determined whether to adjust the data output rate of the front end to obtain the current target data output rate. The target data output function is called and the corresponding target model output data is extracted from the preset cache queue based on the current target data output rate, and the target model output data is output. The step of determining whether to adjust the data output rate of the front end based on the corresponding queue length judgment result includes: If the current queue length of the preset cache queue is greater than or equal to the preset queue length threshold, the data output rate is increased based on the target time, where the target time is the time required to output all model output data in the preset cache queue, determined based on the data output rate and the queue length.

2. The data output method based on a large model according to claim 1, characterized in that, Before receiving the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection, the method further includes: Determine whether the preset cache queue contains model output data obtained through the previous connection; If the preset cache queue contains model output data obtained through the previous connection, then the target response interrupt function is used to clear all model output data in the preset cache queue to interrupt the output.

3. The data output method based on a large model according to claim 1, characterized in that, The step of storing the real-time model output data in a preset cache queue includes: The real-time model output data is parsed into corresponding parsed data based on the target form using the target event processing function, and the parsed data is stored in the preset cache queue.

4. The data output method based on a large model according to claim 3, characterized in that, Before using the target event processing function to parse the real-time model output data into corresponding parsed data based on the target form, the method further includes: Check whether the target parameters of the target connection are preset parameter values; If the target parameter is a preset parameter value, then clear all data in the current preset cache queue and check whether there are any errors in the real-time model output data; If the real-time model output data contains errors, then the real-time model output data is discarded.

5. The data output method based on a large model according to claim 1, characterized in that, Also includes: If the target connection is interrupted, or a preset cleanup instruction is received, the target cleanup function is called to clear all data in the current preset cache queue, thereby terminating the action of outputting the target model output data.

6. The data output method based on a large model according to claim 1, characterized in that, Also includes: Obtain the target data output rate adjustment instruction, and adjust the current target data output rate accordingly based on the target data output rate adjustment instruction to obtain a new current target data output rate.

7. A data output device based on a large model, characterized in that, Applied to front-end, including: The data receiving module is used to establish a target connection with the server using a preset streaming interface, and to receive the real-time model output data of the target large model transmitted by the server in the form of streaming data through the preset streaming interface via the target connection. The connection monitoring module is used to store the real-time model output data in a preset cache queue and monitor in real time whether the target connection is interrupted. The first output rate adjustment module is used to determine whether the current queue length of the preset buffer queue is greater than the preset queue length threshold if the target connection is not interrupted, and to determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result, so as to obtain the current target data output rate. The target data output module is used to call the target data output function and extract the corresponding target model output data from the preset cache queue based on the current target data output rate, and output the target model output data. The first output rate adjustment module includes: The output rate increasing unit is used to increase the data output rate based on a target time if the current queue length of the preset cache queue is greater than or equal to a preset queue length threshold. The target time is the time required to output all model output data in the preset cache queue, determined based on the data output rate and the queue length.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the large model-based data output method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the data output method based on a large model as described in any one of claims 1 to 6.

Images