Data output method, device and equipment based on large model and storage medium

Abstract

The invention discloses a data output method and device based on a large model, equipment and a storage medium, and relates to the field of computers, and the method comprises the steps: building a target connection with a server through a preset streaming interface, and receiving real-time model output data, transmitted by the server in a streaming data form, of a target large model through the preset streaming interface; storing the real-time model output data in a preset cache queue, and monitoring whether the target connection is interrupted or not in real time; if it is monitored that the target connection is not interrupted, whether the queue length of the current preset cache queue is larger than a preset queue length threshold value or not is judged, and whether the data output rate is adjusted or not is determined based on a corresponding queue length judgment result, so that the current target data output rate is obtained; and calling the target data output function, extracting target model output data from the preset cache queue based on the current target data output rate, and outputting the target model output data. The problem of instability of data receiving and display is solved so as to ensure smooth display of the data.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a data output method, device, equipment and storage medium based on a large model. Background Art

[0002] When the front-end processes the call results of the large model in the back-end, if the data obtained through the call is output in real-time response mode, it can provide a good experience when the model is processed quickly, but when the large model is output slowly or there is a delay in network communication, it will cause the front-end page rendering to freeze. For example, the front-end page may freeze after a sudden influx of text, and then output a large amount of text, giving users an unsmooth visual experience. The traditional paging loading or delayed loading method only obtains and outputs fixed data each time, which can alleviate the front-end page rendering freeze caused by changes in data volume in real-time response mode to a certain extent, but the effect is limited.

[0003] Therefore, how to solve the instability problem of data reception and display to ensure smooth data display is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a data output method, device, equipment and storage medium based on a large model, which is applied to the front end and can solve the instability problem of data reception and display to ensure smooth data display. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a data output method based on a large model, which is applied to a front end and includes:

[0006] Establishing a target connection with a server using a preset streaming interface, and receiving real-time model output data of a target large model transmitted by the server in the form of streaming data using the preset streaming interface through the target connection;

[0007] The real-time model output data is stored in a preset cache queue, and the target connection is monitored in real time to determine whether it is interrupted;

[0008] If it is monitored that the target connection is not interrupted, determine whether the queue length of the current preset cache queue is greater than the preset queue length threshold, and determine whether to adjust the data output rate of the front end based on the corresponding queue length determination result to obtain the current target data output rate;

[0009] The target data output function is called and corresponding target model output data is extracted from the preset cache queue based on the current target data output rate, and the target model output data is output.

[0010] Optionally, before receiving the real-time model output data of the target large model transmitted by the server in the form of streaming data through the target connection using the preset streaming interface, the method further includes:

[0011] Determine whether the preset cache queue contains model output data obtained through the last connection;

[0012] If the preset cache queue contains model output data acquired through the last connection, the target response interrupt function is used to clear all model output data in the preset cache queue to interrupt the output.

[0013] Optionally, the storing the real-time model output data in a preset cache queue includes:

[0014] The real-time model output data is parsed into corresponding parsed data based on the target form by using the target event processing function, and the parsed data is stored in the preset cache queue.

[0015] Optionally, before parsing the real-time model output data into corresponding parsed data based on the target form using the target event processing function, the method further includes:

[0016] Checking whether a target parameter of the target connection is a preset parameter value;

[0017] If the target parameter is a preset parameter value, all data in the current preset cache queue is cleared, and the real-time model output data is checked for errors;

[0018] If there is an error in the real-time model output data, the real-time model output data is discarded.

[0019] Optionally, the data output method based on the large model further includes:

[0020] If the target connection is interrupted, or a preset cleanup instruction is obtained, a target cleanup function is called to clear all data in the current preset cache queue to terminate the action of outputting the target model output data.

[0021] Optionally, the data output method based on the large model further includes:

[0022] A target data output rate adjustment instruction is obtained, and based on the target data output rate adjustment instruction, the current target data output rate is adjusted accordingly to obtain a new current target data output rate.

[0023] Optionally, the determining whether to adjust the data output rate of the front end based on the corresponding queue length judgment result includes:

[0024] If the current queue length of the preset cache queue is greater than or equal to a preset queue length threshold, the data output rate is increased based on a target time, where the target time is the time required to output all model output data in the preset cache queue determined based on the data output rate and the queue length.

[0025] In a second aspect, the present application discloses a data output device based on a large model, which is applied to a front end and includes:

[0026] A data receiving module, used to establish a target connection with a server using a preset streaming interface, and receive real-time model output data of a target large model transmitted by the server in the form of streaming data through the target connection using the preset streaming interface;

[0027] A connection monitoring module, used to store the real-time model output data in a preset cache queue and monitor in real time whether the target connection is interrupted;

[0028] A first output rate adjustment module is used to determine whether the queue length of the preset cache queue is greater than a preset queue length threshold if it is detected that the target connection is not interrupted, and determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result to obtain the current target data output rate;

[0029] The target data output module is used to call the target data output function and extract the corresponding target model output data from the preset cache queue based on the current target data output rate, and output the target model output data.

[0030] In a third aspect, the present application discloses an electronic device, comprising:

[0031] Memory, used to store computer programs;

[0032] A processor is used to execute the computer program to implement the aforementioned data output method based on the large model.

[0033] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned large model-based data output method.

[0034] In the present application, the front end establishes a target connection with the server using a preset streaming interface, and receives the real-time model output data of the target large model transmitted by the server in the form of streaming data using the preset streaming interface through the target connection; the real-time model output data is saved in a preset cache queue, and the target connection is monitored in real time to see if it is interrupted; if it is detected that the target connection is not interrupted, it is determined whether the queue length of the current preset cache queue is greater than a preset queue length threshold, and based on the corresponding queue length judgment result, it is determined whether to adjust the data output rate of the front end to obtain the current target data output rate; the target data output function is called and the corresponding target model output data is extracted from the preset cache queue based on the current target data output rate, and the target model output data is output. It can be seen that the present application controls the output of the real-time model output data obtained from the target large model by introducing a cache mechanism and a preset cache queue. The front-end application no longer directly displays the data obtained by calling the back-end large model in real time, but temporarily stores the real-time model output data in the preset cache queue. The front-end can dynamically adjust the data output rate according to the queue length of the preset cache queue, thereby extracting the corresponding target model output data from the preset cache queue based on the current target data output rate for output, thereby solving the instability problem of data reception and display, and ensuring that the data is displayed smoothly by the front-end. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 A flow chart of a data output method based on a large model disclosed in this application;

[0037] Figure 2 A schematic diagram of a system architecture disclosed in this application;

[0038] Figure 3 A flow chart of streaming data processing disclosed in this application;

[0039] Figure 4 A schematic diagram of a process for adjusting the output rate of streaming data by a user disclosed in the present application;

[0040] Figure 5 This is a schematic diagram of the structure of a data output device based on a large model disclosed in this application;

[0041] Figure 6This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] When the front-end processes the call results of the large model on the back-end, if the data obtained through the call is output in real-time response mode, although it can provide a good experience when the model is processed quickly, when the large model outputs slowly or there is a delay in network communication, it will cause the front-end page rendering to freeze. For example, the front-end page may freeze after a sudden influx of text, and then output a large amount of text, giving the user an unsmooth visual experience. The traditional paging loading or delayed loading method only obtains and outputs fixed data each time, which can alleviate the front-end page rendering freeze caused by changes in data volume in real-time response mode to a certain extent, but the effect is limited. In order to solve the above technical problems, the present application discloses a data output method based on a large model, which can solve the instability problem of data reception and display and ensure smooth data display.

[0044] See also Figure 1 As shown, the embodiment of the present invention discloses a data output method based on a large model, which is applied to a front end and includes:

[0045] Step S11, establishing a target connection with a server using a preset streaming interface, and receiving real-time model output data of a target large model transmitted by the server in the form of streaming data using the preset streaming interface through the target connection.

[0046] like Figure 2As shown, in this embodiment, after establishing a target connection with the server using a preset streaming interface, the real-time model output data of the target large model transmitted by the server in the form of streaming data can be received through the target connection using the preset streaming interface, that is, the streaming data receiving function is realized. The preset streaming interface can be EventSource API (EventSource Application Programming Interface, a lightweight application programming interface) or WebSocket API (WebSocket Application Programming Interface, an application programming interface for establishing a two-way communication connection between a client and a server). It can be understood that in this embodiment, the target large model can be a large model located on the server or a large model that only forwards data through the server. In order to avoid outputting the model output data obtained through the last connection as the model output data obtained by the current target connection, in this embodiment, before receiving the real-time model output data, it also includes: determining whether there is model output data obtained through the last connection in the preset cache queue, and if there is model output data obtained through the last connection in the preset cache queue, then using the target response interrupt function to clear all model output data in the preset cache queue to interrupt the output.

[0047] In this embodiment, the interface used when creating a connection can be fetchEventSource, and the parameter signal: controller.value.signal can be added to monitor the connection status, where controller.value=newAbortController(). If it is necessary to process concurrent events, that is, when the current target connection is created and it is found that the previous connection is still not disconnected, controller.value.abort() can be used to cancel the response to the previous connection to achieve the end effect. The EventSource object provides multiple event handlers to monitor the status of the connection, such as open, error, and close. These event handlers are added to handle situations such as connection opening, error occurrence, and connection closing, such as triggering close when events are concurrent, triggering open each time a connection is successfully established, and triggering onresult each time data is received normally.

[0048] In this embodiment, the preset cache queue is a pre-initialized cache queue. Specifically, the cache queue can be a variable named analysisSuggestionsBuffer of an array type defined in a local scope using the let keyword, so that the cache queue can implement the FIFO (First Input First Output) feature of the queue by using convenient methods such as push (adding elements to the end of the array) and shift (removing elements from the beginning of the array) to store the real-time model output data subsequently received from the server. At the same time, since the let keyword is used when defining the cache queue, when the request is canceled or the page is closed, the variables in the local scope will be automatically destroyed, thereby releasing the memory space previously occupied. This is especially important for pages or applications that run for a long time, and can avoid performance degradation or crashes caused by memory leaks.

[0049] Step S12: Save the real-time model output data in a preset cache queue, and monitor in real time whether the target connection is interrupted.

[0050] like Figure 3 As shown, in this embodiment, the specific process of saving the real-time model output data in the preset cache queue may include: using the target event processing function to parse the real-time model output data into corresponding parsed data based on the target form, and saving the parsed data in the preset cache queue. Before using the target event processing function to parse the real-time model output data into corresponding parsed data based on the target form, it is also necessary to check whether the target parameter of the target connection is a preset parameter value; if the target parameter is a preset parameter value, all data in the current preset cache queue is cleared, and the real-time model output data is checked for errors; if the real-time model output data has errors, the real-time model output data is discarded.

[0051] In a specific implementation, the message event (or other custom event) can be monitored, and the real-time model output data can be parsed into corresponding parsed data in the target event processing function. These parsed data are all data in the target form, that is, data suitable for storage in the preset cache queue. Each time before storing the real-time model output data in the preset cache queue, it is necessary to first check whether the state of the controller.value.signal.aborted of the target connection is true. If it is true, all values ​​in the preset cache queue are cleared in time to achieve the effect of stopping. At the same time, it is also necessary to check whether there is an error in the real-time model output data. If there is an error, it is thrown in time. When the close end signal is detected, eventSource.close() can be used to close the target connection to achieve the end of the function. For normal real-time model output data, the push method is used to store the parsed data corresponding to these real-time model output data in the analysisSuggestionsBuffer established at the beginning. During this period, the data in the preset cache queue can be processed to obtain the data required by the front end. In addition, if the amount of data in the preset cache queue continues to grow without being effectively processed, it may cause memory leaks or performance degradation. Therefore, it is necessary to determine in real time whether the length of analysisSuggestionsBuffer is too long to decide whether to clean up the data. By introducing a cache queue mechanism, front-end applications can effectively process large amounts of data output by large models, avoiding performance bottlenecks caused by receiving large amounts of data in a short period of time. This not only reduces the computing burden on the front end, but also improves the efficiency and response speed of data processing, ensuring smooth operation of the application.

[0052] Step S13: If it is monitored that the target connection is not interrupted, determine whether the queue length of the current preset cache queue is greater than the preset queue length threshold, and determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result to obtain the current target data output rate.

[0053] In this embodiment, after the real-time model output data is received and stored in the preset cache queue analysisSuggestionsBuffer, you can choose to use a timer to smoothly output this data. Specifically, use the setInterval or setTimeout function to set a timer, which will trigger a callback function at a specified time interval of 50ms (or other values). In the callback function, data is taken from the analysisSuggestionsBuffer and output. Each time the output is performed, it is necessary to determine whether the target connection of the streaming interface has not been terminated and whether there is still data in the current analysisSuggestionsBuffer. If the target connection has not been terminated and there is still data in the current analysisSuggestionsBuffer, the timer is triggered.

[0054] It is understandable that if the receiving rate of the real-time model output data is much greater than the data output rate of the front end, the queue length of the preset cache queue may grow rapidly and consume a lot of memory. To avoid this situation, consider suspending the reception of new real-time model output data when the queue length reaches the preset queue length threshold, or increasing the data output rate. Since the back-end output cannot be interrupted by the front end, only increasing the data output rate is considered. At the same time, the timer is stopped when the preset cache queue is empty to avoid unnecessary waste of resources.

[0055] In a specific implementation, if it is monitored that the target connection is not interrupted, it is determined whether the queue length of the current preset cache queue is greater than the preset queue length threshold, and based on the corresponding queue length judgment result, it is determined whether to adjust the data output rate of the front end to obtain the current target data output rate. If the target connection is interrupted, or a preset cleanup instruction is obtained, the target cleanup function is called to clear all data in the current preset cache queue to terminate the action of outputting the target model output data. If the target connection is not interrupted, when the queue length of the current preset cache queue is less than the preset queue length threshold and greater than 0, the data output rate of the front end is not adjusted. When determining whether to adjust the data output rate of the front end based on the corresponding queue length judgment result, if the queue length of the current preset cache queue is greater than or equal to the preset queue length threshold, the data output rate is increased based on the target time, and the target time is the time required to output all model output data in the preset cache queue determined based on the data output rate and queue length of the front end.

[0056] Specifically, if the preset cache queue can store 1,000 model output data, the timer is triggered every 50 ms, and the current data output rate is 200 / s, that is, when the target connection is not interrupted and the amount of data in the preset cache queue is sufficient, 10 model output data will be output each time the timer is triggered; if the target connection is not interrupted and the number of model output data in the preset cache queue is less than 10, all model output data in the preset cache queue will be output. When it is monitored that the queue length of the preset cache queue reaches the preset queue length threshold, such as the preset queue length threshold is 600, and the current queue length of the preset cache queue is 800, the target time is 4 seconds. If the target time is set to be greater than 3 seconds, the data output rate of the front end is increased to obtain the current target data output rate, that is, the current target data output rate should be no less than 267 / s.

[0057] Furthermore, if the model data output rate of the target large model is taken into account, the data output rate of the front end can be adjusted based on the model data output rate of the target large model, the data output rate of the front end, and the queue length. If the model data output rate is 300 / s, and the other parameters remain unchanged, the current target time is 4s. If the data output rate is not adjusted, after all model output data in the current preset cache queue is output, that is, after 4s, there will be 1200 model output data in the preset cache queue, which is greater than the storage capacity of the preset cache queue. To solve this problem, consider introducing an adjustable queue length threshold. In this case, the adjusted current target data output rate can be determined based on the target queue length function L. The expression of the target queue length function L is as follows:

[0058] ;

[0059] in, is the model data output rate of the target large model, To adjust the queue length of the pre-set cache queue, is the adjusted current target data output rate, is the target time. Output the model output data in the current preset cache queue After 10 seconds, the queue length of the preset cache queue is L. If L is set not greater than the adjustable queue length threshold, the corresponding adjusted current target data output rate range can be obtained. The upper threshold of the adjustable queue length threshold can be set to 1000, and the lower threshold can be set to 0. Then the preset cache queue will always store model output data, and the number of model output data will not exceed the maximum capacity of the preset cache queue. The threshold range can be adjusted according to user needs. If the adjustable queue length threshold is set to 700, then "800+300*4- *4≤700", thus we get ≥325. That is, if the data output rate is adjusted to 325 items / s, the number of model output data in the preset cache queue will be equal to 700 items after 4 seconds.

[0060] In this embodiment, by adopting a strategy of dynamically adjusting the data output speed, the data display is made smoother and more continuous, avoiding the discomfort caused by the fast and slow data output speed. Users can enjoy a more coherent information flow, which greatly improves the user experience.

[0061] Step S14: calling the target data output function and extracting corresponding target model output data from the preset cache queue based on the current target data output rate, and outputting the target model output data.

[0062] In this embodiment, the target model data is obtained from the preset cache queue by calling the target data output function for output. The number of target model data obtained each time is related to the current target data output rate. Specifically, if the timer is triggered once every 50ms, the current data output rate is 200 pieces / s, that is, when the target connection is not interrupted and the amount of data in the preset cache queue is sufficient, 10 model output data will be output each time the timer is triggered; if the target connection is not interrupted and the number of model output data in the preset cache queue is less than 10, all model output data in the preset cache queue will be output.

[0063] In this embodiment, see Figure 4As shown, in order to enhance the user experience when using the streaming data output function, user interaction feedback can also be introduced to obtain the target data output rate adjustment instruction, and the current target data output rate is adjusted accordingly based on the target data output rate adjustment instruction to obtain the new current target data output rate. That is to say, by designing an intuitive interface control (such as a slider, a button group, etc.) on the interactive interface, the user is allowed to manually adjust the data output speed, stop output, etc. By monitoring the event of the user operating the control, the target data output rate adjustment instruction is obtained, and a global variable (such as userOutputSpeed) is updated based on the target data output rate adjustment instruction to save the new speed value, such as 500 / s. In a specific implementation, a new timer trigger time is determined based on the global variable, but the number of data output by each timer is not adjusted, such as before the adjustment, the timer is triggered once every 50ms, the timer is triggered 20 times per second, 10 data are output each time, and the data output rate is 200 / s; after the user adjusts, the timer is triggered once every 20ms, the timer is triggered 50 times per second, and 10 data are output each time, and the adjusted current target data output rate is 500 / s. In another specific implementation, the amount of data outputted by each timer is adjusted based on the global variable, but the timer triggering time is not changed. For example, after adjustment, the timer is triggered once every 50 ms, the timer is triggered 20 times per second, and 50 data are outputted each time. Then, the adjusted current target data output rate is 500 pieces / s. In addition, the timer triggering time and the amount of data outputted by each timer can also be adjusted at the same time. For example, if the timer triggering time is adjusted to 40 ms, the timer is triggered 25 times per second, and 20 data are outputted each time, the adjusted current target data output rate is still 500 pieces / s.

[0064] It is understandable that in the process of outputting data, various abnormal situations may be encountered, such as network errors, data format errors, etc. In this embodiment, the stability of the application and the user experience can be ensured by adding appropriate exception handling logic: for example, onClose and onErrorr of the fetchEventSource function correspond to the processing logic of function end and function exception capture, respectively. Use controller.value.abort() to stop the interface call to trigger the onClose function of fetchEventSource, turn off the timer here, or set the interface abort value to true. Under normal circumstances, when data output is no longer needed (for example, the page is closed or the user cancels the operation), the timer can be stopped and other related resources can be cleaned up by calling the clearInterval function to ensure that resources related to the timer are cleaned up and destroyed to avoid memory leaks or other potential problems.

[0065] It can be seen that the present application controls the output of the real-time model output data obtained from the target large model by introducing a cache mechanism and a preset cache queue. The front-end application no longer directly displays the data obtained by calling the back-end large model in real time, but temporarily stores the real-time model output data in the preset cache queue. The front-end can dynamically adjust the data output rate according to the queue length of the preset cache queue, thereby extracting the corresponding target model output data from the preset cache queue for output based on the current target data output rate, solving the instability problem of data reception and display, and ensuring that the data is displayed smoothly by the front-end. In addition, due to the introduction of the cache queue and queue control mechanism, the situation where the front-end application directly depends on the processing speed of the back-end model is reduced, reducing the complexity of system maintenance. At the same time, automated data management and exception handling logic simplifies the work of operation and maintenance personnel and reduces the cost of long-term maintenance.

[0066] See also Figure 5 As shown, the present application discloses a data output device based on a large model, which is applied to a front end and includes:

[0067] The data receiving module 11 is used to establish a target connection with the server using a preset streaming interface, and receive the real-time model output data of the target large model transmitted by the server in the form of streaming data through the target connection using the preset streaming interface;

[0068] The connection monitoring module 12 is used to store the real-time model output data in a preset cache queue and monitor in real time whether the target connection is interrupted;

[0069] The first output rate adjustment module 13 is used to determine whether the queue length of the current preset cache queue is greater than a preset queue length threshold if it is monitored that the target connection is not interrupted, and determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result to obtain the current target data output rate;

[0070] The target data output module 14 is used to call the target data output function and extract the corresponding target model output data from the preset cache queue based on the current target data output rate, and output the target model output data.

[0071] It can be seen that the present application controls the output of the real-time model output data obtained from the target large model by introducing a cache mechanism and a preset cache queue. The front-end application no longer directly displays the data obtained by calling the back-end large model in real time, but temporarily stores the real-time model output data in the preset cache queue. The front-end can dynamically adjust the data output rate according to the queue length of the preset cache queue, thereby extracting the corresponding target model output data from the preset cache queue based on the current target data output rate for output, thereby solving the instability problem of data reception and display, and ensuring that the data is displayed smoothly by the front-end.

[0072] Optionally, the device may further include:

[0073] A conditional judgment module, used to judge whether there is model output data obtained through the previous connection in the preset cache queue;

[0074] The data clearing unit is used to clear all the model output data in the preset cache queue by using the target response interruption function to interrupt the output if the preset cache queue contains the model output data obtained through the last connection.

[0075] Optionally, the connection monitoring module 12 may specifically include:

[0076] The parsed data storage unit is used to parse the real-time model output data into corresponding parsed data based on the target form by using the target event processing function, and store the parsed data in the preset cache queue.

[0077] Optionally, the device may further include:

[0078] A target parameter checking module, used to check whether the target parameter of the target connection is a preset parameter value;

[0079] A data checking module, used for clearing all data in the current preset cache queue if the target parameter is a preset parameter value, and checking whether there is any error in the real-time model output data;

[0080] The data discarding module is used to discard the real-time model output data if there is an error in the real-time model output data.

[0081] Optionally, the device may further include:

[0082] The output termination module is used to call a target cleanup function to clear all data in the current preset cache queue if the target connection is interrupted or a preset cleanup instruction is obtained, so as to terminate the action of outputting the target model output data.

[0083] Optionally, the device may further include:

[0084] The second output rate adjustment module is used to obtain a target data output rate adjustment instruction, and adjust the current target data output rate accordingly based on the target data output rate adjustment instruction to obtain a new current target data output rate.

[0085] Optionally, the first output rate adjusting module 13 may specifically include:

[0086] An output rate increasing unit is used to increase the data output rate based on a target time if the queue length of the current preset cache queue is greater than or equal to a preset queue length threshold, wherein the target time is the time required to output all model output data in the preset cache queue determined based on the data output rate and the queue length.

[0087] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0088] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data output method based on the large model disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0089] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0090] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0091] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the data output method based on the large model performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0092] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for outputting data based on a large model is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0093] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0094] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0095] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0096] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0097] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data output method based on a large model, characterized in that: Applied to the front end, including: Establishing a target connection with a server using a preset streaming interface, and receiving real-time model output data of a target large model transmitted by the server in the form of streaming data using the preset streaming interface through the target connection; The real-time model output data is stored in a preset cache queue, and the target connection is monitored in real time to determine whether it is interrupted; If it is monitored that the target connection is not interrupted, determine whether the queue length of the current preset cache queue is greater than the preset queue length threshold, and determine whether to adjust the data output rate of the front end based on the corresponding queue length determination result to obtain the current target data output rate; The target data output function is called and corresponding target model output data is extracted from the preset cache queue based on the current target data output rate, and the target model output data is output.

2. The data output method based on a large model according to claim 1, characterized in that: Before receiving the real-time model output data of the target large model transmitted by the server in the form of streaming data through the target connection using the preset streaming interface, the method further includes: Determine whether the preset cache queue contains model output data obtained through the last connection; If the preset cache queue contains model output data acquired through the last connection, the target response interrupt function is used to clear all model output data in the preset cache queue to interrupt the output.

3. The data output method based on a large model according to claim 1, characterized in that: The step of storing the real-time model output data in a preset cache queue includes: The real-time model output data is parsed into corresponding parsed data based on the target form by using the target event processing function, and the parsed data is stored in the preset cache queue.

4. The data output method based on a large model according to claim 3 is characterized in that: Before parsing the real-time model output data into corresponding parsed data based on the target form using the target event processing function, the method further includes: Checking whether a target parameter of the target connection is a preset parameter value; If the target parameter is a preset parameter value, all data in the current preset cache queue is cleared, and the real-time model output data is checked for errors; If there is an error in the real-time model output data, the real-time model output data is discarded.

5. The data output method based on a large model according to claim 1, characterized in that: Also includes: If the target connection is interrupted, or a preset cleanup instruction is obtained, a target cleanup function is called to clear all data in the current preset cache queue to terminate the action of outputting the target model output data.

6. The data output method based on a large model according to claim 1, characterized in that: Also includes: A target data output rate adjustment instruction is obtained, and based on the target data output rate adjustment instruction, the current target data output rate is adjusted accordingly to obtain a new current target data output rate.

7. The data output method based on a large model according to any one of claims 1 to 6, characterized in that: The determining whether to adjust the data output rate of the front end based on the corresponding queue length judgment result includes: If the current queue length of the preset cache queue is greater than or equal to a preset queue length threshold, the data output rate is increased based on a target time, where the target time is the time required to output all model output data in the preset cache queue determined based on the data output rate and the queue length.

8. A data output device based on a large model, characterized in that: Applied to the front end, including: A data receiving module, used to establish a target connection with a server using a preset streaming interface, and receive real-time model output data of a target large model transmitted by the server in the form of streaming data through the target connection using the preset streaming interface; A connection monitoring module, used to store the real-time model output data in a preset cache queue and monitor in real time whether the target connection is interrupted; A first output rate adjustment module is used to determine whether the queue length of the preset cache queue is greater than a preset queue length threshold if it is detected that the target connection is not interrupted, and determine whether to adjust the data output rate of the front end based on the corresponding queue length judgment result to obtain the current target data output rate; The target data output module is used to call the target data output function and extract the corresponding target model output data from the preset cache queue based on the current target data output rate, and output the target model output data.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the large model-based data output method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the large model-based data output method according to any one of claims 1 to 7 is implemented.

Images