Method and device for generating push information of live broadcast room, live broadcast system and electronic equipment

Abstract

The invention relates to a live broadcast room push information generation method and device, a live broadcast system, electronic equipment and a computer readable storage medium. The method comprises the following steps: collecting chat interaction information between an anchor and audiences in a live broadcasting room; analyzing and processing the chat interaction information to obtain chat text data; inputting the chat text data into a pre-trained prompt template to obtain large language model input data, wherein the prompt template is obtained by performing format optimization training based on a text structure of push information of a pre-labeled live broadcast room; inputting the big language model input data into a pre-trained big language model to obtain push information of the live broadcast room, wherein the big language model is obtained by performing parameter adjustment training based on a general language model and by using text content of the pre-labeled push information of the live broadcast room; according to the technical scheme, the high-quality push information can be generated, so that the user can quickly know the interaction topic of the current live broadcast room, and the daily live volume of the live broadcast platform is improved.

Description

Technical Field

[0001] The present application relates to the technical field of network live broadcasting, and in particular to a method, device, live broadcasting system, electronic device and computer-readable storage medium for generating live broadcast room push information. Background Art

[0002] With the development of network technology, live streaming has been used by most Internet users. With its intuitive, immediacy and interactivity in content and form, live streaming has played an important role in promoting flexible employment, promoting economic and social development, and enriching the spiritual and cultural life of the people. Anchors can better display their talents in live broadcasts, thereby helping more anchors realize their self-worth.

[0003] In the current live broadcast platform, when new viewers just enter a live broadcast room, they often cannot understand the current live broadcast content because they are not clear about the chat topic between the host and other viewers. They cannot immediately participate in the current topic chat, which reduces their sense of participation and reduces their willingness to stay in the live broadcast room. As a result, the live broadcast room is not attractive enough to new viewers, and it is difficult to increase user retention and usage time. In order to make new viewers understand the chat topic content of the live broadcast room, it is usually adopted to generate push information to assist users to understand the live broadcast room, such as generating summary information of the live broadcast room and pushing it to users, so as to assist them to quickly understand the current chat topic content; for example, there is a technical solution that extracts the bullet screen content of the live broadcast room and obtains summary information from the bullet screen sent by the user by extraction, but the summary information obtained by this method is not comprehensive enough and cannot accurately reflect the chat topic of the live broadcast room, and the quality is low; there is a technical solution that summarizes the interaction data between the host and the audience and outputs it to new viewers to improve the interaction rate, but the summary is extracted by extracting keywords, and the diversity and extraction accuracy of keywords are limited, resulting in poor summary quality. The extractive summary extraction method has the problem of low summary quality.

[0004] It can be seen that the existing technical solutions, when forming recommendation information to assist new viewers in understanding the live broadcast room, are mainly based on extracting summary information in an extraction manner, which can easily affect the interactivity of the live broadcast room and affect the daily active users of the live broadcast platform. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, live broadcast system, electronic device and computer-readable storage medium for generating live broadcast room push information, which can generate high-quality live broadcast room summary information.

[0006] A method for generating live broadcast room push information, comprising:

[0007] Collect chat interaction information between the host and the audience in the live broadcast room;

[0008] Analyzing and processing the chat interaction information to obtain chat text data;

[0009] Inputting the chat text data into a pre-trained prompt template to obtain large language model input data; wherein the prompt template is obtained by format optimization training based on the text structure of the push information of the pre-annotated live broadcast room;

[0010] The large language model input data is input into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is trained by adjusting parameters based on a general language model and using the text content of the push information of the live broadcast room that is pre-labeled.

[0011] In one embodiment, the chat interaction information between the host and the audience in the live broadcast room is collected, including:

[0012] Collect the sound signal from the host side of the live broadcast room, separate the sound signal from the human voice and remove the background music to obtain the host's voice signal;

[0013] The speech content input by the audience in the live broadcast room on the public screen is extracted, and the non-text symbols in the speech content are filtered to obtain the speech text.

[0014] In one embodiment, analyzing and processing the chat interaction information to obtain chat text data includes:

[0015] Using speech recognition technology to convert the live speech signal into speech text information;

[0016] Downsampling the speech text to obtain public screen text information;

[0017] The voice text information and the public screen text information are aligned according to the timestamps to obtain the chat text data.

[0018] In one embodiment, the method for generating live broadcast room push information further includes:

[0019] Design an initial template for filling in prompt content based on the information to be pushed;

[0020] Testing using the initial template and the large language model;

[0021] During the test, the format of the initial template is optimized to obtain a prompt template.

[0022] In one embodiment, the method for generating live broadcast room push information further includes fine-tuning the parameters of the large language model, and the fine-tuning method includes:

[0023] Obtain a pre-trained model of a general language model, and add a bypass consisting of a dimension reduction matrix and a dimension increase matrix to a pre-trained weight matrix of the pre-trained model; wherein the matrix product of the dimension reduction matrix and the dimension increase matrix of the bypass is consistent with the dimension of the pre-trained weight matrix;

[0024] In the training phase, the pre-trained weight matrix is ​​fixed and unchanged, and only the dimension reduction matrix and the dimension increase matrix are trained. When the model is output, the matrix product of the dimension increase matrix and the dimension reduction matrix is ​​superimposed with the pre-trained weight matrix for parameterization;

[0025] In the inference phase, the model input is multiplied by the superposition parameters as the model output.

[0026] In one embodiment, the push information includes summary information and recommended speeches; wherein, the summary information is used to display the general information of the current live broadcast room chat topic, and the recommended speeches are speeches that match the style of the current chat content in the live broadcast room for users to choose to send.

[0027] In one embodiment, the prompt template includes a summary information prompt template and a recommended speech prompt template, which are used to generate summary information and recommended speech respectively.

[0028] In one embodiment, the large language model includes a summary large language model and a recommendation speech large language model having the same model structure.

[0029] In one embodiment, the large language model input data is input into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is trained by adjusting parameters based on a general language model and using text content of the push information of the pre-annotated live broadcast room, including:

[0030] Inputting the large language model input data into a pre-trained summary large language model to obtain summary information; inputting the summary information into a pre-trained recommended speech large language model to obtain a recommended speech;

[0031] The push information of the live broadcast room is generated according to the summary information and the recommended speech and displayed through a pop-up window; wherein the user sends the push information by clicking on the candidate recommended speech in the push information.

[0032] In one embodiment, the method for generating live broadcast room push information further includes:

[0033] Obtain the pre-trained weights of the summary language model and the recommendation speech language model;

[0034] In the training phase, the chat text data is used as the input of the summary language model, and the pre-annotated summary information is used as the model output to fine-tune the summary language model; and the annotated summary information is used as the input of the recommendation speech language model, and the pre-annotated recommended speech is used as the model output to fine-tune the recommendation speech language model;

[0035] In the inference stage, the chat text data is input into the summary language model to generate summary information, and the summary information is input into the recommended speech language model to generate recommended speech.

[0036] In one embodiment, the method for generating live broadcast room push information further includes:

[0037] Summary information and recommended speeches used by users are filled in for each group of chat text data, and the summary information and recommended speeches are used to adjust and train the parameters of the large language model.

[0038] A device for generating live broadcast room push information, comprising:

[0039] The collection module is used to collect chat interaction information between the anchor and the audience in the live broadcast room;

[0040] An analysis module, used for analyzing and processing the chat interaction information to obtain chat text data;

[0041] A filling module is used to input the chat text data into a pre-trained prompt template to obtain large language model input data; wherein the prompt template is obtained by format optimization training based on the text structure of the push information of the pre-annotated live broadcast room;

[0042] A generation module is used to input the large language model input data into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is based on a general language model and is trained by adjusting parameters using the text content of the push information of the pre-annotated live broadcast room.

[0043] A live broadcast system, comprising: an anchor terminal, an audience terminal and a live broadcast server; wherein the anchor terminal and the audience terminal are respectively connected to the live broadcast server via a communication network;

[0044] The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server;

[0045] The live broadcast server is used to forward the live broadcast video stream of the anchor to the audience end, and receive the analysis data request from the audience end, and use the live broadcast room push information generation method to generate push information and display it;

[0046] The audience end is used for audience users who access the live broadcast room, receive the live video stream of the anchor for playback, and display the push information.

[0047] An electronic device, comprising:

[0048] one or more processors;

[0049] Memory;

[0050] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the method for generating live broadcast room push information.

[0051] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the method for generating live broadcast room push information.

[0052] The technical solution of the present application collects chat interaction information between the host and the audience in the live broadcast room for analysis and processing to obtain chat text data; fills the chat text data into an optimized prompt template to obtain large language model input data, and then inputs the large language model input data into a large language model after fine-tuning parameters to obtain push information of the live broadcast room; this technical solution uses the chat text data obtained from the chat interaction information in the live broadcast room, processes it through an optimized prompt template, and then inputs it into a large language model trained based on a general language model, so as to generate high-quality push information, which is convenient for users to accurately understand the chat topic content of the current live broadcast room, enhances the interactivity of the live broadcast room, and improves the daily activity of the live broadcast platform.

[0053] Furthermore, summary information of the live broadcast room is generated based on the chat text data of the live broadcast room and the summary big language model, and recommended speeches of the live broadcast room are generated based on the summary information and the recommended speech big language model. As a result, new users can quickly understand the interactive topics of the current live broadcast room through the summary information, and can choose recommended speeches to quickly participate in the chat, thereby greatly improving business indicators such as user retention and usage time. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a schematic diagram of an example live broadcast service application scenario;

[0055] Figure 2 is a flow chart of a method for generating live broadcast room push information according to an embodiment;

[0056] Figure 3 This is a diagram of an example of fine-tuning a large language model.

[0057] Figure 4 Generate a flowchart for a sample summary of information and recommended speeches;

[0058] Figure 5 This is a schematic diagram of an example push information format;

[0059] Figure 6 This is an example of a push information display effect diagram;

[0060] Figure 7 is a schematic diagram of the structure of a device for generating live broadcast room push information according to an embodiment;

[0061] Figure 8 is a block diagram of an example electronic device. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. For example, multiple objects refer to two or more objects. "Include" or "comprising" and other similar words mean that the information appearing before "include" or "comprising" covers the information listed after "include" or "comprising" and its equivalents, and does not exclude other information. The "and / or" mentioned in the embodiments of the present application indicates that there can be three kinds of relationships. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0064] The technical solution provided in the embodiments of the present application can be applied to Figure 1 In the application scenario of the method related to this application shown in FIG. Figure 1 This is a schematic diagram of an example of a live broadcast service application scenario. The live broadcast system may include a live broadcast server, an anchor terminal and an audience terminal. The anchor terminal and the audience terminal communicate with the live broadcast server through a communication network, so that the anchor at the anchor terminal and the audience users at the audience terminal can perform real-time network live broadcast. Among them, for the anchor terminal and the audience terminal, their terminal devices can be but are not limited to various personal computers, laptops, smart phones and tablets. The live broadcast server can be implemented by an independent server or a server cluster composed of multiple servers.

[0065] In order to facilitate users to obtain high-quality push information, the technical solution of the embodiment of the present application collects the chat interaction information between the live broadcast room host and the audience for analysis and processing to obtain chat text data; fills the chat text data into the optimized prompt template to obtain the large language model input data, and then inputs the large language model input data into the large language model after fine-tuning the parameters to obtain the push information of the live broadcast room, so that when new users enter the live broadcast room, they can timely understand the current live broadcast room chat topic information, thereby enhancing the interactivity of the live broadcast room.

[0066] refer to Figure 2 As shown, Figure 2 The flowchart of a method for generating live broadcast room push information according to an embodiment includes the following steps:

[0067] Step S10, collecting chat interaction information between the host and the audience in the live broadcast room.

[0068] In this step, the chat interaction information on the host side and the chat interaction data on the audience side are collected. Generally speaking, it mainly includes the host's voice signal and the text content sent by the audience on the public screen.

[0069] In one embodiment, the method of collecting chat interaction information between the host and the audience in step S10 may include the following:

[0070] The sound signal from the host side of the live broadcast room is collected, and the sound signal is separated from the human voice and the background music is removed to obtain the host's voice signal; the speech content entered by the audience side of the live broadcast room on the public screen is extracted, and the non-text symbols in the speech content are filtered to obtain the speech text.

[0071] Specifically, the chat interaction data of the live broadcast room includes the voice signal of the anchor and the text information entered by the audience on the public screen of the live broadcast room. If it is a scene where the anchor is connected to the microphone, the anchor's voice signal includes the voice signal of the anchor of this live broadcast room and the anchor connected to the microphone.

[0072] As in the above-mentioned embodiment, by collecting chat interaction information from the host side and the audience side, the chat topic content of the current live broadcast room can be more comprehensively reflected.

[0073] Step S20, analyzing and processing the chat interaction information to obtain chat text data.

[0074] In this step, the chat interaction information between the host and the audience in the live broadcast room is semantically analyzed and converted into chat text data.

[0075] In one embodiment, the method of analyzing and processing the chat interaction information to obtain chat text data in step S20 may include the following:

[0076] The live voice signal is converted into voice text information by using voice recognition technology; the speech text is downsampled to obtain public screen text information; and the voice text information and the public screen text information are aligned according to timestamps to obtain chat text data.

[0077] Specifically, voice recognition technology is used to convert voice signals into voice text information, the speech text entered by the audience on the public screen is converted into text information, and the speech is downsampled to obtain high-quality public screen text information.

[0078] For example, assuming that the public screen speech set G has a total of m speeches and the maximum number of sampled speeches is n, the sampled speech set H is:

[0079]

[0080] Among them, RS n (G) is to randomly sample n speeches from G. When m≤n, the entire speech set G is selected; when m>n, n speeches are randomly sampled from G.

[0081] The solution of the above embodiment obtains chat text data of the host and the audience aligned according to timestamps, thereby obtaining high-quality chat text data and ensuring that the order of relevant data input into the large language model is correct.

[0082] Step S30, inputting the chat text data into a pre-trained prompt template to obtain large language model input data; wherein the prompt template is obtained by format optimization training based on the text structure of the push information of the pre-annotated live broadcast room.

[0083] In this step, a prompt template is trained based on the format optimization of the text structure of the push information of the live broadcast room that has been pre-annotated. The optimized prompt template is used to first process the data input into the large language model, thereby guiding the large language model to generate high-quality output results.

[0084] In one embodiment, based on the impact of input prompts on the quality of output results generated by a large language model, the embodiment pre-trains a prompt template with an optimized format for the input data of the large language model to process chat text data. First, an initial template for processing the prompt content format is designed according to the information to be pushed; the initial template and the large language model are used for testing, and the format of the initial template is optimized during the test to obtain the prompt template.

[0085] Specifically, we design the initial template based on the prompt content of the large language model, and use the pre-annotated chat text data of the live broadcast room and the push information generated by the large language model. Through continuous experimentation and tuning of the prompt input method, we optimize and train the initial template, and finally obtain a high-quality prompt template, which can guide the large language model to generate high-quality push information.

[0086] Step S40, inputting the large language model input data into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is trained by adjusting parameters based on a general language model and using the text content of the push information of the live broadcast room that is pre-labeled.

[0087] In this step, based on the general language model and using the pre-annotated push information of the live broadcast room, the text content of the push information is used to perform parameter adjustment training to obtain a large language model that can output high-quality push information. In general, the general language model is used to perform parameter fine-tuning training. Fine-tuning refers to the appropriate adjustment of some parameters of the general language model, so that the large language model not only utilizes the powerful basic performance of the general language model and has powerful information processing capabilities, but also can be combined with the personalized needs of the live broadcast scene in the live broadcast room.

[0088] In the scheme of the above embodiment, when using a large language model, a general language model is used to fine-tune its parameters. Since the research and development of a large language model requires massive data and a large-scale computing cluster, the research and development cost is extremely high. Therefore, a free, open source language model or a paid, closed source language model can be selected according to actual conditions. Since general language models are usually pre-trained for general scenarios, the quality of push information generated in the live broadcast room is often not high. In order to improve the quality of push information generated by the large language model, pre-standardization is performed based on the chat text data of the real live broadcast room, and then used to fine-tune the parameters of the general language model to obtain a large language model.

[0089] In one embodiment, a low rank adaptive (LoRA) algorithm may be used to fine-tune the parameters of a general language model to obtain a large language model. The training method may include the following:

[0090] (1) Obtain a pre-trained model of a general language model, and add a bypass consisting of a dimensionality reduction matrix and a dimensionality increase matrix to a pre-trained weight matrix of the pre-trained model; wherein the matrix product of the dimensionality reduction matrix and the dimensionality increase matrix of the bypass is consistent with the dimension of the pre-trained weight matrix.

[0091] Exemplarily, a bypass consisting of a dimensionality reduction matrix A and a dimensionality increase matrix B is added to the pre-trained weight matrix W of the pre-trained model, and the matrix product BA of the two bypass matrices is consistent with the dimension of the pre-trained weight matrix W.

[0092] (2) During the training phase, the pre-trained weight matrix is ​​fixed and remains unchanged, and only the dimension reduction matrix and the dimension increase matrix are trained. When the model is output, the matrix product of the dimension increase matrix and the dimension reduction matrix is ​​superimposed with the pre-trained weight matrix for parameterization.

[0093] Exemplarily, during training, the pre-trained weight matrix W of the pre-trained model is fixed, and only the dimensionality reduction matrix A and the dimensionality increase matrix B are trained. The dimensionality reduction matrix A is randomly initialized to a Gaussian distribution, and the dimensionality increase matrix B is initialized to 0. When the model is output, the matrix product BA of the dimensionality reduction matrix and the dimensionality increase matrix is ​​superimposed with the pre-trained weight matrix W for parameters.

[0094] like Figure 3 As shown, Figure 3 This is an example of a large language model fine-tuning training diagram, assuming that the pre-trained weight matrix of the large language model is The update can then be expressed as:

[0095] W0+ΔW=W0+BA

[0096] Among them, ΔW represents the update adjustment of the pre-trained weight matrix, Rank r<<min(d,k); During the training phase, keep the pre-training weight matrix W0 fixed, and adjust the dimension reduction matrix A and the dimension increase matrix B.

[0097] (3) In the inference phase, the model input is multiplied by the superposition parameter as the model output.

[0098] For example, during forward reasoning, the pre-trained weight matrix W0 and the update adjustment ΔW are multiplied by the same model input x and finally added together, and the expression is as follows:

[0099] h=W0x+ΔWx=W0x+BAx

[0100] Among them, h is the output of the large language model after fine-tuning. Since the number of parameters of the matrix product BA that needs to be fine-tuned (d×r+r×k) is much smaller than the number of parameters of the pre-trained weight matrix W0 (d×k), it is possible to complete the fine-tuning of the large language model by training only a few parameters.

[0101] As in the technical solutions of the above-mentioned embodiments, considering that the research and development of a general language model requires massive amounts of data and a large-scale computing cluster, and the research and development cost is extremely high, a large language model is obtained by utilizing a commercial or free general language model and then using a low-rank adaptive algorithm to perform parameter fine-tuning training. Since the parameter fine-tuning training of the large language model is based on the chat interaction data of the real live broadcast room, it has the advantages of fewer parameters, saving computing resources, strong generalization, and good original model knowledge inheritance ability, thereby improving the recognition accuracy of the large language model and improving the quality of the push information generated by the large language model.

[0102] In order to make the technical solution of the present application more clear, it is further explained below in conjunction with embodiments.

[0103] In one embodiment, the push information generated based on the method for generating push information in the live broadcast room includes summary information and recommended speeches; the summary information is used to display the summary information of the current live broadcast room chat topic, and the recommended speeches are speeches that match the style of the current chat content in the live broadcast room for users to choose to send.

[0104] Based on this, when generating push information for the live broadcast room, the annotated summary information and recommended speeches can be used to fine-tune the large language model used; the chat text data is filled into a prompt template and then input into the large language model, so that the summary information and recommended speeches of the live broadcast room can be obtained.

[0105] As in the scheme of the above-mentioned embodiment, summary information is generated based on the chat interaction data between the anchor and the audience and output to the audience users, so that when new users enter the live broadcast room, they can quickly understand the current chat topic content of the live broadcast room, thereby improving the interaction rate between the live broadcast room and the users. In addition, recommended speeches are further generated, so that new users can quickly participate in the live broadcast interaction in the live broadcast room with limited information, thereby enhancing the interactivity of the live broadcast in the live broadcast room and improving the daily active users of the live broadcast platform.

[0106] In one embodiment, a prompt template is used to input a large language model to generate a summary and a recommended speech at the same time. The goals of the generated summary information and the recommended speech are different. The quality of the generated summary information is easily affected by the difference between the summary information and the recommended speech. Accordingly, the present application provides a technical solution based on two cascaded large language model recognition, such as Figure 4 As shown, Figure 4 A flowchart is provided for generating summary information and recommended speech for an example; wherein the large language model includes a summary large language model and a recommended speech large language model with the same model structure; correspondingly, the prompt template includes a summary prompt template and a recommended speech prompt template, which are used to generate chat text data and summary information respectively.

[0107] When generating push information for the live broadcast room, the summary language model with fine-tuned parameters can be used to fill the chat text data into the summary prompt template, and then input it into the summary language model to generate summary information; then the output summary information is filled into the recommended speech prompt template, and then input into the recommended speech language model with fine-tuned parameters to generate recommended speech; Figure 5 As shown, Figure 5This is an example of a summary prompt template and a recommended speech prompt template. When displaying, the summary information and recommended speeches can be combined into push information of the live broadcast room and displayed through a pop-up window. For example, the summary information and recommended speeches can be generated every minute and then displayed in a pop-up window. Preferably, in the summary information, each summary contains two versions, a general description and a detailed description, which are used for thumbnail display and expanded display of the summary information respectively. In the recommended speech, by generating multiple candidate recommended speeches, the user can send them by clicking on the candidate recommended speech in the push information, and quickly participate in the chat topic of the current live broadcast room.

[0108] As in the above-mentioned embodiment, the solution fully considers the characteristics of summary information being precise and concise, as well as the characteristics that recommended speeches require accurate meaning and each recommended speech has diversity. For this purpose, a technical solution of cascade model recognition is adopted, and high-quality summary information and recommended speeches can be generated through two large language models.

[0109] In one embodiment, based on the summary large language model and the recommended speech large language model used, in order to improve the model recognition accuracy, when the summary large language model and the recommended speech large language model are trained and fine-tuned using the annotated summary information and the recommended speech, the following scheme can be adopted:

[0110] (1) Obtaining pre-trained weight matrices of a summary large language model and a recommendation speech large language model; specifically, the summary large language model and the recommendation speech large language model based on a general language model have the same model structure and pre-trained weights, and are fine-tuned using different annotated data.

[0111] (2) During the training phase, the chat text data is used as the input of the summary language model, and the pre-annotated summary information is used as the model output to fine-tune the summary language model; and the annotated summary information is used as the input of the recommendation speech language model, and the pre-annotated recommended speech is used as the model output to fine-tune the recommendation speech language model.

[0112] (3) In the inference stage, the chat text data is input into the summary language model to generate summary information, and the summary information is input into the recommendation speech language model to generate recommended speech.

[0113] Exemplarily, the collected chat interaction data to be used for training can be annotated, and the chat text data of each group of anchors and viewers can be filled with corresponding summaries and recommended speeches through annotation, and then the training and fine-tuning of the large language model can be performed; preferably, in order to improve the efficiency of data annotation, a pre-trained large language model can be used to automatically generate pre-annotated summary information and recommended speeches, and then the annotation results can be corrected manually.

[0114] The scheme of the above-mentioned embodiment can use the same large language model to train the summary large language model and the recommended speech large language model, and fine-tune different parameters of the large language model to implement summary large language models and recommended speech large language models with different functions. When in use, summary information and recommended speeches can be generated every one minute, thereby having better timeliness.

[0115] In one embodiment, the technical solution of the present application can also fill in summary information and recommended speeches used by users for each group of chat text data, and use the summary information and recommended speeches to adjust and train the parameters of the large language model.

[0116] Specifically, for the summary information and recommended speeches to be generated by the large language model each time, the relevant input chat text data can be recorded, and then according to the target recommended speeches used by the users in the recommended speeches, the target recommended speeches and the input summary information and chat text information can be used to train the summary large language model and the recommended speech large language model, so that the summary large language model and the recommended speech large language model can improve the quality of the output summary information and recommended speeches.

[0117] The technical solution of the above embodiment can be used in conjunction with an AI assistant on a live broadcast platform, such as Figure 6 As shown, Figure 6 This is an example of a push information display effect diagram; the AI ​​assistant "Ling'er" in the figure is a tool used to assist live broadcasts. When a new viewer enters the live broadcast room, the chat interaction content of the host chatting with other viewers in the past period of time (such as 1 minute) is generated into chat text data, and then input into the summary large language model to obtain summary information and display it to the new viewer, so that the new viewer can quickly understand the content of the previous live broadcast topic; at the same time, the summary information is input into the recommended speech large language model to generate multiple recommended speeches for the new viewer. The new viewer only needs to click on the recommended speech to send the recommended speech content, so as to quickly participate in the topic chat interaction process. As shown in the figure, the left box indicates the summary information and recommended speech displayed by the AI ​​assistant "Ling'er" by default. When the "Ling'er" icon is clicked, the detailed summary information and recommended speech indicated by the right box will be expanded.

[0118] An embodiment of a device for generating live broadcast room push information is described below.

[0119] like Figure 7 As shown, Figure 7 The structure diagram of a device for generating live broadcast room push information according to an embodiment includes:

[0120] The collection module 10 is used to collect chat interaction information between the host and the audience in the live broadcast room;

[0121] An analysis module 20, configured to analyze and process the chat interaction information to obtain chat text data;

[0122] A filling module 30 is used to input the chat text data into a pre-trained prompt template to obtain large language model input data; wherein the prompt template is obtained by format optimization training based on the text structure of the push information of the pre-annotated live broadcast room;

[0123] The generation module 40 is used to input the large language model input data into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is based on a general language model and is trained by adjusting parameters using the text content of the pre-annotated push information of the live broadcast room.

[0124] The device for generating live broadcast room push information of this embodiment can execute a method for generating live broadcast room push information provided by the embodiment of this application. The implementation principle is similar. The actions performed by each module in the device for generating live broadcast room push information in each embodiment of this application correspond to the steps in the method for generating live broadcast room push information in each embodiment of this application. For the detailed functional description of each module in the device for generating live broadcast room push information, please refer to the description in the corresponding method for generating live broadcast room push information shown in the previous text, which will not be repeated here.

[0125] An embodiment of the live broadcast system is described below.

[0126] The live broadcast system of this embodiment refers to Figure 8 As shown, Figure 8 This is an example of a live broadcast system structure diagram. The system framework may include an anchor terminal, a live broadcast server and an audience terminal. The audience terminal and the anchor terminal may access the live broadcast platform through a client installed on an electronic device. For example, the anchor terminal and the audience terminal may be computer devices such as PDAs, smart phones, tablet computers, desktop computers or laptop computers, etc., but this is not limited. They may also be software modules of an application program. The live broadcast server may be implemented by an independent server or a server cluster composed of multiple servers. The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server; the live broadcast server is used to forward the anchor's live video stream to the audience terminal, and receive the analysis data request from the audience terminal, and use the method for generating live broadcast room push information in any of the above-mentioned embodiments to generate push information and display it; the audience terminal is used to access the audience user in the live broadcast room, receive the anchor's live video stream for playback, and display the push information.

[0127] Exemplarily, a technical solution using two cascaded large language models is used. Using a summary large language model with fine-tuned parameters, the chat text data is filled into a summary prompt template, and then input into the summary large language model to generate summary information; the generated summary information is then filled into a recommended speech prompt template, and then input into the recommended speech large language model with fine-tuned parameters to generate recommended speech. In the summary information, each summary contains two versions, a summary description and a detailed description, which are used for thumbnail display and expanded display of the summary information, respectively. In the recommended speech, multiple candidate recommended speeches are generated, and users can send them by clicking on the candidate recommended speeches in the push information, and quickly participate in the chat topic of the current live broadcast room.

[0128] The technical solution of this application uses the chat text data obtained from the interactive chat information in the live broadcast room, and can generate high-quality summary information through optimized prompt templates and large language model push information, which is convenient for users to accurately understand the chat topic content of the current live broadcast room, enhance the interactivity of the live broadcast room, and increase the daily activity of the live broadcast platform. The summary information of the live broadcast room is generated based on the chat text data of the live broadcast room and the summary large language model, and the recommended speech of the live broadcast room is generated based on the summary information and the recommended speech large language model, so that new users can quickly understand the interactive topics of the current live broadcast room through the summary information, and users can quickly participate in the chat by choosing the recommended speech, thereby greatly improving business indicators such as user retention and usage time.

[0129] This application enables new viewers to quickly understand and participate in topic chats in the live broadcast room, which can quickly enhance the interaction between the anchor and the audience, and effectively improve the interaction rate, viewing time and payment rate of the live broadcast room.

[0130] Embodiments of electronic devices and computer-readable storage media are described below.

[0131] The present application provides a technical solution for an electronic device to implement functions related to a method for generating live broadcast room push information. The electronic device of this embodiment includes one or more processors, a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by one or more processors, and the one or more programs are configured for the steps of the method for generating live broadcast room push information of any embodiment.

[0132] like Figure 8 As shown, Figure 81 is a block diagram of an exemplary electronic device; the electronic device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. The electronic device 100 may include one or more of the following components: a processing component 102, a memory 104, a power component 106, a multimedia component 108, an audio component 109, an input / output (I / O) interface 112, a sensor component 114, and a communication component 116.

[0133] The processing component 102 generally controls the overall operation of the electronic device 100, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.

[0134] The memory 104 is configured to store various types of data to support operations in the electronic device 100, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0135] The power supply component 106 provides power to various components of the electronic device 100 .

[0136] The multimedia component 109 includes a screen that provides an output interface between the electronic device 100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). In some embodiments, the multimedia component 108 includes a front camera and / or a rear camera.

[0137] The audio component 109 is configured to output and / or input audio signals.

[0138] I / O interface 112 provides an interface between processing component 102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0139] The sensor component 114 includes one or more sensors for providing various aspects of status assessment for the electronic device 100. The sensor component 114 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact.

[0140] The communication component 116 is configured to facilitate wired or wireless communication between the electronic device 100 and other devices. The electronic device 100 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof.

[0141] The present application provides a technical solution of a computer-readable storage medium, which is used to implement functions related to a method for generating live broadcast room push information. The computer-readable storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded by a processor and executes the method for generating live broadcast room push information of any embodiment.

[0142] In an exemplary embodiment, the computer-readable storage medium may be a non-temporary computer-readable storage medium including instructions, such as a memory including instructions. For example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0143] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for generating push information in a live broadcast room, It is characterized in that include: Collect chat interaction information between the host and the audience in the live broadcast room; Analyzing and processing the chat interaction information to obtain chat text data; Inputting the chat text data into a pre-trained prompt template to obtain large language model input data; wherein the prompt template is obtained by format optimization training based on the text structure of the push information of the pre-annotated live broadcast room; The large language model input data is input into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is trained by adjusting parameters based on a general language model and using the text content of the push information of the live broadcast room that is pre-labeled.

2. The method for generating live broadcast room push information according to claim 1, It is characterized in that Collect chat interaction information between the host and the audience in the live broadcast room, including: Collect the sound signal from the host side of the live broadcast room, separate the sound signal from the human voice and remove the background music to obtain the host's voice signal; The speech content input by the audience in the live broadcast room on the public screen is extracted, and the non-text symbols in the speech content are filtered to obtain the speech text.

3. The method for generating live broadcast room push information according to claim 2, It is characterized in that Analyzing and processing the chat interaction information to obtain chat text data includes: Using speech recognition technology to convert the live speech signal into speech text information; Downsampling the speech text to obtain public screen text information; The voice text information and the public screen text information are aligned according to the timestamps to obtain the chat text data.

4. The method for generating live broadcast room push information according to claim 1, It is characterized in that Also includes: Design an initial template for filling in prompt content based on the information to be pushed; Testing using the initial template and the large language model; During the test, the format of the initial template is optimized to obtain a prompt template.

5. The method for generating live broadcast room push information according to claim 1, It is characterized in that Also includes: Obtain a pre-trained model of a general language model, and add a bypass consisting of a dimension reduction matrix and a dimension increase matrix to a pre-trained weight matrix of the pre-trained model; wherein the matrix product of the dimension reduction matrix and the dimension increase matrix of the bypass is consistent with the dimension of the pre-trained weight matrix; In the training phase, the pre-trained weight matrix is ​​fixed and unchanged, and only the dimension reduction matrix and the dimension increase matrix are trained. When the model is output, the matrix product of the dimension increase matrix and the dimension reduction matrix is ​​superimposed with the pre-trained weight matrix for parameterization; In the inference phase, the model input is multiplied by the superposition parameters as the model output.

6. The method for generating live broadcast room push information according to any one of claims 1 to 5, It is characterized in that The push information includes summary information and recommended speeches; wherein the summary information is used to display the general information of the current live broadcast room chat topic, and the recommended speeches are speeches that match the style of the current chat content in the live broadcast room and are available for users to choose to send.

7. The method for generating live broadcast room push information according to claim 6, It is characterized in that The prompt templates include a summary prompt template and a recommended speech prompt template, which are used to generate summary information and recommended speeches respectively; The large language model includes a summary large language model and a recommendation speech large language model with the same model structure; Inputting the large language model input data into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is trained by adjusting parameters based on a general language model and using the text content of the push information of the pre-annotated live broadcast room, including: Inputting the chat text data into a pre-trained summary language model to obtain summary information; inputting the summary information into a pre-trained recommended speech language model to obtain recommended speech; The push information of the live broadcast room is generated according to the summary information and the recommended speech and displayed through a pop-up window; wherein the user sends the push information by clicking on the candidate recommended speech in the push information.

8. The method for generating live broadcast room push information according to claim 7, It is characterized in that Also includes: Obtain the pre-trained weights of the summary language model and the recommendation speech language model; In the training phase, the chat text data is used as the input of the summary language model, and the pre-annotated summary information is used as the model output to fine-tune the summary language model; and the annotated summary information is used as the input of the recommendation speech language model, and the pre-annotated recommended speech is used as the model output to fine-tune the recommendation speech language model; In the inference stage, the chat text data is input into the summary language model to generate summary information, and the summary information is input into the recommended speech language model to generate recommended speech.

9. The method for generating live broadcast room push information according to claim 1, It is characterized in that Also includes: Summary information and recommended speeches used by users are filled in for each group of chat text data, and the summary information and recommended speeches are used to adjust and train the parameters of the large language model.

10. A device for generating push information in a live broadcast room, It is characterized in that include: The collection module is used to collect chat interaction information between the anchor and the audience in the live broadcast room; An analysis module, used for analyzing and processing the chat interaction information to obtain chat text data; A filling module is used to input the chat text data into a pre-trained prompt template to obtain large language model input data; wherein the prompt template is obtained by format optimization training based on the text structure of the push information of the pre-annotated live broadcast room; A generation module is used to input the large language model input data into a pre-trained large language model to obtain the push information of the live broadcast room; wherein the large language model is based on a general language model and is trained by adjusting parameters using the text content of the push information of the pre-annotated live broadcast room.

11. A live broadcast system, It is characterized in that include: An anchor terminal, an audience terminal and a live broadcast server; wherein the anchor terminal and the audience terminal are respectively connected to the live broadcast server via a communication network; The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server; The live broadcast server is used to forward the live broadcast video stream of the anchor to the audience end, and receive the analysis data request from the audience end, and use the method for generating live broadcast room push information according to any one of claims 1 to 9 to generate push information and display it; The audience end is used for audience users who access the live broadcast room, receive the live video stream of the anchor for playback, and display the push information.

12. An electronic device, It is characterized in that The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the method for generating live broadcast room push information as described in any one of claims 1-9.

13. A computer-readable storage medium, It is characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded by the processor and execute the steps of the method for generating live broadcast room push information as described in any one of claims 1-9.

Images