Live broadcast room video playing method and device, live broadcast system, electronic equipment and medium

Abstract

The invention relates to a live broadcast room video playing method, a live broadcast room video playing device, a live broadcast system, electronic equipment and a computer readable storage medium. The method comprises the following steps: after a user enters a home page of a live broadcast room, displaying a live broadcast room cover and an AI assistant control of each live broadcast room; when the live broadcast room cover browsed on the live broadcast room home page has the highlight content video, displaying a play button of the highlight content video; in response to an operation of dragging the play button to the live broadcast room cover by the user, obtaining a target live broadcast room cover where the user releases the play button; after the user releases the play button, the AI assistant control loads and plays a highlight content video generated by the live broadcast room of the target live broadcast room cover; according to the technical scheme, the user can quickly know the live broadcast room information, the probability that the user enters the live broadcast room to watch live broadcast is improved, and the daily live volume of the live broadcast platform is improved.

Description

Technical Field

[0001] The present application relates to the technical field of network live broadcasting, and in particular to a live broadcast room video playback method, a live broadcast room video playback device, a live broadcast system, an electronic device and a computer-readable storage medium. Background Art

[0002] With the development of network technology, online live streaming has been favored by most users. With its intuitive, immediacy and interactivity in content and form, online live streaming has played an important role in promoting flexible employment, promoting economic and social development, and enriching the spiritual and cultural life of the people. Anchors can better display their talents in live broadcasts, thereby helping more anchors realize their self-worth.

[0003] On current live streaming platforms, live streaming rooms usually provide highlight moment videos to recommend the host of the room, so that users can quickly learn about the host and his / her live content. For example, through the display of talent videos and honor videos, it helps to guide users to enter their favorite live streaming rooms to watch the live broadcast, thereby increasing the chances of attracting users to watch the live content. Currently, on live streaming platforms, in order to obtain the highlight content of a live streaming room, users usually need to actively enter the live streaming room and then search for the entrance to play the highlight content of the live streaming room. The high cost of operation interaction is not conducive to users quickly understanding the information of the live streaming room, restricting users' willingness to watch the live streaming room and affecting the daily active users of the live streaming platform. Summary of the invention

[0004] Based on this, it is necessary to provide a live broadcast room video playback method, device, live broadcast system, electronic device and computer-readable storage medium, which can help users quickly understand the highlight content of the live broadcast room, improve the interactivity of the live broadcast room and the daily activity of the live broadcast platform.

[0005] A live broadcast room video playback method, comprising:

[0006] After the user enters the homepage of the live broadcast room, the cover of each live broadcast room and the AI ​​assistant control are displayed;

[0007] When there is a highlight content video on the live broadcast room cover viewed on the live broadcast room homepage, the play button of the highlight content video is displayed;

[0008] In response to the user dragging the play button to the live broadcast room cover, obtaining the target live broadcast room cover when the user releases the play button;

[0009] After the user releases the play button, the AI ​​assistant control loads the highlight content video generated by the live broadcast room of the target live broadcast room cover and plays it.

[0010] In one embodiment, the live broadcast room video playback method further includes:

[0011] In response to the user sliding the cover of the live broadcast room for browsing on the live broadcast room homepage, sliding to display the list of live broadcast room covers;

[0012] Check whether there is a highlight content video on the live broadcast room cover within the visible range on the current live broadcast room homepage;

[0013] If there is a highlight content video on the cover of any live broadcast room, the play button of the highlight content video will be displayed, otherwise the play button will not be displayed.

[0014] In one embodiment, the live broadcast room video playback method further includes:

[0015] When the user clicks and drags the play button, the AI ​​assistant control will outline and illuminate the border of the live broadcast room cover with highlight content videos within the visible range on the current live broadcast room homepage.

[0016] In one embodiment, the live broadcast room video playback method further includes:

[0017] When the user releases the play button and there is no highlight content video on the cover of the target live broadcast room, the AI ​​assistant control connects to the live broadcast room corresponding to the cover of the target live broadcast room, and calls the preset highlight content video production method to capture the highlight content video from the live video screen of the live broadcast room for caching and playback.

[0018] In one embodiment, the highlight content video production method includes:

[0019] Intercepting a highlight video stream of a preset duration from the talent moment video of the anchor in the live broadcast room, and generating a highlight content video of a preset duration based on the highlight video stream, public screen speech and anchor information;

[0020] Get the live broadcast host's video on the hourly ranking list, start capturing the top1 ranking video of a preset duration when the host triggers the top1 ranking, and generate a highlight content video of a preset duration based on the public screen speech of the top1 ranking video, the host information, and the hourly ranking banner;

[0021] or

[0022] When the host in the live broadcast room receives a big gift, the highest-priced gift MP4 animation is selected, and the big gift video after the audience user gives the gift is captured. A highlight content video with the same length as the gift animation is generated based on the big gift video, gift MP4 animation and host information area.

[0023] In one embodiment, the live broadcast room video playback method further includes:

[0024] When playing highlight content videos, the AI ​​assistant reads the chat interaction information between the host and the audience in the live broadcast room;

[0025] Acquire chat text data according to the chat interaction information and generate recommended speech content using a pre-trained large language model;

[0026] When the user clicks on the recommended speech content, the recommended speech content is sent to the public screen of the live broadcast room for display.

[0027] In one embodiment, acquiring chat text data according to the chat interaction information and generating recommended speech content using a pre-trained large language model includes:

[0028] The live broadcast host's voice signal is converted into voice text information, and the recommended speech content sent by the live broadcast audience on the public screen is generated into public screen text information;

[0029] Aligning the voice text information and the public screen text information according to the timestamps to obtain chat text data;

[0030] Inputting the chat text data into a prompt template trained based on the text structure of the push information in the live broadcast room to obtain large language model input data;

[0031] The large language model input data is input into a large language model trained based on the text content of the push information in the live broadcast room to obtain recommended speech content.

[0032] In one embodiment, the method for the AI ​​assistant control to play the highlight content video includes:

[0033] Placing the highlight content video on the cover of the live broadcast room for playback;

[0034] Placing the highlight content video on a floating window for playback;

[0035] or

[0036] Place the highlight content video in the half-window opened by the AI ​​assistant control for playback;

[0037] In one embodiment, placing the highlight content video in a half-window opened by the AI ​​assistant control for playback includes:

[0038] Pull up the half-window through the AI ​​assistant control;

[0039] A spherical body is displayed in the half-window, a plurality of highlight content videos are loaded, and each highlight content video is placed in a nine-square grid of the spherical body for display.

[0040] In one embodiment, the live broadcast room video playback method further includes:

[0041] When playing the highlight content video, the AI ​​assistant obtains the live broadcast related information of the current live broadcast room, and places the live broadcast related information on the upper layer of the highlight content video for playback;

[0042] Among them, the live broadcast related information includes: at least one of the public screen information of the live broadcast room, anchor information, gift information, and large gift playback content.

[0043] In one embodiment, the live broadcast room video playback method further includes:

[0044] When the user slides up and down on the homepage of the live broadcast room to browse the live broadcast room cover, the AI ​​assistant obtains the sliding direction and the target live broadcast room cover within the visible range, and extracts the video link of the highlight content video generated by the live broadcast room corresponding to each target live broadcast room cover and puts it into the cache queue;

[0045] Through the AI ​​assistant, the order of video links in the cache queue is adjusted according to the up and down sliding direction, and the highlight content video generated by the live broadcast room corresponding to the target live broadcast room cover is selected according to the order.

[0046] A live video playback device, comprising:

[0047] The display module is used to display the live broadcast room cover and AI assistant controls of each live broadcast room after the user enters the live broadcast room homepage;

[0048] The browsing module is used to display the play button of the highlight content video when there is a highlight content video on the cover of the live broadcast room browsed on the homepage of the live broadcast room;

[0049] The drag module is used to obtain the target live broadcast room cover when the user releases the play button in response to the user dragging the play button to the live broadcast room cover;

[0050] The playback module is used for the AI ​​assistant control to load and play the highlight content video generated by the live broadcast room of the target live broadcast room cover after the user releases the play button.

[0051] A live broadcast system, comprising: an anchor terminal, an audience terminal and a live broadcast server; wherein the anchor terminal and the audience terminal are respectively connected to the live broadcast server via a communication network;

[0052] The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server;

[0053] The live broadcast server is used to forward the live broadcast video stream of the anchor to the audience end and generate the highlight content video of the live broadcast room;

[0054] The audience end is used for audience users who access the live broadcast room, receive the live video stream of the anchor for playback, and use the live broadcast room video playback method to play the highlight content video.

[0055] An electronic device, comprising:

[0056] one or more processors;

[0057] Memory;

[0058] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the live broadcast room video playback method.

[0059] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the live video playback method.

[0060] The technical solution of the present application is to display the live broadcast room cover and AI assistant control of each live broadcast room after the user enters the live broadcast room homepage; when there is a highlight content video on the live broadcast room cover browsed on the live broadcast room homepage, the play button of the highlight content video is displayed; the user drags the play button to the target live broadcast room cover and releases it, and after the user releases the play button, the AI ​​assistant control loads the highlight content video generated by the live broadcast room on the target live broadcast room cover and plays it; this technical solution is based on the user dragging the play button of the AI ​​assistant control to the target live broadcast room cover to automatically obtain and play the highlight content video, thereby reducing the user's interaction cost for viewing the highlight content video, facilitating the user to quickly understand the live broadcast room information, increasing the probability of users entering the live broadcast room to watch the live broadcast, and increasing the daily active users of the live broadcast platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a schematic diagram of an example live broadcast service application scenario;

[0062] Figure 2 is a flow chart of a live broadcast room video playback method according to an embodiment;

[0063] Figure 3 This is a schematic diagram of the homepage of a live broadcast room;

[0064] Figure 4 is a schematic diagram of an example play button;

[0065] Figure 5 This is an example of an AI assistant generating a highlight content video flow chart;

[0066] Figure 6 This is an example of a long press and drag diagram of the play button;

[0067] Figure 7 This is a schematic diagram of an example of a floating window playing a highlight content video;

[0068] Figure 8 This is an example of a half-window displaying a video with highlight content;

[0069] Fig. 9 This is a schematic diagram of a video showing highlight content on a spherical surface;

[0070] Fig.10 This is a schematic diagram of an example of playing a highlight content video by arranging sub-windows;

[0071] Fig.11 This is a sample UI diagram showing a highlight content video;

[0072] Fig.12 is a schematic diagram of an example of full-screen playback of a highlight content video;

[0073] Fig.13 is a schematic diagram of the structure of a live broadcast room video playback device according to an embodiment;

[0074] Fig.14 This is a schematic diagram of the live broadcast system structure;

[0075] Fig.15 is a block diagram of an example electronic device. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0077] In the embodiments of the present application, the words "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. "at least one" means one or more, and "multiple" means two or more. For example, multiple objects refer to two or more objects. "Include" or "comprises" and similar words mean that the information appearing before "include" or "comprises" covers the information listed after "include" or "comprises" and its equivalents, and does not exclude other information. "And / or" mentioned in the embodiments of the present application indicates that three types of relationships may exist. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0078] The technical solution provided in the embodiments of the present application can be applied to Figure 1 In the application scenario of the method related to this application shown in FIG. Figure 1 This is a schematic diagram of an example of a live broadcast service application scenario. The live broadcast system may include a live broadcast server, an anchor terminal and an audience terminal. The anchor terminal and the audience terminal communicate with the live broadcast server through a communication network, so that the anchor at the anchor terminal and the audience users at the audience terminal can perform real-time network live broadcast. Among them, for the anchor terminal and the audience terminal, their terminal devices can be but are not limited to various personal computers, laptops, smart phones and tablets. The live broadcast server can be implemented by an independent server or a server cluster composed of multiple servers.

[0079] In order to reduce the interactive operation cost between users and the live broadcast room and facilitate users to quickly understand the relevant information of the live broadcast room, the technical solution of the embodiment of the present application uses an AI assistant to assist the live broadcast room in live broadcasting. When the user browses the cover of the live broadcast room on the homepage of the live broadcast room, the play button of the AI ​​assistant control is dragged to assist in playing the highlight content video of the live broadcast room, and the user is provided with recommended speech content assistance, as well as a more interactive playback interface, thereby realizing convenient interaction between users and the live broadcast room. Users can quickly understand the relevant information of the live broadcast room, enhancing the interactivity of the live broadcast room.

[0080] refer to Figure 2 As shown, Figure 2 The flowchart of a live video playback method of an embodiment includes the following steps:

[0081] Step S10, after the user enters the live broadcast room homepage, the live broadcast room cover and AI assistant controls of each live broadcast room are displayed.

[0082] In this step, when the user enters the homepage of the live broadcast room for the first time, the terminal interface displays the cover of each live broadcast room, and displays the AI ​​assistant at the set position to guide the user to use it. Among them, the AI ​​assistant is a live broadcast auxiliary tool configured based on AI artificial intelligence technology. It is configured to process voice, text, pictures, etc., and can connect search, large models, intelligent agents, etc., to meet the needs of diversified search and data processing capabilities.

[0083] For example, when the client detects that the user has entered the homepage of the live broadcast room, the default AI assistant is displayed on the display interface. Figure 3 As shown, Figure 3This is a page diagram of an example live broadcast room homepage, which includes six live broadcast room covers, each of which can display a picture or the current live broadcast screen; the AI ​​assistant control is displayed in the lower right corner of the homepage page, and the AI ​​assistant image can be a two-dimensional avatar in the default state; the AI ​​assistant can set a variety of functions according to needs, such as analyzing the live broadcast room and its live video.

[0084] Step S20, when there is a highlight content video on the cover of the live broadcast room browsed on the homepage of the live broadcast room, a play button of the highlight content video is displayed.

[0085] In this step, when the user browses the live broadcast room cover on the live broadcast room homepage, if there is a highlight content video on the live broadcast room cover within the visible range on the live broadcast room homepage, a play button for the highlight content video is displayed on the AI ​​assistant control. For example, Figure 4 As shown, Figure 4 It is a schematic diagram of an example play button, which can be displayed in the form of a play icon, which can be a floating button generated by an AI assistant control, or a floating button transformed from an AI assistant control; preferably, a prompt message that allows the user to drag the button to the cover of the live broadcast room can be generated next to the play button, guiding the user to drag the play button to the cover of the target live broadcast room to watch the highlight content video.

[0086] In one embodiment, when the AI ​​assistant control determines whether to display the play button, it mainly includes the following steps:

[0087] In response to the user's operation of sliding the live broadcast room cover on the live broadcast room homepage for browsing, the live broadcast room cover list is displayed by sliding; it is detected whether there is a highlight content video on the live broadcast room cover within the visible range on the current live broadcast room homepage; if there is a highlight content video on any live broadcast room cover, the play button of the highlight content video is displayed, otherwise the play button is not displayed.

[0088] Specifically, the user slides the cover of the live broadcast room on the homepage to browse, and the homepage of the live broadcast room will slide to display the list of live broadcast room covers. At this time, the AI ​​assistant controls whether there are highlight content videos pre-stored in the live broadcast room cover within the visible range on the current live broadcast room homepage. If so, the play button of the highlight content video will be displayed to prompt the user to watch it. If not, the play button will not be displayed.

[0089] For example, for highlight content videos, highlight content videos generally refer to the highlight moments of the host's live broadcast, the host's talent moments, honor moments and other related video content; video content generally includes live video streams, public screen speeches and host information, etc., which can be generated according to pre-set strategies, or generated in real time by an AI assistant.

[0090] For example, a solution for an AI assistant to generate a highlight content video in real time; Figure 5 As shown, Figure 5 This is an example of an AI assistant generating a highlight content video flow chart. For example, for a talent moment video, you can capture a 10-second highlight video stream from the host's highlight performance of more than 30 seconds, and combine the live video stream, the user's public screen speech flow, and the host's information area to generate a 10-second talent moment video; for another example, for an honor moment video, through the user's video of obtaining the top 1 of the hourly list, start capturing the corresponding video stream when the top 1 is triggered, and combine the live video stream, the user's public screen speech flow, the host's information area, and the hourly list banner to generate a 5-second honor moment video; for another example, for a gift-giving honor moment video, it can be executed when the host obtains a big gift, select the gift MP4 animation with the highest price, capture the video starting 3 seconds after the user gives the gift, and combine the live video stream, the gift MP4 animation, and the host's information area to generate a gift-giving honor moment video with the same length as the gift animation.

[0091] Step S30, in response to the user dragging the play button to the live broadcast room cover, obtain the target live broadcast room cover where the user is when the play button is released.

[0092] In this step, when the user wants to play the highlight content video of a live room on the homepage of the live room, he can drag the play button to the live room cover. When the user releases the play button, the AI ​​assistant control will determine the target live room cover when the user releases the play button.

[0093] In one embodiment, Figure 6 As shown, Figure 6 It is an example of a long press and drag diagram of a play button; when the user clicks and drags the play button, the AI ​​assistant control will outline and illuminate the borders of the live broadcast room covers that have highlight content videos within the visible range on the current live broadcast room homepage; this can indicate to the user which live broadcast room covers have pre-stored highlight content videos.

[0094] In one embodiment, when the user drags the play button to the target live broadcast room cover area, the three stages of starting, moving and ending the dragging of the play button can be established through the HTML5 drag API (Application Programming Interface), and an event listener can be added; then the event of the user long pressing and dragging the play button to the live broadcast room cover is monitored; if the live broadcast room cover in the current live broadcast room cover area is pre-configured to support the playback of highlight content videos, the live broadcast room cover is set to a responsive state, and when the user releases the long press of the play button, the target live broadcast room cover where the long press operation is released and its corresponding target live broadcast room are determined.

[0095] As in the technical solution of the above embodiment, the user can long press and drag the play button to the cover of the live broadcast room. The AI ​​assistant control will respond to the user's dragging operation in the background and synchronously display the picture effect of the play button moving to the cover of the live broadcast room in real time, thereby visually presenting the target live broadcast room where the highlight content video can be watched to the user.

[0096] Step S40, after the user releases the play button, the AI ​​assistant control loads the highlight content video generated by the live broadcast room of the target live broadcast room cover and plays it.

[0097] In this step, after the user releases the play button, the AI ​​assistant control loads the highlight content video generated by the live broadcast room on the cover of the target live broadcast room, and uses the playback window to play the corresponding highlight content video. The user can quickly complete the playback interaction process of the highlight content video of the target live broadcast room by dragging and dropping, shortening the UI interaction path between the user and the live broadcast room, reducing the user interaction cost, and making it easier for the user to quickly understand the live broadcast information of the live broadcast room.

[0098] In one embodiment, when the user releases the play button and there is no highlight content video on the cover of the target live broadcast room, the AI ​​assistant control connects to the live broadcast room corresponding to the cover of the target live broadcast room, and calls a preset highlight content video production method to capture the highlight content video from the live video screen of the live broadcast room for caching and playback.

[0099] Specifically, if the user drags the play button to the cover of a live broadcast room where there is no highlight content video and releases the play button, since there is no highlight content video to play, the AI ​​assistant control connects to the corresponding live broadcast room, calls the preset highlight content video production method, and captures a highlight content video from the current live video screen of the live broadcast room for caching and playback, so that users can timely understand the live broadcast information of the live broadcast room.

[0100] In order to make the technical solution of the present application more clear, more embodiments of playing highlight content videos are described below.

[0101] In one embodiment, when the AI ​​assistant control plays the highlight content video, it can play the highlight content video in a pop-up floating window; Figure 7 As shown, Figure 7 The figure is a schematic diagram of an example of a floating window playing a highlight content video. The floating window is displayed on the homepage of the live broadcast room, and then the highlight content video is played in the pop-up floating window.

[0102] In one embodiment, the AI ​​assistant control can also pull up a half-window on the live broadcast room page, and then place the loaded highlight content video in the half-window for playback; Figure 8 As shown, Figure 8This is an example of a half-window window displaying a highlight content video. At this time, the user can also click on the cover of the live broadcast room to enter the current live broadcast room. For example, the AI ​​assistant control can place multiple highlight content videos such as talent moments and honor moments in the form of sub-windows on the half-window window for playback.

[0103] Furthermore, during the process of playing the highlight content video in the half-window, when the user clicks on the live broadcast room homepage to browse the covers of other live broadcast rooms, the half-window of the AI ​​assistant control is folded. At this time, the AI ​​assistant control can be displayed as a play button in the initial state.

[0104] As in the solution of the above-mentioned embodiment, based on the AI ​​assistant automatically assisting the user to obtain and play the highlight content video, the user can automatically obtain the highlight content video, thereby reducing the operation interaction cost, facilitating the user to quickly understand the live broadcast information of the live broadcast room, increasing the user's chance of watching the live broadcast room, and increasing the daily activity of the live broadcast platform.

[0105] In one embodiment, when the AI ​​assistant control plays a highlight content video in a half-window, a spherical body can be displayed in the half-window, multiple highlight content videos can be loaded, and each highlight content video can be placed in a nine-square grid of the spherical body for display. Fig. 9 As shown, Fig. 9 The figure is a schematic diagram of an example of displaying highlight content videos on a sphere. As shown in the figure, when playing multiple highlight content videos, the highlight content videos can be placed on a nine-square grid on the sphere for display. The user can slide the sphere left or right or up and down to select the highlight content video to watch.

[0106] In one embodiment, when the AI ​​assistant control plays the highlight content video, it can be placed on the cover of the live broadcast room for playback; specifically, when loading the highlight content video and placing it in the half-window for playback, multiple highlight content videos can be loaded, and the highlight content videos can be placed in sub-windows arranged on the half-window for playback; that is, multiple arranged sub-windows are placed in the half-window, and then the multiple highlight content videos are played in each of the sub-windows, and the user can swipe left and right to select the highlight content video to watch.

[0107] like Fig.10 As shown, Fig.10 This is a schematic diagram of an example of playing highlight content videos by arranging sub-windows. In the figure, three sub-windows are used to play multiple honor moment videos. Considering the sub-window size setting, two sub-windows can be displayed simultaneously in the visible range. When other sub-windows need to be selected, they can be selected by sliding left and right.

[0108] In one embodiment, when playing a highlight content video, in order to facilitate users to better participate in the interaction in the current live broadcast room, when playing a highlight content video, the AI ​​assistant control can also generate and display speech content. When the user clicks on the speech content, the speech content is sent to the public screen of the live broadcast room for display.

[0109] Accordingly, the live broadcast room video playback method of this embodiment may further include:

[0110] The chat interaction information between the host and the audience in the live broadcast room is read through the AI ​​assistant; the chat text data is obtained based on the chat interaction information and the recommended speech content is generated using the pre-trained large language model; when the user clicks on the recommended speech content, the recommended speech content is sent to the public screen of the live broadcast room for display.

[0111] The solution of the above-mentioned embodiment can enable users to quickly interact with the live broadcast room, thereby improving the interactivity between users and the live broadcast room.

[0112] In one embodiment, when the chat text data is acquired according to the chat interaction information and the recommended speech content is generated using the pre-trained large language model, the following steps may be included:

[0113] (1) Convert the host voice signal of the live broadcast host side into voice text information, and generate public screen text information from the recommended speech content sent by the audience side of the live broadcast room on the public screen; for example, collect the sound signal of the live broadcast host side, separate the sound signal from the human voice and remove the background music to obtain the host voice signal; extract the speech content input by the audience side of the live broadcast room on the public screen, and filter the non-text symbols in the speech content to obtain the speech text; obtain the current live broadcast room chat topic content by collecting the chat interaction information between the host side and the audience side. Use speech recognition technology to convert the voice signal into voice text information, convert the speech text input by the audience on the public screen into text information, and perform downsampling and other operations on the speech to obtain high-quality public screen text information.

[0114] (2) Aligning the voice text information and the public screen text information according to the timestamp to obtain chat text data; illustratively, aligning the voice text information and the public screen text information according to the timestamp to obtain chat text data, thereby obtaining high-quality chat text data and ensuring that the order of relevant data input into the large language model is correct.

[0115] (3) Inputting the chat text data into a prompt template trained based on the text structure of the push information of the live broadcast room to obtain input data for the large language model; illustratively, designing an initial template for processing the prompt content format according to the information to be pushed, performing format optimization training on the initial template based on the text structure of the push information of the live broadcast room that is pre-annotated, performing format optimization on the initial template during the test process to obtain a prompt template, inputting the chat text data into the prompt template to obtain input data for the large language model, and using the optimized prompt template to first process the data input into the large language model, thereby guiding the large language model to generate high-quality output results.

[0116] (4) Inputting the large language model input data into a large language model trained based on the text content of the push information of the live broadcast room to obtain recommended speech content; illustratively, parameter adjustment training is performed based on a general language model and using the pre-annotated text content of the push information of the live broadcast room to obtain a large language model, and the large language model input data is input into the large language model to obtain recommended speech content.

[0117] Exemplarily, a general language model can be used to perform parameter fine-tuning training. Fine-tuning refers to making appropriate adjustments to some parameters of the general language model so that the large language model not only utilizes the powerful basic performance of the general language model and has powerful information processing capabilities, but also can be combined with the personalized needs of the live broadcast scene in the live broadcast room.

[0118] For example, first add a bypass consisting of a dimension reduction matrix A and a dimension increase matrix B to the pre-trained model's pre-trained weight matrix W, and the matrix product BA of the two bypass matrices is consistent with the dimension of the pre-trained weight matrix W; then, in the training phase, fix the pre-trained model's pre-trained weight matrix W, and only train the dimension reduction matrix A and the dimension increase matrix B. The dimension reduction matrix A is randomly initialized to a Gaussian distribution, and the dimension increase matrix B is initialized to 0. When the model is output, the matrix product BA of the dimension reduction matrix and the dimension increase matrix is ​​superimposed with the pre-trained weight matrix W. Assume that the pre-trained weight matrix of the large language model is The update can then be expressed as:

[0119] W0+ΔW=W0+BA

[0120] Among them, ΔW represents the update adjustment of the pre-trained weight matrix, Rank r<<min(d,k); During the training phase, keep the pre-training weight matrix W0 fixed, and adjust the dimension reduction matrix A and the dimension increase matrix B.

[0121] Finally, in the inference phase, the model input is multiplied by the superposition parameter as the model output; in forward inference, the pre-trained weight matrix W0 and the update adjustment ΔW are multiplied by the same model input x and finally added together, as shown in the following expression:

[0122] h=W0x+ΔWx=W0x+BAx

[0123] Among them, h is the output of the large language model after fine-tuning. Since the number of parameters of the matrix product BA that needs to be fine-tuned (d×r+r×k) is much smaller than the number of parameters of the pre-trained weight matrix W0 (d×k), it is possible to complete the fine-tuning of the large language model by training only a few parameters.

[0124] As in the technical solution of the above-mentioned embodiment, a large language model is obtained by parameter fine-tuning training. Since the parameter fine-tuning training of the large language model is based on the chat interaction data of the real live broadcast room, it has the advantages of fewer parameters, saving computing resources, strong generalization, and good original model knowledge inheritance. Therefore, the recognition accuracy of the large language model can be improved, and the quality of the push information generated by the large language model can be improved.

[0125] like Fig.11 As shown, Fig.11 This is an example of a UI diagram showing a highlight content video. As shown in the figure, the AI ​​assistant can generate comments such as "Is this prop amazing?", "The host is really smart", "I want to know too", "Come in quickly" and so on. Users can click on these comments to make comments. For the generated comments, the AI ​​assistant can be used to identify the application scenario of the current live broadcast room, so as to obtain comments that are suitable for the application scenario. In addition, the half-window of the AI ​​assistant can also display other summary information about the highlight content video.

[0126] When the user needs to exit the half-window display of the highlight content video, the client can respond to the user's operation of sliding down to close the half-window window and close the half-window window of the AI ​​assistant control to watch the live broadcast. In addition, it can also be closed by sliding down. At this time, the client responds to the user's operation of sliding down to close the half-window window and closes the half-window window of the AI ​​assistant to watch the live broadcast.

[0127] In one embodiment, in order to facilitate users to have a deeper understanding of the real-time dynamics of the live broadcast room when watching the highlight content video, according to the solution of this embodiment, when playing the highlight content video, the AI ​​assistant obtains the public screen information, anchor information, gift information, big gift playback content and other live broadcast related information of the current live broadcast room, and places these live broadcast related information on the upper layer of the highlight content video for playback; for example, the user clicks the zoom button to switch to full-screen playback, and the AI ​​assistant obtains the public screen information, anchor information, gift information, big gift playback content, etc. of the current live broadcast room and places them on the upper layer of the highlight content video for playback.

[0128] like Fig.12 As shown, Fig.12This is a schematic diagram of an example of full-screen playback of a highlight content video. The figure shows three different types of highlight content videos. For example, the left figure is a schematic diagram of a highlight moment video in the game, and the middle and right figures are schematic diagrams of honor moment videos. For live broadcast related information, it can replicate the public screen information of the current live broadcast room through the AI ​​assistant, and use the AI ​​assistant to pull the anchor information and the gift information of the live broadcast room, obtain the big gift information, and play the big gift animation on the upper layer of the highlight content video playback, so that users can obtain the dynamic information of the current live broadcast room when watching the highlight content video, thereby attracting users to enter the live broadcast room to watch the live broadcast and interact with the live broadcast, thereby improving the interactivity between users and the live broadcast room.

[0129] In one embodiment, in order to adapt to the user's operation of playing the highlight content video when browsing the cover of the live broadcast room, the AI ​​assistant control may include the following when generating the highlight content video:

[0130] When the user slides up and down on the homepage of the live broadcast room to browse the live broadcast room cover, the AI ​​assistant obtains the sliding direction and the target live broadcast room cover within the visible range, and extracts the video links of the highlight content videos generated by the live broadcast room corresponding to each target live broadcast room cover and puts them into the cache queue; the AI ​​assistant adjusts the order of the video links in the cache queue according to the up and down sliding direction, and selects the highlight content video generated by the live broadcast room corresponding to the target live broadcast room cover according to the order.

[0131] Specifically, by caching video links through the cache queue, the AI ​​assistant can identify the target live broadcast room cover within the visible range according to the user's sliding direction, and sort the video links of its highlight content videos. During the user's continuous sliding and browsing process, the user's browsing intention is predicted, and the sorting of video links in the cache queue is adjusted in real time, so that the appropriate highlight content video can be quickly loaded for playback, ensuring the playback speed, thereby obtaining a better interactive experience, thereby enhancing the interactivity between the user and the live broadcast room.

[0132] An embodiment of a live broadcast room video playback device is described below.

[0133] like Fig.13 As shown, Fig.13 The structure diagram of a live video playback device in an embodiment is as follows:

[0134] Display module 10, used to display the live broadcast room cover and AI assistant controls of each live broadcast room after the user enters the live broadcast room homepage;

[0135] The browsing module 20 is used to display a play button of the highlight content video when there is a highlight content video on the cover of the live broadcast room browsed on the homepage of the live broadcast room;

[0136] The drag module 30 is used to obtain the target live broadcast room cover when the user releases the play button in response to the user dragging the play button to the live broadcast room cover;

[0137] The playback module 40 is used for the AI ​​assistant control to load and play the highlight content video generated by the live broadcast room of the target live broadcast room cover after the user releases the play button.

[0138] The live broadcast room video playback device of this embodiment can execute a live broadcast room video playback method provided by the embodiment of the present application, and the implementation principle is similar. The actions performed by each module in the live broadcast room video playback device in each embodiment of the present application correspond to the steps in the live broadcast room video playback method in each embodiment of the present application. For the detailed functional description of each module of the live broadcast room video playback device, please refer to the description in the corresponding live broadcast room video playback method shown in the previous text, which will not be repeated here.

[0139] An embodiment of the live broadcast system is described below.

[0140] The live broadcast system of this embodiment refers to Fig.14 As shown, Fig.14 This is a schematic diagram of the structure of an example live broadcast system. The system framework may include an anchor terminal, a live broadcast server and an audience terminal. The audience terminal and the anchor terminal may enter the live broadcast platform through a client installed on an electronic device. For example, the anchor terminal and the audience terminal may be computer devices, such as PDA, smart phone, tablet computer, desktop computer or laptop computer, etc., which are not limited to this. They may also be software modules of an application program. The live broadcast server may be implemented by an independent server or a server cluster composed of multiple servers. The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server; the live broadcast server is used to forward the anchor's live video stream to the audience terminal, and use the live broadcast room video playback method of any of the above embodiments to play the highlight content video; specifically, the live broadcast server uses the AI ​​assistant on the client to provide front-end services, and based on the AI ​​server to provide back-end analysis and processing capabilities, it can provide active auxiliary functions for users to watch the live content of the live broadcast room, so that users can quickly understand the relevant information of the live broadcast room and guide users to enter the live broadcast room to watch the live broadcast; the audience terminal is used to access the audience users in the live broadcast room, receive the anchor's live video stream for playback, and play the highlight content video through the AI ​​assistant.

[0141] The live broadcast system of this embodiment automatically assists users to obtain and play highlight content videos based on an AI assistant, thereby reducing the interaction cost of users needing to play highlight content videos, facilitating users to quickly understand the live broadcast room, increasing users' willingness to watch the live broadcast room, and increasing the daily activity of the live broadcast platform.

[0142] Embodiments of electronic devices and computer-readable storage media are described below.

[0143] The present application provides a technical solution for an electronic device to implement functions related to a live broadcast room video playback method. The electronic device of this embodiment includes one or more processors, a memory; one or more applications, wherein one or more applications are stored in the memory and configured to be executed by one or more processors, and one or more programs are configured for the steps of the live broadcast room video playback method of any embodiment.

[0144] like Fig.15 As shown, Fig.15 1 is a block diagram of an exemplary electronic device; the electronic device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. The electronic device 100 may include one or more of the following components: a processing component 102, a memory 104, a power component 106, a multimedia component 108, an audio component 109, an input / output (I / O) interface 112, a sensor component 114, and a communication component 116.

[0145] The processing component 102 generally controls the overall operation of the electronic device 100, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.

[0146] The memory 104 is configured to store various types of data to support operations in the electronic device 100, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0147] The power supply component 106 provides power to various components of the electronic device 100 .

[0148] The multimedia component 109 includes a screen that provides an output interface between the electronic device 100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). In some embodiments, the multimedia component 108 includes a front camera and / or a rear camera.

[0149] The audio component 109 is configured to output and / or input audio signals.

[0150] I / O interface 112 provides an interface between processing component 102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0151] The sensor component 114 includes one or more sensors for providing various aspects of status assessment for the electronic device 100. The sensor component 114 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact.

[0152] The communication component 116 is configured to facilitate wired or wireless communication between the electronic device 100 and other devices. The electronic device 100 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof.

[0153] The present application provides a technical solution of a computer-readable storage medium, which is used to implement functions related to a live video playback method. The computer-readable storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded by a processor and executes the live video playback method of any embodiment.

[0154] In an exemplary embodiment, the computer-readable storage medium may be a non-temporary computer-readable storage medium including instructions, such as a memory including instructions. For example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0155] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for playing live video, characterized in that: include: After the user enters the homepage of the live broadcast room, the cover of each live broadcast room and the AI ​​assistant control are displayed; When there is a highlight content video on the live broadcast room cover viewed on the live broadcast room homepage, the play button of the highlight content video is displayed; In response to the user dragging the play button to the live broadcast room cover, obtaining the target live broadcast room cover when the user releases the play button; After the user releases the play button, the AI ​​assistant control loads the highlight content video generated by the live broadcast room of the target live broadcast room cover and plays it.

2. The method for playing live video in a live studio according to claim 1, characterized in that: Also includes: In response to the user sliding the cover of the live broadcast room for browsing on the live broadcast room homepage, sliding to display the list of live broadcast room covers; Check whether there is a highlight content video on the live broadcast room cover within the visible range on the current live broadcast room homepage; If there is a highlight content video on the cover of any live broadcast room, the play button of the highlight content video will be displayed, otherwise the play button will not be displayed.

3. The method for playing live video in a live studio according to claim 2, characterized in that: Also includes: When the user clicks and drags the play button, the AI ​​assistant control will outline and illuminate the border of the live broadcast room cover with highlight content videos within the visible range on the current live broadcast room homepage.

4. The live video playback method according to claim 1, characterized in that: Also includes: When the user releases the play button and there is no highlight content video on the cover of the target live broadcast room, the AI ​​assistant control connects to the live broadcast room corresponding to the cover of the target live broadcast room, and calls the preset highlight content video production method to capture the highlight content video from the live video screen of the live broadcast room for caching and playback.

5. The method for playing live video in a live studio according to claim 4, characterized in that: The highlight content video production method comprises: Intercepting a highlight video stream of a preset duration from the talent moment video of the anchor in the live broadcast room, and generating a highlight content video of a preset duration based on the highlight video stream, public screen speech and anchor information; Get the live broadcast host's video on the hourly ranking list, start capturing the top1 ranking video of a preset duration when the host triggers the top1 ranking, and generate a highlight content video of a preset duration based on the public screen speech of the top1 ranking video, the host information, and the hourly ranking banner; or When the host in the live broadcast room receives a big gift, the highest-priced gift MP4 animation is selected, and the big gift video after the audience user gives the gift is captured. A highlight content video with the same length as the gift animation is generated based on the big gift video, gift MP4 animation and host information area.

6. The live video playback method according to claim 1, characterized in that: Also includes: When playing highlight content videos, the AI ​​assistant reads the chat interaction information between the host and the audience in the live broadcast room; Acquire chat text data according to the chat interaction information and generate recommended speech content using a pre-trained large language model; When the user clicks on the recommended speech content, the recommended speech content is sent to the public screen of the live broadcast room for display.

7. The live video playback method according to claim 6, characterized in that: Acquiring chat text data according to the chat interaction information and generating recommended speech content using a pre-trained large language model, including: The live broadcast host's voice signal is converted into voice text information, and the recommended speech content sent by the live broadcast audience on the public screen is generated into public screen text information; Aligning the voice text information and the public screen text information according to the timestamps to obtain chat text data; Inputting the chat text data into a prompt template trained based on the text structure of the push information in the live broadcast room to obtain large language model input data; The large language model input data is input into a large language model trained based on the text content of the push information in the live broadcast room to obtain recommended speech content.

8. The live video playback method according to claim 1, characterized in that: The method for the AI ​​assistant control to play the highlight content video includes: Placing the highlight content video on the cover of the live broadcast room for playback; Placing the highlight content video on a floating window for playback; or Place the highlight content video in the half-window opened by the AI ​​assistant control for playback.

9. The method for playing live video in a broadcast room according to claim 6, characterized in that: Placing the highlight content video in the half-window opened by the AI ​​assistant control for playback includes: Pull up the half-window through the AI ​​assistant control; A spherical body is displayed in the half-window, a plurality of highlight content videos are loaded, and each highlight content video is placed in a nine-square grid of the spherical body for display.

10. The live video playback method according to claim 1, characterized in that: Also includes: When playing the highlight content video, the AI ​​assistant obtains the live broadcast related information of the current live broadcast room, and places the live broadcast related information on the upper layer of the highlight content video for playback; Among them, the live broadcast related information includes: at least one of the public screen information of the live broadcast room, anchor information, gift information, and large gift playback content.

11. The live video playback method according to claim 1, characterized in that: Also includes: When the user slides up and down on the homepage of the live broadcast room to browse the live broadcast room cover, the AI ​​assistant obtains the sliding direction and the target live broadcast room cover within the visible range, and extracts the video link of the highlight content video generated by the live broadcast room corresponding to each target live broadcast room cover and puts it into the cache queue; Through the AI ​​assistant, the order of video links in the cache queue is adjusted according to the up and down sliding direction, and the highlight content video generated by the live broadcast room corresponding to the target live broadcast room cover is selected according to the order.

12. A live video playback device, characterized in that: include: The display module is used to display the live broadcast room cover and AI assistant controls of each live broadcast room after the user enters the live broadcast room homepage; The browsing module is used to display the play button of the highlight content video when there is a highlight content video on the cover of the live broadcast room browsed on the homepage of the live broadcast room; The drag module is used to obtain the target live broadcast room cover when the user releases the play button in response to the user dragging the play button to the live broadcast room cover; The playback module is used for the AI ​​assistant control to load and play the highlight content video generated by the live broadcast room of the target live broadcast room cover after the user releases the play button.

13. A live broadcast system, characterized in that: include: An anchor terminal, an audience terminal and a live broadcast server; wherein the anchor terminal and the audience terminal are respectively connected to the live broadcast server via a communication network; The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server; The live broadcast server is used to forward the live broadcast video stream of the anchor to the audience end and generate the highlight content video of the live broadcast room; The audience end is used for audience users accessing the live broadcast room, receiving the live broadcast video stream of the anchor for playback, and using the live broadcast room video playback method described in any one of claims 1-11 to play the highlight content video.

14. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the live broadcast room video playback method described in any one of claims 1-11.

15. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded by the processor and execute the steps of the live video playback method described in any one of claims 1-11.

Images