Live Stream Video Playback Method, Apparatus, Live Streaming System, Electronic Device, and Medium

Abstract

The present application relates to a live broadcast room video playing method, a live broadcast room video playing device, a live broadcast system, an electronic device and a computer-readable storage medium; the method includes: after a user enters the home page of the live broadcast room, displaying the live broadcast room covers of each live broadcast room and an AI assistant control; when there is a highlight content video on the live broadcast room cover browsed on the home page of the live broadcast room, displaying a play button for the highlight content video; in response to an operation of the user dragging the play button to the live broadcast room cover, obtaining the target live broadcast room cover where the user releases the play button; after the user releases the play button, the AI assistant control loads and plays the highlight content video generated by the live broadcast room of the target live broadcast room cover; this technical solution facilitates the user to quickly understand the live broadcast room information, improves the probability of the user entering the live broadcast room to watch the live broadcast, and increases the daily active users of the live broadcast platform.

Description

Technical Field

[0001] This application relates to the field of network live broadcast technology, and in particular to a live broadcast room video playing method, a live broadcast room video playing device, a live broadcast system, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of network technology, network live broadcast has been popular among most users. Network live broadcast plays an important role in promoting flexible employment, promoting economic and social development, and enriching the spiritual and cultural life of the people with its intuitiveness, immediacy, and interactivity in content and form. The anchor can better display his or her talents in the live broadcast, thus realizing self-worth for more anchors.

[0003] In current live broadcast platforms, the live broadcast room usually provides highlight moment videos to recommend the anchor in the live broadcast room, so that users can quickly understand the anchor and his or her live broadcast content in the live broadcast room. For example, through the display of talent videos, honor videos, etc., it helps to guide users into the favorite live broadcast room to watch the live broadcast, increasing the probability of attracting users to watch the live broadcast content. Currently in the live broadcast platform, for users to obtain the highlight content of the live broadcast room, they usually need to actively enter the live broadcast room and then find the entry to play the highlight content of the live broadcast room to operate. The operation interaction cost is relatively high, which is not conducive to users quickly understanding the information of the live broadcast room, restricting the willingness of users to watch the live broadcast room, and also affecting the daily activity of the live broadcast platform. Summary of the Invention

[0004] Based on this, it is necessary to provide a live broadcast room video playing method, device, live broadcast system, electronic device, and computer-readable storage medium, which helps users quickly understand the highlight content of the live broadcast room, improves the interactivity of the live broadcast room and the daily activity of the live broadcast platform.

[0005] A live broadcast room video playing method includes:

[0006] After the user enters the home page of the live broadcast room, display the live broadcast room covers of each live broadcast room and the AI assistant control;

[0007] When there is a highlight content video on the live broadcast room cover browsed on the home page of the live broadcast room, display the play button of the highlight content video;

[0008] In response to the user dragging the play button to the live broadcast room cover, obtain the target live broadcast room cover where the user releases the play button;

[0009] After the user releases the play button, the AI assistant control loads and plays the highlight content video generated by the live broadcast room corresponding to the target live broadcast room cover.

[0010] In one embodiment, the live broadcast room video playing method further includes:

[0011] In response to the operation of the user sliding and browsing the live room cover on the home page of the live room, a list of live room covers is slid and displayed;

[0012] Detect whether there is a high-light content video in the live room covers within the visible range on the current live room home page;

[0013] If there is a high-light content video in any one of the live room covers, display the play button for the high-light content video, otherwise do not display the play button.

[0014] In one embodiment, the live room video playing method further includes:

[0015] When the user clicks and drags the play button, the AI assistant control will perform a glow display with a border on the live room cover where there is a high-light content video within the visible range on the current live room home page.

[0016] In one embodiment, the live room video playing method further includes:

[0017] When there is no high-light content video in the target live room cover where the user releases the play button, the AI assistant control accesses the live room corresponding to the target live room cover, and calls a preset high-light content video production method to intercept a high-light content video from the live video screen of the live room for caching and playing.

[0018] In one embodiment, the high-light content video production method includes:

[0019] Intercept a high-light video stream with a preset duration from the talent moment video of the live room anchor, and generate a high-light content video with a preset duration according to the high-light video stream, public screen speech and anchor information;

[0020] Obtain the video of the live room anchor on the hourly leaderboard, start intercepting the top 1 ranking video with a preset duration when the anchor triggers the top 1 ranking, and generate a high-light content video with a preset duration according to the top 1 ranking video, public screen speech, anchor information and hourly leaderboard banner;

[0021] Or

[0022] When the live room anchor receives a large gift, select the MP4 animation of the gift with the highest price, intercept the large gift video after the audience user gives the gift, and generate a high-light content video with the same length as the gift animation according to the large gift video, gift MP4 animation and anchor information area.

[0023] In one embodiment, the live room video playing method further includes:

[0024] When playing the high-light content video, the AI assistant reads the chat interaction information between the live room anchor and the audience;

[0025] Obtain chat text data according to the chat interaction information and generate recommended speech content using a pre-trained large language model;

[0026] When the user clicks on the recommended speech content, send the recommended speech content to the public screen of the live broadcast room for display.

[0027] In one embodiment, obtaining chat text data according to the chat interaction information and generating recommended speech content using a pre-trained large language model includes:

[0028] Convert the host voice signal on the host side of the live broadcast room into voice text information, and generate public screen text information from the recommended speech content sent on the public screen on the viewer side of the live broadcast room;

[0029] Align the voice text information and the public screen text information according to the time stamp to obtain chat text data;

[0030] Input the chat text data into a prompt template trained based on the text structure of the push information of the live broadcast room to obtain large language model input data;

[0031] Input the large language model input data into a large language model trained based on the text content of the push information of the live broadcast room to obtain recommended speech content.

[0032] In one embodiment, the method for an AI assistant control to play the highlight content video includes:

[0033] Place the highlight content video on the live broadcast room cover for playback;

[0034] Place the highlight content video on a floating window for playback;

[0035] Or

[0036] Place the highlight content video in a half-window pulled up by the AI assistant control for playback;

[0037] In one embodiment, placing the highlight content video in a half-window pulled up by the AI assistant control for playback includes:

[0038] Pull up a half-window through the AI assistant control;

[0039] Display a sphere in the half-window, load multiple highlight content videos, and place each highlight content video in a nine-square grid of the sphere for display.

[0040] In one embodiment, the method for playing the live broadcast room video further includes:

[0041] When playing a video of high - light content, the AI assistant obtains the live - related information of the current live - broadcast room and plays the live - related information on top of the video of the high - light content;

[0042] Among them, the live - related information includes at least one of the following: the public - screen information of the live - broadcast room, the host information, the gift - giving information, and the playing content of large gifts.

[0043] In one embodiment, the method for playing a live - broadcast room video further includes:

[0044] When the user swipes up and down on the home page of the live - broadcast room to browse the live - broadcast room covers, the AI assistant obtains the swiping direction and the target live - broadcast room covers within the visible range, and extracts the video links of the high - light content videos generated by the live - broadcast rooms corresponding to each target live - broadcast room cover and puts them into the cache queue;

[0045] The AI assistant adjusts the sorting of the video links in the cache queue according to the up - and - down swiping direction, and selects the high - light content video generated by the live - broadcast room corresponding to the target live - broadcast room cover according to the sorting.

[0046] A device for playing a live - broadcast room video includes:

[0047] A display module, configured to display the live - broadcast room covers of each live - broadcast room and the AI assistant control after the user enters the home page of the live - broadcast room;

[0048] A browsing module, configured to display the play button of the high - light content video when there is a high - light content video on the live - broadcast room cover browsed on the home page of the live - broadcast room;

[0049] A dragging module, configured to obtain the target live - broadcast room cover where the user releases the play button in response to the operation of the user dragging the play button to the live - broadcast room cover;

[0050] A play module, configured to, after the user releases the play button, the AI assistant control loads the high - light content video generated by the live - broadcast room of the target live - broadcast room cover and plays it.

[0051] A live - broadcast system includes: a host end, an audience end, and a live - broadcast server; among them, the host end and the audience end are respectively connected to the live - broadcast server through a communication network;

[0052] The host end is used to access the host of the live - broadcast room and collect the live - broadcast video stream of the host and upload it to the live - broadcast server;

[0053] The live - broadcast server is used to forward the live - broadcast video stream of the host to the audience end and generate the high - light content video of the live - broadcast room;

[0054] The viewer terminal is used for viewer users to access the live broadcast room, receive and play the live video stream of the host, and play the highlight content video by using the live broadcast room video playing method described above.

[0055] An electronic device, which includes:

[0056] One or more processors;

[0057] A memory;

[0058] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the live broadcast room video playing method.

[0059] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to execute the steps of the live broadcast room video playing method.

[0060] The technical solution of this application, after the user enters the home page of the live broadcast room, displays the live broadcast room covers of each live broadcast room and the AI assistant control; when there is a highlight content video on the live broadcast room cover browsed on the home page of the live broadcast room, displays the play button of the highlight content video; the user drags the play button to the target live broadcast room cover and releases it, and after the user releases the play button, the AI assistant control loads and plays the highlight content video generated by the live broadcast room of the target live broadcast room cover; this technical solution automatically obtains and plays the highlight content video based on the user dragging the play button of the AI assistant control to the target live broadcast room cover, thereby reducing the interaction cost for the user to view the highlight content video, facilitating the user to quickly understand the live broadcast room information, increasing the probability of the user entering the live broadcast room to watch the live broadcast, and increasing the daily active users of the live broadcast platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic diagram of an example live broadcast service application scenario;

[0062] Figure 2 It is a flowchart of the live broadcast room video playing method of an embodiment;

[0063] Figure 3 It is a schematic diagram of the page of an example live broadcast room home page;

[0064] Figure 4 It is a schematic diagram of an example play button;

[0065] Figure 5 It is a flowchart of an example of the AI assistant generating the highlight content video;

[0066] Figure 6 It is a schematic diagram of long - pressing and dragging a sample play button;

[0067] Figure 7 It is a schematic diagram of playing a high - light content video in a floating window;

[0068] Figure 8 It is a schematic diagram of displaying a high - light content video in a half - window;

[0069] Figure 9 It is a schematic diagram of displaying a high - light content video on a spherical body;

[0070] Figure 10 It is a schematic diagram of playing a high - light content video by arranging sub - windows;

[0071] Figure 11 It is a schematic diagram of the UI for displaying a high - light content video;

[0072] Figure 12 It is a schematic diagram of playing a high - light content video in full - screen;

[0073] Figure 13 It is a schematic diagram of the structure of a live - broadcast room video playback device in an embodiment;

[0074] Figure 14 It is a schematic diagram of the structure of a sample live - broadcast system;

[0075] Figure 15 It is a block diagram of a sample electronic device. Detailed implementation manners

[0076] In order to make the purpose, technical solutions and advantages of this application clearer, the following further elaborates on this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0077] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and roles. "At least one" means one or more, and the meaning of "multiple" is two or more. For example, multiple objects refer to two or more objects. "Including" or "comprising" and other similar words mean that the information appearing before "including" or "comprising" covers the information listed after "including" or "comprising" and its equivalents, and does not exclude other information. In the embodiments of this application, the mention of "and / or" indicates that three relationships can exist. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0078] The technical solution provided by the embodiments of this application can be applied to, for example, Figure 1 the application scenarios of the related methods of this application as shown in Figure 1 which is a schematic diagram of an exemplary live broadcast service application scenario. The live broadcast system may include a live broadcast server, a host terminal, and an audience terminal. The host terminal and the audience terminal communicate with the live broadcast server through a communication network, so that the host of the host terminal and the audience users of the audience terminal can conduct real-time network live broadcasts. Among them, for the host terminal and the audience terminal, the terminal devices can be, but are not limited to, various personal computers, laptop computers, smart phones, and tablet computers. The live broadcast server can be implemented by an independent server or a server cluster composed of multiple servers.

[0079] In order to reduce the interaction operation cost between the user and the live broadcast room and facilitate the user to quickly understand the relevant information of the live broadcast room, the technical solution of the embodiments of this application uses an AI assistant to assist the live broadcast room in live broadcasting. When the user browses the live broadcast room cover on the live broadcast room home page, by dragging the play button of the AI assistant control to assist in playing the highlight content video of the live broadcast room, and provides recommended speech content assistance for the user, and provides a more interactive play interface, so as to achieve convenient interaction between the user and the live broadcast room. The user can quickly understand the relevant information of the live broadcast room, enhancing the interactivity of the live broadcast room.

[0080] Refer to Figure 2 as shown in Figure 2 which is a flowchart of a method for playing a live broadcast room video in an embodiment, including the following steps:

[0081] Step S10, after the user enters the live broadcast room home page, display the live broadcast room covers of each live broadcast room and the AI assistant control.

[0082] In this step, when the user first enters the live broadcast room home page, the live broadcast room covers of each live broadcast room are displayed on the terminal interface, and the AI assistant is displayed at a set position to guide the user to use it. Among them, the AI assistant is a live broadcast assistance tool configured based on AI artificial intelligence technology, and its configuration has processing capabilities for voice, text, pictures, etc., and can connect search, large models, intelligent agents, etc., and can realize diversified search and data processing capacity requirements.

[0083] Exemplarily, when the client detects that the user enters the live broadcast room home page, the AI assistant in the default state is displayed on the display interface. As Figure 3 shown in Figure 3It is a schematic diagram of the homepage of an exemplary live broadcast room, which includes the live broadcast covers of six live broadcast rooms, and pictures or the current live broadcast images can be displayed on each live broadcast cover; an AI assistant control is displayed at the lower right corner of the homepage. By default, the image of the AI assistant can be a character with a second-generation avatar; the AI assistant can be set with multiple functions according to requirements. For example, functions such as analyzing the live broadcast room and its live video.

[0084] Step S20: When there is a highlight content video on the live broadcast cover browsed on the homepage of the live broadcast room, display the play button for the highlight content video.

[0085] In this step, when the user browses the live broadcast covers on the homepage of the live broadcast room, if there is a highlight content video on the live broadcast covers within the visible range on the homepage of the live broadcast room, at this time, a play button for the highlight content video is displayed on the AI assistant control. Exemplarily, as Figure 4 shown, Figure 4 It is a schematic diagram of an exemplary play button. The play button can be displayed in the form of a play icon. It can be a floating button generated by the AI assistant control or a floating button transformed from the AI assistant control; preferably, a prompt message for dragging the button to the live broadcast cover can also be generated beside the play button to guide the user to drag the play button to the target live broadcast cover to watch the highlight content video.

[0086] In one embodiment, when the AI assistant control determines whether to display the play button, it mainly includes the following steps:

[0087] Respond to the user's operation of swiping the live broadcast covers on the homepage of the live broadcast room to browse, and slide to display the list of live broadcast covers; detect whether there is a highlight content video on the live broadcast covers within the visible range on the current homepage of the live broadcast room; if there is a highlight content video on any live broadcast cover, display the play button for the highlight content video, otherwise do not display the play button.

[0088] Specifically, when the user swipes the live broadcast covers on the homepage of the live broadcast room to browse, the homepage of the live broadcast room will slide to display the list of live broadcast covers. At this time, the AI assistant control checks whether there is a pre-stored highlight content video on the live broadcast covers within the visible range on the current homepage of the live broadcast room. If there is, display the play button for the highlight content video to prompt the user to watch. If not, do not display the play button.

[0089] Exemplarily, for the highlight content video, the highlight content video generally refers to the highlight moments of the host's live broadcast, such as the host's talent moments, honor moments and other related video content; the video content generally includes the live video stream, public screen messages and host information, etc. It can be generated according to a pre-set strategy or in real time by the AI assistant.

[0090] For example, a solution for an AI assistant to generate a highlight content video in real time; Figure 5 As shown, Figure 5 This is an example of an AI assistant generating a highlight content video flow chart. For example, for a talent moment video, a 10-second highlight video stream can be captured from the host's highlight performance of more than 30 seconds, and the 10-second talent moment video can be generated by combining the live video stream, the user's public screen speech flow and the host's information area; for another example, for an honor moment video, through the user's video of obtaining the top 1 of the hourly list, the corresponding video stream can be captured when the top 1 is triggered, and a 5-second honor moment video can be generated by combining the live video stream, the user's public screen speech flow, the host's information area and the hourly list banner; for another example, for a gift-giving honor moment video, it can be executed when the host receives a large gift, the gift MP4 animation with the highest price is selected, and the video starting 3 seconds after the user gives the gift is captured, and the live video stream, the gift MP4 animation and the host's information area are combined to generate a gift-giving honor moment video with the same length as the gift animation.

[0091] Step S30, in response to the user dragging the play button to the live broadcast room cover, obtain the target live broadcast room cover where the user is when the play button is released.

[0092] In this step, when the user wants to play the highlight content video of a live broadcast room on the homepage of the live broadcast room, he can drag the play button to the live broadcast room cover. When the user releases the play button, the AI assistant control will determine the target live broadcast room cover when the user releases the play button.

[0093] In one embodiment, Figure 6 As shown, Figure 6 It is an example diagram of long pressing and dragging the play button; when the user clicks and drags the play button, the AI assistant control will outline and illuminate the borders of the live broadcast room covers that have highlight content videos within the visible range on the current live broadcast room homepage; this can indicate to the user which live broadcast room covers have pre-stored highlight content videos.

[0094] In one embodiment, when the user drags the play button to the target live broadcast room cover area, the three stages of dragging the play button, namely, start, move, and end, can be established through the HTML5 drag API (Application Programming Interface), and an event listener can be added; then the event of the user long pressing and dragging the play button to the live broadcast room cover is monitored; if the live broadcast room cover of the current live broadcast room cover area is pre-configured to support the playback of highlight content videos, the live broadcast room cover is set to a responsive state, and when the user releases the long press of the play button, the target live broadcast room cover where the long press operation is released and its corresponding target live broadcast room are determined.

[0095] According to the technical solution of the above embodiment, the user can long-press and drag the play button to the live room cover. The AI assistant control responds to the user's drag operation in the background and synchronously displays the visual effect of the play button moving to the live room cover in real time, so as to visually present the target live room where the highlight content video can be watched to the user.

[0096] Step S40, after the user releases the play button, the AI assistant control loads the highlight content video generated by the live room of the target live room cover and plays it.

[0097] In this step, after the user releases the play button, the AI assistant control loads the highlight content video generated by the live room of the target live room cover, and uses the play window to play the corresponding highlight content video. The user can quickly complete the play interaction process of the highlight content video of the target live room by dragging, shortening the UI interaction path between the user and the live room, reducing the user interaction cost, and facilitating the user to quickly understand the live information of the live room.

[0098] In one embodiment, when there is no highlight content video on the target live room cover where the user releases the play button, the AI assistant control accesses the live room corresponding to the target live room cover, and calls the preset highlight content video production method to intercept the highlight content video from the live video screen of the live room for caching and playing.

[0099] Specifically, if the user drags the play button to the live room cover without highlight content video and releases the play button, since there is no highlight content video to play, at this time, the AI assistant control accesses the corresponding live room, calls the preset highlight content video production method, and intercepts a segment of highlight content video from the current live video screen of the live room for caching and playing, so that the user can timely understand the live information of the live room.

[0100] To make the technical solution of the present application clearer, more embodiments of playing the highlight content video are described below.

[0101] In one embodiment, when playing the highlight content video, the AI assistant control can play the highlight content video on a popped-up floating window; as Figure 7 shown, Figure 7 is a schematic diagram of playing the highlight content video on an example floating window. The floating window is displayed on the home page of the live room, and then the highlight content video is played on the popped-up floating window.

[0102] In one embodiment, the AI assistant control can also pull up a half-window on the live room page, and then place the loaded highlight content video in the half-window for playing; as Figure 8 shown, Figure 8This is a schematic diagram of a semi-window displaying high-light content videos. At this time, the user can also click on the live room cover to enter the current live room. Exemplarily, the AI assistant control can play multiple high-light content videos such as talent moments and honor moments in the form of sub-windows on the semi-window.

[0103] Furthermore, during the process of playing high-light content videos in the semi-window, when the user clicks on the home page of the live room to view the covers of other live rooms, the semi-window of the AI assistant control is collapsed. At this time, the AI assistant control can be displayed in the form of a play button in its initial state.

[0104] As in the solution of the above embodiment, based on the AI assistant automatically assisting the user to obtain and play high-light content videos, the user can automatically obtain high-light content videos, thereby reducing the operation interaction cost, facilitating the user to quickly understand the live information of the live room, increasing the probability of the user watching the live room, and increasing the daily activity of the live platform.

[0105] In one embodiment, when the AI assistant control plays high-light content videos in the semi-window, it can display a sphere in the semi-window, load multiple high-light content videos, and place each high-light content video in the nine-square grid of the sphere for display. As Figure 9 shown, Figure 9 This is a schematic diagram of high-light content videos displayed on a sphere. As shown in the figure, when playing multiple high-light content videos, the high-light content videos can be placed on the nine-square grid of the sphere for display, and the user can slide the sphere left or right or up and down to select the high-light content video to watch.

[0106] In one embodiment, when the AI assistant control plays high-light content videos, it can be placed on the live room cover for playing; specifically, when loading high-light content videos and placing them in the semi-window for playing, multiple high-light content videos can be loaded, and the high-light content videos can be placed in the sub-windows arranged on the semi-window for playing; that is, multiple arranged sub-windows are placed in the semi-window, and then multiple high-light content videos are placed in each sub-window for playing, and the user can slide left and right to select the high-light content video to watch.

[0107] As Figure 10 shown, Figure 10 This is a schematic diagram of playing high-light content videos through arranged sub-windows. As shown in the figure, three sub-windows are used to play multiple honor moment videos. Considering the size setting of the sub-windows, two sub-windows can be displayed within the visible range. When other sub-windows need to be selected, they can be selected by sliding left and right.

[0108] In one embodiment, when playing a high-light content video, in order to facilitate the user's better participation in the interaction of the current live broadcast room, when playing the high-light content video, the AI assistant control can also generate speech content and display it. When the user clicks on the speech content, the speech content is sent to the public screen of the live broadcast room for display.

[0109] Accordingly, the method for playing a live broadcast room video in this embodiment may further include:

[0110] The AI assistant reads the chat interaction information between the host and the audience in the live broadcast room; obtains chat text data according to the chat interaction information and uses a pre-trained large language model to generate recommended speech content; when the user clicks on the recommended speech content, the recommended speech content is sent to the public screen of the live broadcast room for display.

[0111] The solution of the above embodiment can enable the user to quickly enter the communication and interaction with the live broadcast room, enhancing the interactivity between the user and the live broadcast room.

[0112] In one embodiment, when obtaining chat text data according to the chat interaction information and using a pre-trained large language model to generate recommended speech content as described above, it may include the following steps:

[0113] (1) Convert the host voice signal on the host side of the live broadcast room into voice text information, and generate public screen text information from the recommended speech content sent on the public screen on the audience side of the live broadcast room; for example, collect the voice signal on the host side of the live broadcast room, and perform voice separation on the voice signal to remove background music to obtain the host voice signal; extract the speech content input on the public screen on the audience side of the live broadcast room, and filter out non-text symbols in the speech content to obtain speech text; obtain the chat theme content of the current live broadcast room by collecting the chat interaction information on the host side and the audience side. Use speech recognition technology to convert the voice signal into voice text information, convert the speech text input by the audience on the public screen into text information, and perform operations such as downsampling on the speech to obtain high-quality public screen text information.

[0114] (2) Align the voice text information and the public screen text information according to the time stamp to obtain chat text data; for example, align the voice text information and the public screen text information according to the time stamp to obtain chat text data, so as to obtain high-quality chat text data and ensure the correct order of the relevant data input into the large language model.

[0115] (3) Input the chat text data into a prompt template trained based on the text structure of the push information in the live stream room to obtain large language model input data; Exemplarily, design an initial template for processing the prompt content format according to the information to be pushed, perform format optimization training on the initial template based on the pre-annotated text structure of the push information in the live stream room, perform format tuning on the initial template during the testing process to obtain the prompt template, input the chat text data into the prompt template to obtain large language model input data, and use the optimized prompt template to process the data input into the large language model first, so as to guide the large language model to generate high-quality output results.

[0116] (4) Input the large language model input data into a large language model trained based on the text content of the push information in the live stream room to obtain recommended speech content; Exemplarily, perform parameter adjustment training on a general language model using the pre-annotated text content of the push information in the live stream room to obtain a large language model, and input the large language model input data into the large language model to obtain recommended speech content.

[0117] Exemplarily, a general language model can be used for parameter fine-tuning training. Fine-tuning means making appropriate adjustments to some parameters of the general language model, so that the large language model not only utilizes the powerful basic performance of the general language model and has powerful information processing capabilities, but can also combine the personalized needs in the live stream scenario of the live stream room.

[0118] For example, first add a bypass composed of a dimensionality reduction matrix A and a dimensionality increase matrix B to the pre-training weight matrix W of the pre-trained model. The matrix product BA of the two matrices in the bypass is consistent with the dimension of the pre-training weight matrix W; then in the training stage, fix the pre-training weight matrix W of the pre-trained model, and only train the dimensionality reduction matrix A and the dimensionality increase matrix B. The dimensionality reduction matrix A is randomly initialized to a Gaussian distribution, and the dimensionality increase matrix B is initialized to 0. When the model outputs, the matrix product BA of the dimensionality reduction matrix and the dimensionality increase matrix is superimposed with the pre-training weight matrix W. Assume the pre-training weight matrix of the large language model pre-training Then the update can be expressed as:

[0119] W0 + ΔW = W0 + BA

[0120] Among them, ΔW represents the update adjustment amount of the pre-training weight matrix, The rank r << min(d, k); in the training stage, keep the pre-training weight matrix W0 fixed and adjust the dimensionality reduction matrix A and the dimensionality increase matrix B.

[0121] Finally, in the inference stage, multiply the model input by the superimposed parameters as the model output; among them, during forward inference, the pre-training weight matrix W0 and the update adjustment amount ΔW are multiplied by the same model input x and finally added together to obtain, and the expression is as follows:

[0122] h = W0x + ΔWx = W0x + BAx

[0123] Among them, h is the output of the large language model after fine-tuning. Since the number of parameters (d×r + r×k) of the matrix product BA that needs to be fine-tuned is much smaller than the number of parameters (d×k) of the pre-trained weight matrix W0, it is thus possible to complete the fine-tuning of the large language model by training only a very small number of parameters.

[0124] According to the technical solution of the above embodiment, a large language model is trained by parameter fine-tuning. Since the parameter fine-tuning training of the large language model is based on the real chat interaction data in the live broadcast room, it has the advantages of few parameters, saving computing resources, strong generalization ability, and good inheritance ability of the original model knowledge. Therefore, the recognition accuracy of the large language model can be improved, and the quality of the push information generated by the large language model can be enhanced.

[0125] As Figure 11 shown, Figure 11 is a UI schematic diagram showing a sample highlight content video. As shown in the figure, the AI assistant can generate speech contents such as "Is this prop amazing?", "The host is so smart", "I also want to know", "Hurry up and fast forward"... etc. The user can click on these speech contents to make a speech; for the generated speech contents, the AI assistant can identify the application scenario of the current live broadcast room, so as to obtain speech contents adapted to the application scenario. In addition, some summary information about the highlight content video and the like can also be displayed using the half-window of the AI assistant.

[0126] When the user needs to exit the half-window display of the highlight content video, the client can respond to the user's operation of swiping down to close the half-window, and close the half-window of the AI assistant control to watch the live broadcast. In addition, it can also be closed by swiping down. At this time, the client responds to the user's operation of swiping down to close the half-window, and closes the half-window of the AI assistant to watch the live broadcast.

[0127] In one embodiment, in order to facilitate the user to more deeply understand the real-time dynamics of the live broadcast room when watching the highlight content video, accordingly, in the solution of this embodiment, when playing the highlight content video, the AI assistant obtains the public screen information, host information, gift-giving information, large gift playback content and other live broadcast-related information of the current live broadcast room, and plays these live broadcast-related information on top of the highlight content video; for example, when the user clicks the zoom button to switch to full-screen playback, the AI assistant obtains the public screen information, host information, gift-giving information, large gift playback content and the like of the current live broadcast room and plays them on top of the highlight content video.

[0128] As Figure 12 shown, Figure 12It is a schematic diagram of an example full-screen playing highlight content video. There are three different types of highlight content videos in the figure. For example, the left figure is a schematic diagram of a highlight moment video in a game, and the middle and right figures are schematic diagrams of honor moment videos; for live broadcast associated information, it can reproduce the public screen information of the current live broadcast room through an AI assistant, pull the host information and the gift-giving information in the live broadcast room through the AI assistant, obtain the big gift information, and play the big gift animation on the upper layer of the highlight content video. Thus, when the user watches the highlight content video, they can obtain the dynamic information of the current live broadcast room, which can attract users to enter the live broadcast room to watch the live broadcast and interact in the live broadcast, thereby enhancing the interactivity between the user and the live broadcast room.

[0129] In one embodiment, in order to adapt to the user's operation of browsing the live broadcast room cover and play the highlight content video, the AI assistant control can include the following when generating the highlight content video:

[0130] When the user swipes up and down on the live broadcast room home page to browse the live broadcast room cover, the AI assistant obtains the swipe direction and the target live broadcast room covers within the visible range, and extracts the video links of the highlight content videos generated by the live broadcast rooms corresponding to each target live broadcast room cover and puts them into the cache queue; the AI assistant adjusts the sorting of the video links in the cache queue according to the up and down swipe direction, and selects the highlight content video generated by the live broadcast room corresponding to the target live broadcast room according to the sorting.

[0131] Specifically, by using the cache queue to cache the video links, the AI assistant can identify the target live broadcast room covers within the visible range according to the user's swipe direction, sort the video links of their highlight content videos, predict the user's browsing intention during the user's continuous swiping and browsing process, and adjust the sorting of the video links in the cache queue in real time. Thus, it can quickly load and play the appropriate highlight content video, ensure the playing speed, and obtain a better interaction experience, thereby enhancing the interactivity between the user and the live broadcast room.

[0132] The embodiments of the live broadcast room video playing device are described below.

[0133] As Figure 13 shown, Figure 13 It is a schematic diagram of the structure of a live broadcast room video playing device in an embodiment, including:

[0134] A display module 10, configured to display the live broadcast room covers of each live broadcast room and the AI assistant control after the user enters the live broadcast room home page;

[0135] A browsing module 20, configured to display the play button of the highlight content video when there is a highlight content video on the live broadcast room cover browsed on the live broadcast room home page;

[0136] A drag module 30, configured to obtain a target live room cover where the play button is released by the user in response to an operation of the user dragging the play button to the live room cover.

[0137] A play module 40, configured to play a highlight content video generated by the live room where the target live room cover is loaded by the AI assistant control after the user releases the play button.

[0138] The live room video playing device in this embodiment can execute a live room video playing method provided in an embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the live room video playing device in each embodiment of the present application correspond to the steps in the live room video playing method in each embodiment of the present application. For a detailed functional description of each module of the live room video playing device, reference can be specifically made to the description in the corresponding live room video playing method shown above, and details are not described herein again.

[0139] Embodiments of the live broadcast system are described below.

[0140] The live broadcast system in this embodiment refers to Figure 14 as shown Figure 14 is a schematic structural diagram of an exemplary live broadcast system. The system framework may include a host end, a live broadcast server, and an audience end. The audience end and the host end can enter the live broadcast platform through a client installed on an electronic device. Exemplarily, the host end and the audience end can be computer devices such as PDAs, smartphones, tablet computers, desktop computers, or laptop computers, etc., which are not limited thereto, or can also be software modules of application programs. The live broadcast server can be implemented by an independent server or a server cluster composed of multiple servers. The host end is used to access the host of the live room and collect the live video stream of the host and upload it to the live broadcast server; the live broadcast server is used to forward the live video stream of the host to the audience end, and play the highlight content video by using the live room video playing method of any of the foregoing embodiments; specifically, the live broadcast server uses the AI assistant on the client to provide front-end services and the AI server to provide back-end analysis and processing capabilities, so as to provide an active assistance function for users to watch the live content of the live room, so that users can quickly understand the relevant information of the live room and guide users to enter the live room to watch the live broadcast; the audience end is used to access the audience users of the live room, receive and play the live video stream of the host, and play the highlight content video through the AI assistant.

[0141] The live broadcast system in this embodiment automatically assists users to obtain and play the highlight content video based on the AI assistant, thereby reducing the interaction cost for users to play the highlight content video, facilitating users to quickly understand the live room, enhancing the willingness of users to watch the live room, and increasing the daily active users of the live broadcast platform.

[0142] Embodiments of an electronic device and a computer-readable storage medium are described below.

[0143] This application provides a technical solution for an electronic device to implement functions related to the live broadcast room video playback method. The electronic device in this embodiment includes one or more processors and a memory; one or more applications, where one or more applications are stored in the memory and configured to be executed by one or more processors, and one or more programs are configured to perform the steps of the live broadcast room video playback method in any embodiment.

[0144] As Figure 15 shown, Figure 15 is a block diagram of an exemplary electronic device; the electronic device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. The electronic device 100 may include one or more of the following components: a processing component 102, a memory 104, a power component 106, a multimedia component 108, an audio component 109, an input / output (I / O) interface 112, a sensor component 114, and a communication component 116.

[0145] The processing component 102 generally controls the overall operation of the electronic device 100, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations.

[0146] The memory 104 is configured to store various types of data to support the operation of the electronic device 100. Such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0147] The power component 106 provides power to various components of the electronic device 100.

[0148] The multimedia component 109 includes a screen that provides an output interface between the electronic device 100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). In some embodiments, the multimedia component 108 includes a front camera and / or a rear camera.

[0149] The audio component 109 is configured to output and / or input audio signals.

[0150] The I / O interface 112 provides an interface between the processing component 102 and a peripheral interface module, and the peripheral interface module may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

[0151] The sensor component 114 includes one or more sensors for providing a status assessment of various aspects for the electronic device 100. The sensor component 114 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact.

[0152] The communication component 116 is configured to facilitate communication between the electronic device 100 and other devices in a wired or wireless manner. The electronic device 100 may access a wireless network based on communication standards, such as WiFi, a carrier network (such as 2G, 3G, 4G, or 5G), or a combination thereof.

[0153] This application provides a technical solution for a computer-readable storage medium to implement functions related to the live broadcast room video playback method. The computer-readable storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to perform the live broadcast room video playback method of any embodiment.

[0154] In an exemplary embodiment, the computer-readable storage medium may be a non-temporary computer-readable storage medium including instructions, such as a memory including instructions. For example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0155] The above embodiments only represent several implementation manners of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limitations on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application shall be subject to the appended claims.

Claims

1. A method for playing live video, characterized in that: include: After the user enters the live broadcast room homepage, the live broadcast room cover and AI assistant controls of each live broadcast room are displayed; When a highlight video exists on the cover of the live broadcast room viewed on the homepage of the live broadcast room, the play button of the highlight video will be displayed; In response to the user dragging the play button to the live broadcast room cover, obtaining the target live broadcast room cover when the user releases the play button; After the user releases the play button, the AI assistant control loads the highlight content video generated by the live broadcast room of the target live broadcast room cover and plays it.

2. The live video playback method according to claim 1, characterized in that: Also includes: In response to the user sliding the live broadcast room cover on the live broadcast room homepage to browse, sliding to display the live broadcast room cover list; Check whether there is a highlight content video within the visible range of the live broadcast room cover on the current live broadcast room homepage; If there is a highlight content video on the cover of any live broadcast room, the play button of the highlight content video will be displayed, otherwise the play button will not be displayed.

3. The live video playback method according to claim 2, characterized in that: Also includes: When the user clicks and drags the play button, the AI assistant control will outline and illuminate the border of the live broadcast room cover with highlight content videos within the visible range on the current live broadcast room homepage.

4. The live video playback method according to claim 1, characterized in that: Also includes: When the user releases the play button and there is no highlight content video on the cover of the target live broadcast room, the AI assistant control connects to the live broadcast room corresponding to the cover of the target live broadcast room, and calls the preset highlight content video production method to capture the highlight content video from the live video screen of the live broadcast room for caching and playback.

5. The live video playback method according to claim 4, characterized in that: The highlight content video production method includes: Intercepting a preset-length highlight video stream from the live stream host's talent moment video, and generating a preset-length highlight content video based on the highlight video stream, public screen speech, and host information; Get the live broadcast host's video on the hourly ranking list. When the host triggers the top 1 ranking, start capturing the top 1 ranking video of a preset duration. Generate a highlight content video of a preset duration based on the public screen speech of the top 1 ranking video, the host information, and the hourly ranking banner. or When the host in the live broadcast room receives a big gift, the highest-priced gift MP4 animation is selected, and the big gift video after the audience user gives the gift is captured. Based on the big gift video, gift MP4 animation and host information area, a highlight content video with the same length as the gift animation is generated.

6. The live video playback method according to claim 1, characterized in that: Also includes: When playing highlight content videos, the AI assistant reads the chat interaction information between the live broadcast host and the audience; Acquire chat text data based on the chat interaction information and generate recommended speech content using a pre-trained large language model; When the user clicks on the recommended speech content, the recommended speech content is sent to the public screen of the live broadcast room for display.

7. The live video playback method according to claim 6, characterized in that: Acquiring chat text data based on the chat interaction information and generating recommended speech content using a pre-trained large language model includes: Convert the live broadcast host's voice signal into voice text information, and generate public screen text information from the recommended speech content sent by the live broadcast audience on the public screen; Aligning the voice text information and the public screen text information according to the timestamps to obtain chat text data; Inputting the chat text data into a prompt template trained based on the text structure of the push information in the live broadcast room to obtain large language model input data; The large language model input data is input into the large language model trained based on the text content of the push information in the live broadcast room to obtain the recommended speech content.

8. The live video playback method according to claim 1, characterized in that: The method for the AI assistant control to play the highlight content video includes: Place the highlight content video on the cover of the live broadcast room for playback; Placing the highlight content video on a floating window for playback; or Place the highlight content video in the half-window opened by the AI assistant control for playback.

9. The live video playback method according to claim 6, characterized in that: Placing the highlight content video in the half-window opened by the AI assistant control for playback includes: Pull up the half-window through the AI assistant control; A spherical body is displayed in the half window, a plurality of highlight content videos are loaded, and each highlight content video is placed in a nine-square grid of the spherical body for display.

10. The live video playback method according to claim 1, characterized in that: Also includes: When playing a highlight content video, the AI assistant obtains the live broadcast related information of the current live broadcast room and places the live broadcast related information on top of the highlight content video for playback; Among them, the live broadcast related information includes: at least one of the public screen information of the live broadcast room, anchor information, gift information, and big gift playback content.

11. The live video playback method according to claim 1, characterized in that: Also includes: When a user swipes up and down on the homepage of a live broadcast room to browse the cover of the live broadcast room, the AI assistant obtains the swiping direction and the target live broadcast room cover within the visual range, and extracts the video link of the highlight content video generated by the live broadcast room corresponding to each target live broadcast room cover and puts it into the cache queue; Use the AI assistant to adjust the order of video links in the cache queue according to the up and down sliding direction, and select the highlight content video generated by the live broadcast room corresponding to the target live broadcast room cover based on the order.

12. A live video playback device, characterized in that: include: The display module is used to display the live broadcast room cover and AI assistant controls of each live broadcast room after the user enters the live broadcast room homepage; The browsing module is used to display the play button of the highlight content video when there is a highlight content video on the live broadcast room cover browsed on the live broadcast room homepage; The drag module is used to respond to the user's operation of dragging the play button to the live broadcast room cover and obtain the target live broadcast room cover when the user releases the play button; The playback module is used to load the highlight content video generated by the live broadcast room of the target live broadcast room cover and play it after the user releases the play button.

13. A live broadcast system, characterized in that: include: An anchor terminal, an audience terminal, and a live broadcast server; wherein the anchor terminal and the audience terminal are respectively connected to the live broadcast server via a communication network; The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server; The live broadcast server is used to forward the live broadcast video stream of the anchor to the audience end and generate the highlight content video of the live broadcast room; The audience end is used for audience users accessing the live broadcast room, receiving the host's live video stream for playback, and using the live broadcast room video playback method described in any one of claims 1-11 to play highlight content videos.

14. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the live broadcast room video playback method described in any one of claims 1-11.

15. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded by the processor and executes the steps of the live room video playback method described in any one of claims 1-11.

Images