Live video processing method, device, computer readable medium and electronic device

Abstract

The embodiments of the present application provide a method, device, computer-readable medium and electronic device for processing live video. The method for processing live video includes: receiving a joining request for a target live room, the joining request including identification information of a user requesting to join the target live room; determining the live content missed by the user based on the broadcast duration of the target live room and the identification information of the user; selecting at least one video clip from the live content missed by the user; generating a video highlight clip based on the selected video clip, and presenting the video highlight clip to the user. The technical solution of the embodiments of the present application enables the user to efficiently understand the broadcast content of the live room based on the video highlight clip when entering the live room, thereby improving the acceptability of the live content.

Description

Technical Field

[0001] The present application relates to the field of computer and communication technology, and in particular to a method, device, computer-readable medium and electronic device for processing live video. Background Art

[0002] Live video streaming is a way of live streaming using the Internet and streaming media technology. In related technologies, when users enter a virtual live broadcast room for live video streaming, the real-time live content will be played directly. Since users have missed the previous live content, it is difficult for them to keep up with the current live progress, which not only affects the acceptability of the live content, but also reduces the user stickiness of the live broadcast platform. Summary of the invention

[0003] The embodiments of the present application provide a method, device, computer-readable medium and electronic device for processing live video, thereby enabling users to efficiently understand the broadcast content of the live room based on video highlights when entering the live room, at least to a certain extent, thereby improving the acceptability of the live content.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a method for processing live video is provided, including: receiving a joining request for a target live room, the joining request including identification information of a user requesting to join the target live room; determining live content missed by the user based on the broadcast duration of the target live room and the identification information of the user; selecting at least one video clip from the live content missed by the user; generating a video highlight clip based on the selected video clip, and presenting the video highlight clip to the user.

[0006] According to one aspect of an embodiment of the present application, a method for processing live video is provided, including: displaying information of a live broadcast room that is being broadcast; if it is detected that a user has entered a target live broadcast room, sending a join request for the target live broadcast room to a server, the join request including identification information of the user; receiving a video highlight segment transmitted by the server, the video highlight segment being generated based on a video segment selected from live content missed by the user; and playing the video highlight segment.

[0007] According to one aspect of an embodiment of the present application, a live video processing device is provided, including: a first receiving unit, configured to receive a joining request for a target live broadcast room, the joining request including identification information of a user requesting to join the target live broadcast room; a determining unit, configured to determine the live content missed by the user based on the broadcast duration of the target live broadcast room and the identification information of the user; a selecting unit, configured to select at least one video clip from the live content missed by the user; a first processing unit, configured to generate a video highlight clip based on the selected video clip, and present the video highlight clip to the user.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the determination unit is configured to: obtain the time point when the user joins the target live broadcast room according to the identification information of the user; determine the live broadcast time period missed by the user according to the broadcast time of the target live broadcast room and the time point when the user joins the target live broadcast room, so as to determine the live broadcast content missed by the user based on the live broadcast time period missed by the user.

[0009] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit is further configured to: before selecting at least one video clip from the live broadcast content missed by the user, determine the live broadcast duration missed by the user according to the live broadcast period missed by the user; if the live broadcast duration missed by the user is less than or equal to the first duration, present the real-time live broadcast content of the target live broadcast room to the user; the selection unit is configured to: if the live broadcast duration missed by the user is greater than the first duration, execute the process of selecting at least one video clip from the live broadcast content missed by the user.

[0010] In some embodiments of the present application, based on the aforementioned scheme, the live video processing device also includes: a second processing unit, configured to segment the broadcast content of the target live room based on the second duration as a segmentation basis to obtain multiple video time periods; and split the video content in the video time period to obtain multiple video clips corresponding to the broadcast content.

[0011] In some embodiments of the present application, based on the above solution, the second processing unit is configured to split the video content in the video period in at least one of the following ways:

[0012] Performing semantic analysis on the video content in the video period to split the video content in the video period based on the semantic analysis result;

[0013] Performing content recognition processing on the video content in the video period to split the video content in the video period based on the content recognition result;

[0014] Performing heat analysis on the video content in the video period to split the video content in the video period based on the heat analysis result.

[0015] In some embodiments of the present application, based on the aforementioned scheme, the selection unit is configured to: determine at least one video time period to which the live content missed by the user belongs based on multiple video clips corresponding to the played content; and select at least one video clip from the at least one video time period.

[0016] In some embodiments of the present application, based on the aforementioned scheme, the selection unit is configured to: select video clips matching the user characteristics from the at least one video period according to the user characteristics of the user; if no video clip matching the user characteristics is selected from the target video period in the at least one video period according to the user characteristics, then select at least one video clip from the video clips split from the target video period according to the heat value of the video clip.

[0017] In some embodiments of the present application, based on the aforementioned scheme, the live video processing device also includes: a third processing unit, configured to obtain interaction data corresponding to each video clip; determine the heat factor of each video clip according to the interaction data corresponding to each video clip; calculate the interaction change trend corresponding to each video clip according to the heat factor of each video clip; calculate the heat value of each video clip based on the heat factor of each video clip and the interaction change trend corresponding to each video clip.

[0018] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit is configured to: synthesize the selected video segments according to the time sequence of the videos to obtain synthesized video segments; based on the synthesized video segments, generate a video highlight segment with a set duration range.

[0019] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit is configured as follows: if the duration of the synthesized video segment exceeds the set duration range, and at least two video segments are selected from the same video time period, then according to the heat value or the degree of match with the user feature, video segments are eliminated from the at least two video segments until the duration of the synthesized video segment is within the set duration range; the synthesized video segment within the set duration range is used as the video highlight segment.

[0020] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit is configured as follows: if the duration of the synthesized video segment exceeds the set duration range, a video segment that meets the set duration range is intercepted from the synthesized video segment as the video highlight segment.

[0021] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit is configured as follows: if the duration of the synthesized video segment does not reach the set duration range, then selecting video segments from other video segments of the at least one video time period according to the heat value until the duration of the synthesized video segment is within the set duration range; and using the synthesized video segment within the set duration range as the video highlight segment.

[0022] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit is also configured to: in the process of presenting the video highlight segment to the user, if a skip play instruction for the video highlight segment is detected, identify the target video segment presented when the skip play instruction is received; and adjust the user characteristics of the user according to the attribute label of the target video segment.

[0023] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit is also configured to: perform semantic analysis and / or content recognition processing on the video clips of the content broadcast in the target live broadcast room to obtain attribute labels of the video clips, and the attribute labels are used to calculate the degree of matching with the user features.

[0024] In some embodiments of the present application, based on the aforementioned solution, the first processing unit is further configured to: adjust the attribute label of the video segment according to the interactive data corresponding to the video segment.

[0025] According to one aspect of an embodiment of the present application, a live video processing device is provided, including: a display unit, configured to display information of a live broadcast room being broadcast; a sending unit, configured to send a joining request for the target live broadcast room to a server if it detects that a user has entered the target live broadcast room, the joining request including identification information of the user; a second receiving unit, configured to receive a video highlight segment transmitted by the server, the video highlight segment being generated based on a video segment selected from the live broadcast content missed by the user; and a playing unit, configured to play the video highlight segment.

[0026] In some embodiments of the present application, based on the aforementioned scheme, the playback unit is further configured to: if a skip playback instruction for the video highlight segment is received, or if the playback of the video highlight segment is completed, play the real-time live content of the target live room.

[0027] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the live video processing method as described in the above embodiment is implemented.

[0028] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the live video processing method as described in the above embodiments.

[0029] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the live video processing method provided in the above-mentioned various optional embodiments.

[0030] In the technical solutions provided in some embodiments of the present application, the live content missed by the user is determined based on the broadcast time of the target live room and the user's identification information, and then video clips are selected from the live content missed by the user, and video highlights clips are generated based on the selected video clips for presentation to the user. When the user enters the live room, he can efficiently understand the broadcast content of the live room based on the video highlights clips, and then quickly keep up with the progress of the video live broadcast, which not only improves the acceptability of the live content, but also improves the user's live viewing experience, and can also improve the user stickiness of the live broadcast platform.

[0031] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0033] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied;

[0034] Figure 2A flowchart of a method for processing live video according to an embodiment of the present application is shown;

[0035] Figure 3 A flowchart of a method for processing live video according to an embodiment of the present application is shown;

[0036] Figure 4 A flowchart of calculating a video heat value according to an embodiment of the present application is shown;

[0037] Figure 5 A flowchart of a method for processing live video according to an embodiment of the present application is shown;

[0038] Figure 6 A flowchart of a method for processing live video according to an embodiment of the present application is shown;

[0039] Figure 7 A schematic diagram showing splitting of played content according to time according to an embodiment of the present application is shown;

[0040] Figure 8 A schematic diagram of splitting a large fragment according to an embodiment of the present application is shown;

[0041] Fig. 9 A schematic diagram showing splitting large fragments according to heat values ​​according to an embodiment of the present application is shown;

[0042] Fig.10 A schematic diagram of labeling small fragments according to an embodiment of the present application is shown;

[0043] Fig.11 A schematic diagram showing small segments selected according to user's interest tags according to an embodiment of the present application is shown;

[0044] Figure 12 to Figure 14 A schematic diagram of an interface of a live broadcast client according to an embodiment of the present application is shown;

[0045] Fig.15 A block diagram of a live video processing device according to an embodiment of the present application is shown;

[0046] Fig.16 A block diagram of a live video processing device according to an embodiment of the present application is shown;

[0047] Fig.17 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0049] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the present application.

[0050] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0051] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0052] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0053] It is understandable that this application can display a prompt interface or pop-up window before collecting relevant user data (such as user data such as user characteristics) and during the process of collecting relevant user data. The prompt interface or pop-up window is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining user-related data after obtaining the user's confirmation operation on the prompt interface or pop-up window, otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining user-related data are terminated, that is, the user's relevant data is not obtained. In other words, all user data collected by this application are collected with the user's consent and authorization, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0054] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown.

[0055] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a game server 102, a live broadcast server 103, a terminal device 104, and a network 105. The terminal device 101 and the terminal device 104 are used by users and may be electronic devices with a live video broadcast function or a live video viewing function. Specifically, the terminal device 101 and the terminal device 104 may be electronic devices such as mobile phones, tablet computers, wearable devices, PCs (Personal Computers), and the like.

[0056] Optionally, the terminal device 101 and the terminal device 104 may be installed with a live broadcast client, which refers to a client used by a user to explain a live video or watch a live video, and the live broadcast client may have the functions of collecting, publishing, downloading and playing live videos. In an illustrative embodiment, the terminal device 101 and the terminal device 104 are provided with a display screen and / or an audio acquisition device, the display screen is used to display the live video, and the audio acquisition device is used to collect the audio data of the anchor.

[0057] In a possible implementation, assuming that the terminal device 101 is a client for live broadcasting of games, and the terminal device 104 is a client for watching live broadcasting of games, then in addition to the live broadcasting client, the terminal device 101 may also be installed with a game client, which may be a game client for shooting games, first-person shooting games (First-Person Shooting game, referred to as FPS), third-person shooting games, multiplayer gunfight survival games, multiplayer online tactical competitive games (Multiplayer Online Battle Arena Games, referred to as MOBA), role-playing games, real-time strategy games, racing games, music games, etc. In addition, if the game server 102 can provide other services, such as virtual reality services, three-dimensional map services, simulation services, etc., then the terminal device 101 may also install corresponding applications for live video broadcasting.

[0058] The game server 102 is a server that provides game services. The game server 102 may include one server or multiple servers. Optionally, the game server 102 may be a server that provides cloud games.

[0059] The live broadcast server 103 is a server that provides background services for the live broadcast client. The live broadcast server 103 may include one server or multiple servers. Optionally, the live broadcast server 103 may be a cloud server.

[0060] like Figure 1 As shown, the terminal device 101, the game server 102, the live broadcast server 103, and the terminal device 104 can communicate with each other through the network 105. For example, the terminal device 101 and the game server 102 can communicate with each other through a wired network or a wireless network; the terminal device 101 and the live broadcast server 103 can communicate with each other through a wired network or a wireless network; the terminal device 104 and the live broadcast server 103 can communicate with each other through a wired network or a wireless network; and the game server 102 and the live broadcast server 103 can communicate with each other through a wired network or a wireless network.

[0061] In one embodiment of the present application, the terminal device 101 is a client for playing live games, and the terminal device 104 is a client for watching live games. The terminal device 101 can log in to the game server 102 through a first account to play games, and can log in to the live broadcast server 103 through a second account to broadcast the ongoing game. The terminal device 104 can obtain the game video broadcasted by the terminal device 101 from the live broadcast server 103, and provide it to the user for viewing.

[0062] Specifically, the terminal device 101 can create a live broadcast room through the live broadcast server 103, and broadcast the game live based on the live broadcast room. The terminal device 104 can choose to enter the live broadcast room corresponding to the terminal device 101 through the live broadcast room list provided by the live broadcast server 103, and watch the corresponding game live broadcast video. Because when the terminal device 104 enters the live broadcast room corresponding to the terminal device 101, the live broadcast room may have been broadcasting for a period of time. When the user needs to know the broadcasted content, one live broadcast solution is to find the playback video released by the anchor to choose to watch the live broadcast playback when the current live broadcast ends and the anchor releases the playback video. This solution causes the user to be unable to view the broadcasted content in time, and the viewing experience is poor. Another live broadcast solution is to choose to watch the broadcasted content by dragging the progress bar when entering the online live broadcast, and then switch back to the current live broadcast progress through operation, but this solution is complicated to operate, has a high learning cost, and cannot accurately and quickly select the content you want to watch.

[0063] In order to prevent the user of the terminal device 104 from failing to keep up with the progress of the current live broadcast, in one embodiment of the present application, after detecting the request sent by the terminal device 104 to join the live broadcast room, the live broadcast server 103 can determine the live broadcast content missed by the user of the terminal device 104 based on the broadcast time of the live broadcast room and the identification information of the user corresponding to the terminal device 104 (the identification information of the user can be the user identification information of the live broadcast client installed on the terminal device 104), and then select at least one video clip from the live broadcast content missed by the user of the terminal device 104 (for example, a video clip matching the user characteristics of the user can be selected), and then generate a video highlight clip based on the selected video clip, and send the video highlight clip to the terminal device 104 for playback, so that the user of the terminal device 104 can efficiently understand the broadcast content of the live broadcast room by watching the video highlight clip, and can quickly keep up with the progress of the video live broadcast, which not only improves the acceptability of the live broadcast content, but also improves the user's live viewing experience, and can also improve the user stickiness of the live broadcast platform.

[0064] Exemplarily, when the terminal device 104 finishes playing the video highlight segment or the user chooses to skip playing the video highlight segment, the real-time live content of the live broadcast room can be played.

[0065] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0066] Figure 2 A flowchart of a method for processing live video according to an embodiment of the present application is shown. The method for processing live video can be executed by a live broadcast server, which can be Figure 1 The live broadcast server 103 shown in FIG. Figure 2As shown, the live video processing method at least includes steps S210 to S240, which are described in detail as follows:

[0067] In step S210, a joining request for a target live broadcast room is received, wherein the joining request includes identification information of a user requesting to join the target live broadcast room.

[0068] In one embodiment of the present application, when a user needs to enter a live broadcast room to watch a live broadcast, he can log in to the live broadcast client and select a target live broadcast room through the live broadcast room list displayed by the live broadcast client. Then, the live broadcast client can send a joining request to the live broadcast server based on the target live broadcast room selected by the user, and add the user's identification information (such as the user's account information in the live broadcast application, etc.) and the information of the target live broadcast room in the joining request.

[0069] Optionally, the target live broadcast room may be a live broadcast room that is currently broadcasting live, and the live broadcast content may be a game battle, a video commentary, an exhibition activity, etc. The joining request may be a joining request triggered when the user enters the target live broadcast room for the first time, or may be a joining request triggered when the user exits midway and then enters the live broadcast room again.

[0070] In step S220, the live broadcast content missed by the user is determined based on the broadcast duration of the target live broadcast room and the identification information of the user.

[0071] In one embodiment of the present application, since the user has identification information and there is information about the time point of joining when the user joins the target live broadcast room, the time point when the user joins the target live broadcast room can be obtained based on the user's identification information, and then the live broadcast time period missed by the user can be determined based on the broadcast time of the target live broadcast room and the time point when the user joined the target live broadcast room, and then the live broadcast content missed by the user can be determined based on the live broadcast time period missed by the user.

[0072] For example, if a live broadcast room starts broadcasting at 9:00 and a user joins the room at 10:15, the user will miss the live broadcast time between 9:00-10:15, that is, the live broadcast content between 9:00-10:15. If a user joins the live broadcast room at 10:15, exits the room at 10:30, and then joins the room again at 10:55, the user will miss the live broadcast time between 10:30-10:55, that is, the live broadcast content between 10:30-10:55.

[0073] In step S230, at least one video clip is selected from the live broadcast content missed by the user.

[0074] In one embodiment of the present application, selecting at least one video clip from the live content missed by the user may be performed randomly, or a video clip may be selected for each live time period, or the selection may be performed based on user characteristics, or the selection may be performed based on the popularity value of the video clip.

[0075] Specifically, if the selection is based on user characteristics, the video clip can be added with an attribute tag, which is used to indicate the type of the video clip. For example, for a game video, it can be a team battle tag, a tag of a game hero, a tag of a hero type (such as a mage hero, an auxiliary hero, etc.), etc., and then the matching degree can be calculated based on the attribute tag and the user characteristics. The user characteristics are used to characterize the user's interests and hobbies. For example, if the user is interested in a game hero or a hero type, then the video clip with the tag of the game hero or the tag of the hero type is a video clip that matches the user's user characteristics. The technical solution of this embodiment can improve the accuracy of the selected at least one video clip, which is conducive to personalized selection.

[0076] If the selection is based on the heat values ​​of the video segments, at least one video segment may be selected in descending order of heat values.

[0077] Optionally, semantic analysis and / or content recognition processing can be performed on the video clips of the content broadcast in the target live broadcast room to obtain attribute labels of the video clips. Among them, video voice analysis and content recognition processing are both based on artificial intelligence technology, specifically computer vision technology, to analyze and process the video clips.

[0078] Specifically, artificial intelligence (AI) technology is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0079] Computer vision (CV) technology is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0080] In one embodiment of the present application, after the attribute label of the video clip is determined, the attribute label of the video clip can be adjusted according to the interactive data corresponding to the video clip. For example, if a video clip has a large number of views or a large number of likes, the attribute label of the video clip can be adjusted, such as adjusting it to a popular video.

[0081] In one embodiment of the present application, when selecting a video clip that matches the user's characteristics from the live broadcast content that the user missed based on the user's characteristics, the similarity between the user's characteristics and the attribute label of the video clip can be calculated. If the similarity is greater than a set threshold, it means that the user's characteristics match the attribute label of the video clip. Optionally, the similarity can be determined by calculating the distance between the user's characteristics and the attribute label, such as the Euclidean distance, the Minkowski distance, the Manhattan distance, etc. The smaller the distance, the greater the similarity.

[0082] Continue to refer to Figure 2 As shown, in step S240, a video highlight segment is generated according to the selected video segment, and the video highlight segment is presented to the user.

[0083] In one embodiment of the present application, the selected video segments may be synthesized and the synthesized video segments may be used as the generated video highlight segments. Optionally, the process of presenting the video highlight segments to the user may be to send the generated video highlight segments to the live broadcast client of the viewer, and then the live broadcast client plays them.

[0084] For example, in the process of presenting a video highlight segment to the user, if a skip play instruction for the video highlight segment is detected, the target video segment presented when the skip play instruction is received can be identified, and then the user characteristics of the user can be adjusted according to the attribute label of the target video segment. Specifically, since the video highlight segment is generated by the selected video segment, when a skip play instruction for the video highlight segment is detected, it may be that the user is not interested in the currently playing video segment, so the user characteristics of the user can be adjusted based on this, and the accuracy of subsequent recommendations can be optimized.

[0085] based on Figure 2 The technical solution of the embodiment shown in the figure, in one embodiment of the present application, as Figure 3 As shown, after step S220, step S310 may also be included, i.e., determining whether the live broadcast duration missed by the user is greater than the first duration, for example, determining the live broadcast duration missed by the user according to the live broadcast period missed by the user.

[0086] If the live broadcast duration missed by the user is greater than the first duration, then step S230 and step S240 are performed again, i.e., the process of selecting a video segment and generating a video highlight segment. If the live broadcast duration missed by the user is less than or equal to the first duration, the real-time live broadcast content of the target live broadcast room can be directly presented to the user. The technical solution of this embodiment enables the real-time live broadcast content to be directly presented to the user when the live broadcast duration missed by the user is short. Optionally, the first duration can be, for example, half an hour, one hour, etc.

[0087] based on Figure 2 and Figure 3 The technical solution of the illustrated embodiment, in one embodiment of the present application, can also segment the broadcast content of the target live broadcast room to generate video clips, so as to select video clips from the generated video clips. Specifically, the broadcast content of the target live broadcast room can be segmented based on the second duration as the segmentation benchmark to obtain multiple video time periods, and then the video content in the video time period can be split to obtain multiple video clips corresponding to the broadcast content. For example, the broadcast content of the target live broadcast room can be divided into multiple 10-minute video time periods based on 10 minutes as the segmentation benchmark, and then these video time periods can be split to obtain multiple video clips corresponding to the broadcast content. Optionally, the second duration can be set according to actual needs, and in addition to being 10 minutes, it can also be, for example, 5 minutes, 15 minutes, 20 minutes, etc.

[0088] In one embodiment of the present application, when splitting the video content in a video period, the video content in the video period can be subjected to semantic analysis, and then the video content in the video period can be split based on the semantic analysis result; or the video content in the video period can be subjected to content recognition processing, and then the video content in the video period can be split based on the content recognition result; or the video content in the video period can be subjected to heat analysis, and then the video content in the video period can be split based on the heat analysis result. Of course, the video content in the video period can also be split by performing multiple processes including semantic analysis, content recognition processing, and heat analysis on the video content. For example, split into video clips of each game hero, split into video clips containing heat peaks, or split into video clips of several game heroes fighting each other, etc.

[0089] After the broadcast content of the target live broadcast room is segmented, in one embodiment of the present application, the process of selecting video clips matching the user's characteristics from the live content missed by the user may be: based on multiple video clips corresponding to the broadcast content, determine at least one video period to which the live content missed by the user belongs, and then select at least one video clip from at least one video period. That is, in order to ensure the content continuity of the generated video highlights clips, at least one video clip may be selected from each video period missed by the user for synthesis processing.

[0090] Optionally, when selecting video clips from each video period missed by the user, it is possible to select video clips matching the user characteristics from each video period only by relying on the user characteristics, or to select video clips with higher heat values ​​from each video period only by relying on the heat values, or to select video clips combining the user characteristics and the heat values. Specifically, if a video clip matching the user characteristics is not selected from the target video period missed by the user based on the user characteristics, then at least one video clip can be selected from the video clips obtained by splitting the target video period based on the heat values ​​of the video clips. That is, one or more video clips can be selected in descending order of heat values.

[0091] In one embodiment of the present application, a solution for calculating the heat value of a video clip is also proposed. Figure 4 As shown, the following steps are included:

[0092] Step S410, obtaining the interaction data corresponding to each video segment.

[0093] In one embodiment of the present application, the interactive data corresponding to the video clip may include: data on the number of viewers, data on the number of comments, barrage data, and live interaction data.

[0094] Step S420: determining the heat factor of each video segment according to the interaction data corresponding to each video segment.

[0095] In one embodiment of the present application, the heat factor may be, for example, the number of viewers, the number of comments, the number of bullet screens, the duration of the live broadcast interaction, the number of gifts received, etc. If a video clip has multiple heat factors, then these multiple heat factors may be counted (such as averaging, or weighted averaging, etc.) to obtain the final heat factor of the video clip.

[0096] Step S430, calculating the interaction change trend corresponding to each video segment according to the heat factor of each video segment.

[0097] In one embodiment of the present application, the interaction change trend is used to feedback the interaction change of the live content, for example, the interaction change trend can be measured by the change trend of the heat factor over time. Optionally, the interaction change trend can be represented by the natural index of the derivative of the heat factor.

[0098] Step S440, calculating the heat value of each video segment based on the heat factor of each video segment and the interaction change trend corresponding to each video segment.

[0099] In one embodiment of the present application, the heat factor of the video clip and the interaction change trend corresponding to the video clip may be combined to obtain the heat value of the video clip.

[0100] Specifically, the heat value of a video clip can be calculated using the following formula:

[0101] H=Base(P)+e f '(P)

[0102] Among them, H represents the heat value of the video clip; P represents the heat factor; Base(P) represents the base of the heat factor, such as the number of viewers; f'(P) represents the derivative of the dynamic change of the heat factor, which reflects the trend of interactive changes.

[0103] In one embodiment of the present application, Figure 2 and Figure 3 The process of generating a video highlight segment based on the selected video segments in step S240 shown may be: synthesizing the selected video segments in the time sequence of the videos to obtain synthesized video segments, and then generating a video highlight segment with a set duration range based on the synthesized video segments.

[0104] Specifically, if the duration of the synthesized video segment is within the set duration range, the synthesized video segment can be directly used as a video highlights segment.

[0105] If the duration of the synthesized video clip exceeds the set duration range, and at least two video clips are selected from the same video period, then video clips can be removed from the at least two video clips according to the heat value or the degree of match with the user characteristics until the duration of the synthesized video clip is within the set duration range, and then the synthesized video clip within the set duration range is used as the video collection clip. Optionally, video clips with lower heat values ​​or video clips with lower matching degrees with user characteristics can be removed.

[0106] Of course, if the duration of the synthesized video clip exceeds the set duration range, the video clip that meets the set duration range can be directly intercepted from the synthesized video clip to serve as the video highlights clip. For example, the second half of the synthesized video clip can be intercepted, that is, intercepted from the back to the front, to ensure that the generated video highlights clip has a high correlation with the real-time live content. Alternatively, a portion can be intercepted from the first half, the middle part, and the second half of the synthesized video clip to generate a video highlights clip.

[0107] If the duration of the synthesized video segment does not reach the set duration range, video segments can be selected from other video segments of at least one video time period according to the heat value until the duration of the synthesized video segment is within the set duration range, and then the synthesized video segment within the set duration range is used as the video highlight segment.

[0108] Figures 2 to 4 This is an explanation from the perspective of the live broadcast server. Figure 5 The technical solution of the embodiment of the present application is described in detail from the perspective of the live broadcast client:

[0109] Figure 5 A flowchart of a method for processing live video according to an embodiment of the present application is shown. The method for processing live video can be executed by a live client, which can be run on Figure 1 In the terminal device 104 shown in FIG. Figure 5 As shown, the live video processing method at least includes steps S510 to S540, which are described in detail as follows:

[0110] In step S510, the live broadcast room information is displayed.

[0111] In one embodiment of the present application, a live broadcast client (specifically, a client for watching live broadcasts) can obtain information about the live broadcast room that is being broadcast from the live broadcast server, and then display the information about the live broadcast room that is being broadcast on an interface on the live broadcast client (such as being displayed in the form of a list or in the form of thumbnails, etc.). The live broadcast room information can include the name, brief introduction information, live broadcast content, live broadcast time period, etc. of the live broadcast room.

[0112] In step S520, if it is detected that the user enters the target live broadcast room, a joining request for the target live broadcast room is sent to the server, and the joining request includes the user's identification information.

[0113] In one embodiment of the present application, when a user needs to enter a live broadcast room to watch a live broadcast, he can log in to the live broadcast client and select a target live broadcast room through the live broadcast room information displayed by the live broadcast client. Then, the live broadcast client can send a joining request to the live broadcast server based on the target live broadcast room selected by the user, and add the user's identification information (such as the user's account information in the live broadcast application, etc.) and the information of the target live broadcast room in the joining request.

[0114] In step S530, a video highlight segment transmitted by the server is received, where the video highlight segment is generated based on a video segment selected from the live broadcast content missed by the user.

[0115] The process of generating the video highlight segments on the server side may refer to the technical solution of the aforementioned embodiment and will not be described in detail.

[0116] In step S540, the video highlights segment is played.

[0117] Optionally, when the user clicks to enter the target live broadcast room and obtains the video highlight clip, the video highlight clip can be played directly on the interface of the live broadcast client. Alternatively, a playback control of the video highlight clip can be displayed on the interface of the live broadcast client, and the video highlight clip can be played after a trigger instruction to the playback control is detected.

[0118] In one embodiment of the present application, during the process of playing the video highlight segment, if a skip play instruction for the video highlight segment is received, or if the video highlight segment is played, the real-time live content of the target live broadcast room can be played. Optionally, in this embodiment, the play window of the video highlight segment and the play window of the real-time live content of the live broadcast room can be the same.

[0119] In addition, in one embodiment of the present application, a playback window for video highlights and a playback window for real-time live content can be displayed on the interface of the live client, and then the user can choose whether to play the video highlights or the real-time live content based on the trigger action of the user.

[0120] The technical solutions of the embodiments of the present application are described above from the perspectives of the live broadcast server and the live broadcast client. Figures 6 to 14 The implementation details of the technical solution of the embodiment of the present application are elaborated in detail:

[0121] like Figure 6 As shown, a method for processing live video according to an embodiment of the present application includes the following steps:

[0122] Step S601: After the user enters the live broadcast room, the live broadcast client sends the time node and user ID to the live broadcast server.

[0123] Specifically, when a user enters the live broadcast room and starts watching the online live broadcast, the live broadcast client sends a request to the live broadcast server, and the request may include time node information and user ID. Optionally, the time node information may be the time point when the user enters the live broadcast room, and the live broadcast server may determine the live broadcast time period missed by the user based on the time node information; or the time node information may also be the live broadcast time period missed by the user determined by the live broadcast client.

[0124] Step S602: The live broadcast server matches the interest tags under the user ID and the popularity of the video clips, generates a video highlight and sends it to the client.

[0125] In one embodiment of the present application, when the anchor of the live broadcast room starts broadcasting, the live broadcast service end can start to obtain the broadcast content of the live broadcast room in real time, and then split it into large segments according to time (i.e., the video time period in the above embodiment), and store it in sequence as Video1, Video2, Video3... Figure 7 As shown, the durations of the large segments divided into segments may be the same or different, and the duration of each large segment may be set according to actual needs.

[0126] For the video content in each large clip, we can combine semantic analysis and AI algorithms for picture content recognition, or split it according to the heat distribution of the video content (such as the peak value of the band) to obtain small clips and name them in sequence. Figure 8 As shown, Video1 can be split into small segments v1.1, v1.2, v1.3, v1.4; Video2 can be split into small segments v2.1, v2.2, v2.3, v2.4; Video3 can be split into small segments v3.1, v3.2, and so on.

[0127] Assuming that Video1 is split according to the heat distribution, then Fig. 9As shown in the figure, the heat curve of Video1 contains the peaks and troughs of the heat value, so we can split it with the peak of each heat value as the center to get small segments v1.1, v1.2, v1.3, and v1.4. When the AI ​​algorithm combining semantic analysis and screen content recognition is used for splitting, the large segment can be divided into multiple small segments with relatively related content, such as small segments of each game hero.

[0128] After the large clip is split into small clips, semantic analysis and picture content recognition can be performed on each small clip to label each small clip with a corresponding attribute label, and the small clip and the attribute label are stored together in the live broadcast server. Fig.10 As shown, for example, the small segment v1.1 is tagged with attribute a; the small segment v1.2 is tagged with attribute b; the small segment v1.3 is tagged with attribute a; the small segment v1.4 is tagged with attribute c; the small segment v2.1 is tagged with attribute d; the small segment v2.2 is tagged with attribute f; the small segment v2.3 is tagged with attribute g; the small segment v2.4 is tagged with attribute a; the small segment v3.1 is tagged with attribute b; the small segment v3.2 is tagged with attribute f.

[0129] Optionally, these attribute tags may be, for example, game hero tags, to indicate that the small clip is a small clip corresponding to a certain game hero; or the attribute tag may be a hero type tag, to indicate that the small clip is a small clip corresponding to a certain hero type; or the attribute tag may be a battle type, to indicate whether the small clip is a team video clip or a one-on-one video clip, etc.

[0130] In one embodiment of the present application, the heat value score of each small segment can also be obtained according to the amount of interaction per unit time of each small segment. The amount of interaction can include factors such as the number of viewers, the number of comments, and the number of bullet screens of the small segment. At the same time, since live broadcast and real-time data are related, variables with time-varying trends can also be considered when calculating the heat value score of each small segment. Optionally, the natural exponent of the derivative of the heat factor P can be taken to represent the changing trend.

[0131] For example, in one embodiment, the heat value score of each small segment can be calculated by the following formula: H = Base (P) + e f ' (P) .

[0132] Among them, P represents the heat factor, which can be the number of people watching the live broadcast, or the amount of gifts sent, the average number of live comments per person, the number of barrages, the length of interaction with the microphone, etc. When there are multiple heat factors, the weighted average of multiple heat factors can be taken to achieve the purpose of balancing operations such as screen swiping. Base(P) represents the cardinality of the heat factor, such as the number of people watching the live broadcast in real time; and f′'(P) represents the derivative of the dynamic change of the number of people, reflecting the trend of changes in live interaction. Optionally, the above expression can be understood as: when the base number of people watching the live broadcast is large, the interactions are high, and the number of people in the past unit time has increased rapidly or the number of interactions has increased suddenly, the heat is high.

[0133] In one embodiment of the present application, the user's interest tag is the user feature, and the attribute tag of the video clip is the feature of the video clip. The user feature and the feature of the video clip are two related concepts. The initial features of the video clip can be divided manually, such as the anchor can choose the corresponding category when starting the broadcast, or the platform manually tags according to the content. The features of the video clip are not static. They can be determined by the weighted interaction of the users who watch the video as a whole and the interactive behavior. For example, the higher the interaction of users with corresponding interest tags with a certain video clip and the longer the viewing time, the greater the influence of the user's interest tag on the attribute tag of the video clip, and the attribute tag of the video clip can be adjusted accordingly.

[0134] User features are feature vectors formed based on the favorite categories actively selected by users and their behavior data (such as following behavior, commenting and liking behavior in the live broadcast room, etc.).

[0135] When associating user features with features of a small video clip, the user features can be used as feature vector x, and the features of the small video clip can be used as feature vector y, for example, feature vector x = (x1, x2, ..., x n ), eigenvector y=(y1,y2,...,y n ), these two eigenvectors can be represented as two points in Euclidean space, then the Euclidean distance d between point x and point y is Euclidean (x,y) or d Euclidean (y,x) can be expressed as:

[0136]

[0137] The Euclidean distance between two feature vectors can indicate the correlation between the feature vectors. The smaller the value, the higher the matching degree between the two feature vectors. Therefore, one or more small clips with high matching degree can be selected from each large clip to synthesize video highlights and show them to users, so as to achieve the purpose of different faces for different users.

[0138] Specifically, if Fig.11 As shown, assuming that the user's interest tags are a, d, e, and g, then by matching them with the attribute tags of the video clips, the matching video clips can be v1.1, v1.3, v2.1, v2.3, and v2.4.

[0139] Since there are no small clips matching the user's interest tags in the large clip Video3, and in order to ensure the continuity of the video highlights content, it is necessary to select at least one small clip from each large clip. Therefore, a small clip can be selected from Video3 according to the heat value. Assume that the small clips finally selected are v1.1, v1.3, v2.1, v2.3, v2.4, and v3.2.

[0140] After screening out the small clips, they can be edited and synthesized into a video highlight in the live broadcast server in chronological order. If the duration of the synthesized video highlights is greater than the preset 1 minute (and ± 5 seconds), it can be sorted according to the heat value, and the small clips with low heat values ​​can be removed to meet the duration of the video highlights equal to 1 minute (and ± 5 seconds). After removing the small clips, it is also necessary to meet the requirement that at least 1 small clip is selected in each large clip to ensure the continuity of the content. Of course, in one embodiment of the present application, if the duration of the synthesized video highlights is greater than the preset 1 minute (and ± 5 seconds), a duration of 1 minute (and ± 5 seconds) can also be intercepted therefrom as the final video highlights.

[0141] If the duration of the synthesized video highlights is less than the preset 1 minute (and ±5 seconds), other video clips can be added in order according to the heat value until the synthesized video highlights meet the duration requirement. The duration requirement in this embodiment is only an example and can be flexibly set according to needs in actual application.

[0142] After the live broadcast client generates the video highlights, it can be sent to the live broadcast client.

[0143] Continue to refer to Figure 6 As shown, the following steps are also included:

[0144] Step S603, the live broadcast client starts playing the video highlights, for example, playing the video highlights on the live broadcast room interface of the live broadcast client.

[0145] Step S604, the live broadcast client determines whether to skip the video highlights. If not, after playing the video highlights, the live broadcast progress of the current live broadcast room is played; if skipping the video highlights, the live broadcast progress of the current live broadcast room is played directly.

[0146] Step S605, if the live broadcast client skips the video highlights, the live broadcast server identifies the attribute tag a of the video segment played when skipping, and then reduces the weight of tag a in the interest tag under the current user ID to reduce the recommendation of such video segments.

[0147] based on Figure 6 The technical solution of the embodiment shown in the figure, in an application scenario of the present application, such as Fig.12 As shown, the live broadcast list interface of the live broadcast client displays multiple live broadcast rooms that are broadcasting live. Users can enter a live broadcast room by clicking. For example, after clicking area 1201 in the interface, they enter a live broadcast room named "National Service Jungle: I'm Back, Go Go Go". After the user enters the live broadcast room, the live broadcast server will automatically generate a 1-minute (for example only) video collection in real time by extracting several clips from the broadcast content according to the current user's interest tags (optionally, it will only be generated when the broadcast content that the user has not watched exceeds a certain threshold, such as 1 hour), and play it through the live broadcast client. As shown in the figure below, Fig.13 As shown, after entering the live broadcast room, the video highlights will be played in the window 1301 of the live broadcast interface, and the lower area of ​​the window 1301 can display a play progress bar 1303. At the same time, the window 1301 can also display a guide button 1302 to prompt the user to skip the video highlights by triggering the guide button 1302.

[0148] After the live client finishes playing the video highlights, Fig.14 As shown, the current live broadcast progress can be automatically played in window 1401. Of course, the user can also click the guide button 1302 to skip the video highlights and directly enter Fig.14 Click the interface shown to watch the current live broadcast progress.

[0149] It should be noted that: if the user exits the live broadcast room while watching a live broadcast, when he enters the live broadcast room again, if the time interval of not watching exceeds the threshold time of 1 hour for generating video highlights (just an example), a new video highlight of the unwatched period will also be generated in real time.

[0150] The technical solutions of the aforementioned embodiments of the present application enable users to efficiently understand the broadcast content of the live broadcast room based on the video highlights when entering the live broadcast room, and then quickly keep up with the progress of the live video broadcast, which not only improves the acceptability of the live broadcast content, but also improves the user's live viewing experience, and can also increase the user stickiness of the live broadcast platform.

[0151] The following describes an embodiment of the device of the present application, which can be used to execute the live video processing method in the above embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the live video processing method in the above embodiment of the present application.

[0152] Fig.15 A block diagram of a live video processing device according to an embodiment of the present application is shown. The live video processing device can be set in a live broadcast server, which can be Figure 1 The live broadcast server 103 shown in .

[0153] Reference Fig.15 As shown, according to an embodiment of the present application, a live video processing device 1500 includes: a first receiving unit 1502, a determining unit 1504, a selecting unit 1506 and a first processing unit 1508.

[0154] Among them, the first receiving unit 1502 is configured to receive a joining request for a target live broadcast room, and the joining request includes identification information of a user requesting to join the target live broadcast room; the determination unit 1504 is configured to determine the live broadcast content missed by the user based on the broadcast time of the target live broadcast room and the identification information of the user; the selection unit 1506 is configured to select at least one video clip from the live broadcast content missed by the user; the first processing unit 1508 is configured to generate a video highlight clip based on the selected video clip, and present the video highlight clip to the user.

[0155] In some embodiments of the present application, based on the aforementioned scheme, the determination unit 1504 is configured to: obtain the time point when the user joins the target live broadcast room according to the identification information of the user; determine the live broadcast time period missed by the user according to the broadcast time of the target live broadcast room and the time point when the user joins the target live broadcast room, so as to determine the live broadcast content missed by the user based on the live broadcast time period missed by the user.

[0156] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit 1508 is further configured to: before selecting at least one video clip from the live broadcast content missed by the user, determine the live broadcast duration missed by the user according to the live broadcast period missed by the user; if the live broadcast duration missed by the user is less than or equal to the first duration, present the real-time live broadcast content of the target live broadcast room to the user; the selection unit is configured to: if the live broadcast duration missed by the user is greater than the first duration, execute the process of selecting at least one video clip from the live broadcast content missed by the user.

[0157] In some embodiments of the present application, based on the aforementioned scheme, the live video processing device 1500 also includes: a second processing unit, configured to segment the broadcast content of the target live room based on the second duration as a segmentation basis, to obtain multiple video time periods; and split the video content in the video time period to obtain multiple video clips corresponding to the broadcast content.

[0158] In some embodiments of the present application, based on the above solution, the second processing unit is configured to split the video content in the video period in at least one of the following ways:

[0159] Performing semantic analysis on the video content in the video period to split the video content in the video period based on the semantic analysis result;

[0160] Performing content recognition processing on the video content in the video period to split the video content in the video period based on the content recognition result;

[0161] Performing heat analysis on the video content in the video period to split the video content in the video period based on the heat analysis result.

[0162] In some embodiments of the present application, based on the aforementioned scheme, the selection unit 1506 is configured to: determine at least one video time period to which the live content missed by the user belongs based on multiple video clips corresponding to the played content; and select at least one video clip from the at least one video time period.

[0163] In some embodiments of the present application, based on the aforementioned scheme, the selection unit 1506 is configured to: select video clips matching the user characteristics from the at least one video period according to the user characteristics of the user; if no video clip matching the user characteristics is selected from the target video period in the at least one video period according to the user characteristics, then select at least one video clip from the video clips split from the target video period according to the heat value of the video clip.

[0164] In some embodiments of the present application, based on the aforementioned scheme, the live video processing device 1500 also includes: a third processing unit, configured to obtain interaction data corresponding to each video clip; determine the heat factor of each video clip according to the interaction data corresponding to each video clip; calculate the interaction change trend corresponding to each video clip according to the heat factor of each video clip; calculate the heat value of each video clip based on the heat factor of each video clip and the interaction change trend corresponding to each video clip.

[0165] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit 1508 is configured to: synthesize the selected video segments according to the time sequence of the video to obtain synthesized video segments; based on the synthesized video segments, generate a video highlight segment with a set duration range.

[0166] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit 1508 is configured as follows: if the duration of the synthesized video segment exceeds the set duration range, and at least two video segments are selected from the same video time period, then according to the heat value or the degree of match with the user feature, video segments are eliminated from the at least two video segments until the duration of the synthesized video segment is within the set duration range; and the synthesized video segment within the set duration range is used as the video highlight segment.

[0167] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit 1508 is configured as: if the duration of the synthesized video segment exceeds the set duration range, then a video segment that meets the set duration range is intercepted from the synthesized video segment as the video highlight segment.

[0168] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit 1508 is configured as follows: if the duration of the synthesized video segment does not reach the set duration range, then video segments are selected from other video segments of the at least one video time period according to the heat value until the duration of the synthesized video segment is within the set duration range; and the synthesized video segment within the set duration range is used as the video highlight segment.

[0169] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit 1508 is also configured to: in the process of presenting the video highlight segment to the user, if a skip play instruction for the video highlight segment is detected, identify the target video segment presented when the skip play instruction is received; and adjust the user characteristics of the user according to the attribute label of the target video segment.

[0170] In some embodiments of the present application, based on the aforementioned scheme, the first processing unit 1508 is also configured to: perform semantic analysis and / or content recognition processing on the video clips of the content played in the target live broadcast room to obtain attribute labels of the video clips, and the attribute labels are used to calculate the degree of match with the user features.

[0171] In some embodiments of the present application, based on the aforementioned solution, the first processing unit 1508 is further configured to: adjust the attribute label of the video segment according to the interactive data corresponding to the video segment.

[0172] Fig.16 A block diagram of a live video processing device according to an embodiment of the present application is shown. The live video processing device can be set in a live client, which can be Figure 1 The terminal device 104 shown in .

[0173] Reference Fig.16 As shown, a live video processing device 1600 according to an embodiment of the present application includes: a display unit 1602, a sending unit 1604, a second receiving unit 1606 and a playing unit 1608.

[0174] Among them, the display unit 1602 is configured to display the information of the live broadcast room being broadcast; the sending unit 1604 is configured to send a joining request for the target live broadcast room to the server if it detects that a user has entered the target live broadcast room, and the joining request includes the identification information of the user; the second receiving unit 1606 is configured to receive the video highlight clips transmitted by the server, and the video highlight clips are generated based on the video clips selected from the live content missed by the user; the playing unit 1608 is configured to play the video highlight clips.

[0175] In some embodiments of the present application, based on the aforementioned scheme, the playback unit 1608 is further configured to: if a skip playback instruction for the video highlight segment is received, or if the playback of the video highlight segment is completed, play the real-time live content of the target live room.

[0176] Fig.17 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0177] It should be noted that Fig.17 The computer system 1700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0178] like Fig.17 As shown, the computer system 1700 includes a central processing unit (CPU) 1701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1702 or the program loaded from the storage part 1708 to the random access memory (RAM) 1703, such as executing the method described in the above embodiment. In RAM 1703, various programs and data required for system operation are also stored. CPU 1701, ROM 1702 and RAM 1703 are connected to each other through bus 1704. Input / output (I / O) interface 1705 is also connected to bus 1704.

[0179] The following components are connected to the I / O interface 1705: an input section 1706 including a keyboard, a mouse, etc.; an output section 1707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1708 including a hard disk, etc.; and a communication section 1709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1709 performs communication processing via a network such as the Internet. A drive 1710 is also connected to the I / O interface 1705 as needed. A removable medium 1711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1710 as needed so that a computer program read therefrom is installed into the storage section 1708 as needed.

[0180] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1709, and / or installed from a removable medium 1711. When the computer program is executed by a central processing unit (CPU) 1701, various functions defined in the system of the present application are executed.

[0181] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0182] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0184] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0185] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0186] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0187] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0188] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing live video, characterized in that: include: Receiving a joining request for a target live broadcast room, wherein the joining request includes identification information of a user requesting to join the target live broadcast room; Determining the live broadcast content missed by the user according to the broadcasting time of the target live broadcast room and the identification information of the user; Selecting at least one video clip from the live broadcast content missed by the user; generating a video highlight segment according to the selected video segment, and presenting the video highlight segment to the user; If the video highlight segment is played completely or a skip play instruction for the video highlight segment is received, the real-time live content of the target live broadcast room is played.

2. The method for processing live video according to claim 1, characterized in that: Determining the live broadcast content missed by the user according to the broadcasting time of the target live broadcast room and the identification information of the user, including: According to the identification information of the user, obtaining the time point when the user joins the target live broadcast room; The live broadcast time period missed by the user is determined according to the broadcasting time of the target live broadcast room and the time point when the user joins the target live broadcast room, so as to determine the live broadcast content missed by the user based on the live broadcast time period missed by the user.

3. The method for processing live video according to claim 2, characterized in that: Before selecting at least one video clip from the live content missed by the user, the live video processing method further includes: Determine the length of the live broadcast time missed by the user according to the live broadcast time period missed by the user; If the live broadcast duration missed by the user is less than or equal to the first duration, presenting the real-time live broadcast content of the target live broadcast room to the user; If the live broadcast duration missed by the user is greater than the first duration, a process of selecting at least one video clip from the live broadcast content missed by the user is executed.

4. The method for processing live video according to claim 1, characterized in that: Before selecting at least one video clip from the live content missed by the user, the live video processing method further includes: Using the second duration as a segmentation benchmark, segmenting the broadcasted content of the target live broadcast room to obtain multiple video time periods; The video content in the video time period is split to obtain a plurality of video clips corresponding to the played content.

5. The method for processing live video according to claim 4, characterized in that: The video content in the video period is split, including at least one of the following methods: Performing semantic analysis on the video content in the video period to split the video content in the video period based on the semantic analysis result; Performing content recognition processing on the video content in the video period to split the video content in the video period based on the content recognition result; Performing heat analysis on the video content in the video period to split the video content in the video period based on the heat analysis result.

6. The method for processing live video according to claim 4, characterized in that: Selecting at least one video clip from the live broadcast content missed by the user, comprising: Determine, based on the multiple video clips corresponding to the played content, at least one video time period to which the live content missed by the user belongs; At least one video segment is selected from the at least one video period respectively.

7. The method for processing live video according to claim 6, characterized in that: Selecting at least one video segment from the at least one video period respectively includes: According to the user characteristics of the user, respectively select video clips matching the user characteristics from the at least one video period; If no video segment matching the user feature is selected from the target video segment in the at least one video segment according to the user feature, at least one video segment is selected from the video segments split from the target video segment according to the heat values ​​of the video segments.

8. The method for processing live video according to claim 7, characterized in that: The live video processing method further includes: Obtaining interactive data corresponding to each video clip; Determining the heat factor of each video segment according to the interaction data corresponding to each video segment; Calculating the interaction change trend corresponding to each video clip according to the heat factor of each video clip; Based on the heat factor of each video segment and the interaction change trend corresponding to each video segment, the heat value of each video segment is calculated.

9. The method for processing live video according to any one of claims 6 to 8, characterized in that: Generate video highlights based on the selected video clips, including: Performing synthesis processing on the selected video clips according to the time sequence of the videos to obtain synthesized video clips; Based on the synthesized video segments, a video highlight segment with a set duration range is generated.

10. The method for processing live video according to claim 9, characterized in that: Based on the synthesized video clips, a video highlight clip with a set duration range is generated, including: If the duration of the synthesized video segment exceeds the set duration range, and at least two video segments are selected from the same video period, then according to the heat value or the matching degree with the user feature, video segments are removed from the at least two video segments until the duration of the synthesized video segment is within the set duration range; The synthesized video segments within the set duration range are used as the video highlights segments.

11. The method for processing live video according to claim 9, characterized in that: Based on the synthesized video clips, a video highlight clip with a set duration range is generated, including: If the duration of the synthesized video segments exceeds the set duration range, a video segment that meets the set duration range is intercepted from the synthesized video segments to serve as the video highlights segment.

12. The method for processing live video according to claim 9, characterized in that: Based on the synthesized video clips, a video highlight clip with a set duration range is generated, including: If the duration of the synthesized video segment does not reach the set duration range, then select video segments from other video segments of the at least one video period according to the heat value until the duration of the synthesized video segment is within the set duration range; The synthesized video segments within the set duration range are used as the video highlights segments.

13. The method for processing live video according to any one of claims 7 to 8, characterized in that: The live video processing method further includes: In the process of presenting the video highlight segment to the user, if a skip play instruction for the video highlight segment is detected, identifying a target video segment presented when the skip play instruction is received; The user characteristics of the user are adjusted according to the attribute label of the target video segment.

14. The method for processing live video according to any one of claims 7 to 8, characterized in that: The live video processing method further includes: The video clip of the content played in the target live broadcast room is subjected to semantic analysis and / or content recognition processing to obtain an attribute label of the video clip, and the attribute label is used to calculate the matching degree between the video clip and the user feature.

15. The method for processing live video according to claim 14, characterized in that: The live video processing method further includes: The attribute label of the video segment is adjusted according to the interactive data corresponding to the video segment.

16. A method for processing live video, characterized in that: include: Display the live broadcast room information; If it is detected that the user enters the target live broadcast room, a joining request for the target live broadcast room is sent to the server, wherein the joining request includes the identification information of the user; Receiving a video highlight segment transmitted by the server, wherein the video highlight segment is generated based on a video segment selected from the live broadcast content missed by the user; wherein the live broadcast content missed by the user is determined based on the broadcast duration of the target live broadcast room and the identification information of the user; Play the video highlights clip; If the video highlight segment is played completely or a skip play instruction for the video highlight segment is received, the real-time live content of the target live broadcast room is played.

17. A live video processing device, characterized in that: include: A first receiving unit is configured to receive a joining request for a target live broadcast room, wherein the joining request includes identification information of a user requesting to join the target live broadcast room; A determination unit, configured to determine the live broadcast content missed by the user according to the broadcasting time of the target live broadcast room and the identification information of the user; A selection unit, configured to select at least one video clip from the live broadcast content missed by the user; The first processing unit is configured to generate a video highlight segment based on the selected video segment, present the video highlight segment to the user, and play the real-time live content of the target live broadcast room if the video highlight segment is played or a skip instruction for the video highlight segment is received.

18. A live video processing device, characterized in that: include: A display unit, configured to display live broadcast room information being broadcast; A sending unit, configured to send a joining request for the target live broadcast room to a server if it is detected that the user enters the target live broadcast room, wherein the joining request includes identification information of the user; a second receiving unit configured to receive a video highlight segment transmitted by the server, wherein the video highlight segment is generated based on a video segment selected from the live broadcast content missed by the user; wherein the live broadcast content missed by the user is determined based on the broadcast duration of the target live broadcast room and the identification information of the user; The playback unit is configured to play the video highlight segment, and if the video highlight segment is played or a skip playback instruction for the video highlight segment is received, play the real-time live content of the target live broadcast room.

19. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the live video processing method as described in any one of claims 1 to 15; or implements the live video processing method as described in claim 16.

20. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the live video processing method as described in any one of claims 1 to 15; or implement the live video processing method as described in claim 16.

21. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium. The processor of the electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device executes the live video processing method described in any one of claims 1 to 15; or implements the live video processing method described in claim 16.

Images