Video content extraction method and apparatus

Abstract

The application provides a video content extraction method and device, wherein the video content extraction method comprises the following steps: determining target interaction data corresponding to a target moment in a video to be extracted, wherein the target interaction data comprises barrage data and gift sending data; performing fuzzification processing on the gift sending data of the target moment to obtain fuzzy gift data of the target moment; determining a high-energy value of the target moment according to the fuzzy gift data and the barrage data of the target moment; and extracting a high-energy segment from the video to be extracted based on the high-energy values of each moment in the video to be extracted. In this way, the characteristics of sending barrages and sending gifts by users during video watching are utilized, the cost is extremely low, only the historical data of the video to be extracted needs to be counted, a large amount of processing resources can be saved, and the sensitive gift sending data is desensitized, thereby improving the security of the sensitive data.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method for extracting video content. This application also relates to a video content extraction apparatus, a computing device, and a computer-readable storage medium. Background Technology

[0002] With the continuous development of computer and internet technology, all kinds of videos are emerging. Video content is often extensive and may include a lot of redundant information, making it difficult for users to quickly access the exciting and high-energy content.

[0003] In existing technologies, AI (Artificial Intelligence) content understanding can be used to understand audio and video, identify exciting and high-energy content in videos, and provide it to users for viewing. However, AI-based content understanding methods require a large amount of computing resources and need to be developed for different fields, resulting in long development cycles, low efficiency in video content extraction, and a low return on investment. Summary of the Invention

[0004] In view of this, embodiments of this application provide a video content extraction method. This application also relates to a video content extraction apparatus, a computing device, and a computer-readable storage medium, to solve the problems of high computing resource consumption, low efficiency of video content extraction, and low input-output ratio in the prior art.

[0005] According to a first aspect of the embodiments of this application, a video content extraction method is provided, including:

[0006] Determine the target interaction data corresponding to the target moment in the video to be extracted. The target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data.

[0007] The gift-giving data at the target time is fuzzed to obtain fuzzy gift data at the target time.

[0008] Based on the fuzzy gift data and bullet screen data at the target time, determine the high-energy value at the target time;

[0009] High-energy segments are extracted from the video based on the high-energy values ​​at various times in the video.

[0010] According to a second aspect of the embodiments of this application, a video content extraction apparatus is provided, comprising:

[0011] Determine the target interaction data corresponding to the target moment in the video to be extracted. The target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data.

[0012] The gift-giving data at the target time is fuzzed to obtain fuzzy gift data at the target time.

[0013] Based on the fuzzy gift data and bullet screen data at the target time, determine the high-energy value at the target time;

[0014] High-energy segments are extracted from the video based on the high-energy values ​​at various times in the video.

[0015] According to a third aspect of the embodiments of this application, a computing device is provided, comprising:

[0016] Memory and processor;

[0017] Memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the following methods:

[0018] Determine the target interaction data corresponding to the target moment in the video to be extracted. The target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data.

[0019] The gift-giving data at the target time is fuzzed to obtain fuzzy gift data at the target time.

[0020] Based on the fuzzy gift data and bullet screen data at the target time, determine the high-energy value at the target time;

[0021] High-energy segments are extracted from the video based on the high-energy values ​​at various times in the video.

[0022] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of an arbitrary video content extraction method.

[0023] The video content extraction method provided in this application determines the target interaction data corresponding to a target moment in the video to be extracted, wherein the target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data; the gift-giving data at the target moment is blurred to obtain blurred gift data at the target moment; based on the blurred gift data and bullet screen data at the target moment, the high-energy value of the target moment is determined; and based on the high-energy values ​​at each moment in the video to be extracted, high-energy segments are extracted from the video to be extracted.

[0024] In this scenario, interactive data such as bullet comments and gift-giving data from the video to be extracted can be obtained. Since gift-giving data may be sensitive and cannot be directly used or displayed, it can be obfuscated to obtain fuzzy gift data. This data is then combined with the bullet comment data and fuzzy gift data to determine high-energy values. Based on these high-energy values ​​at various moments in the video, the most exciting high-energy segments are automatically extracted. This approach leverages the characteristics of users sending bullet comments and gifts while watching videos, at extremely low cost. Only historical data from the video to be extracted needs to be collected, saving significant processing resources and resulting in a high return on investment. It can quickly generate high-energy segments from the video, with high extraction efficiency, facilitating subsequent user-generated content. Furthermore, the sensitive gift-giving data has been anonymized, improving the security of sensitive data. Attached Figure Description

[0025] Figure 1 This is a flowchart of a video content extraction method provided in an embodiment of this application;

[0026] Figure 2a This is a bar chart of bullet screen data provided in one embodiment of this application;

[0027] Figure 2b This is a bar chart of fuzzy gift data provided in one embodiment of this application;

[0028] Figure 2c This is a bar chart of high-energy values ​​provided in one embodiment of this application;

[0029] Figure 3 This is a flowchart illustrating a video content extraction method for live video scenarios provided in an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of the structure of a video content extraction device provided in one embodiment of this application;

[0031] Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this application. Detailed Implementation

[0032] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0033] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a” and “the” as used in one or more embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0035] First, the terms and concepts involved in one or more embodiments of this application will be explained.

[0036] Live broadcast: The English term for live broadcast is "Live broadcast." In a broad sense, live broadcast also includes live broadcasts by television stations, but here we generally refer to online video live broadcasts. Live audio and video are pushed to the server in the form of media streams (pushing). If viewers want to watch the live broadcast, the server receives the user's request and transmits the video to the website, app, or client's player for real-time playback.

[0037] Video on demand (VOD): VOD, also known as video-on-demand, developed abroad in the 1990s. As the name suggests, it's a video-on-demand system that plays programs according to viewer requests, transmitting the video content clicked or selected by the user to the requesting user. Video on demand is a relatively new media format, a product of the integration of computer technology, network communication technology, multimedia technology, television technology, and digital compression technology.

[0038] HLS (HTTP Live Streaming): A dynamic bitrate adaptive technology. Primarily used for audio and video services on PCs and mobile terminals. Includes an m3u(8) index file (or mpd index file), TS / m4s media segment files, and a key encryption string file.

[0039] Live time-shifting: This refers to a technology that allows viewers to rewind to earlier parts of a live stream. It's a technology that combines live streaming and video-on-demand, and from the player's perspective, it's indistinguishable from the live streaming protocol. Viewers can rewind to a specific point in time and continue watching until the video ends.

[0040] High-Energy Progress Bar: A wave-shaped "heat" curve is added to the original video progress bar. This curve represents the density of comments or seek intensity in the current area. In other words, the higher the peak, the more heated the discussion. (See attached image: Example of a high-energy progress bar)

[0041] "Bullet comments" is a term used in online slang to refer to commentary text that pops up when watching videos online.

[0042] Player seek: This is a common operation in video players, allowing you to select a suitable point in the video's progress bar to jump to the next point in the playback.

[0043] It's important to note that due to the unique nature of live streaming, missing certain moments means missing them entirely. Therefore, there's a need for on-demand playback of the highlights. However, most scenes in a live stream may be useless. The common requirement is to select the most memorable moments for on-demand playback. Generally, AI content understanding can be used to perform video or audio analysis, identifying key moments in the live stream and then providing them via HLS (High-Speed ​​Live Streaming). However, AI-based content understanding methods require significant computing resources and need to be developed specifically for different domains, resulting in long development cycles and a relatively low return on investment. Furthermore, existing on-demand solutions cannot be directly transferred to live streaming due to limitations of the live streaming environment.

[0044] In practical applications, the logic of the on-demand video highlight progress bar is based on a weighted statistical analysis of data such as player seek operations, the number of bullet comments, and live stream time shifts during the viewing of on-demand videos. This can be used to determine which scenes in the on-demand video are more popular with users. Therefore, this application provides a method for extracting video content. Based on on-demand video data, it uses live stream bullet comment data and live stream gift data for weighted calculation, and then determines the highlight moments. This method utilizes the characteristics of live streaming, is extremely low-cost, and highly versatile; this logic can be applied to any type of live stream.

[0045] This application provides a video content extraction method, and also relates to a video content extraction apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0046] Figure 1The flowchart illustrates a video content extraction method according to an embodiment of this application, which specifically includes the following steps:

[0047] Step 102: Determine the target interaction data corresponding to the target moment in the video to be extracted. The target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data.

[0048] Specifically, the video to be extracted refers to the video from which the most exciting and impactful moments need to be extracted. It's important to note that due to the unique nature of live streams, missing certain scenes means missing them entirely. Therefore, there's a need to provide on-demand playback of the highlights. However, most scenes in a live stream may be useless; the focus should be on selecting the most engaging and impactful moments. Thus, the video to be extracted can refer to historically acquired live stream videos, and historical interaction data such as chat comments and gift-giving data can also be acquired. Subsequently, the chat comment and gift-giving data from various moments within the video to be extracted can be analyzed to identify and extract the most exciting and impactful segments.

[0049] In practical applications, any moment in the video to be extracted can be used as the target moment. By analyzing the target interaction data corresponding to the target moment, the user's level of interest in that target moment can be determined. Based on the user's level of interest in each moment in the video to be extracted, the exciting and high-energy moments in the video can be analyzed.

[0050] Among them, the bullet screen data can refer to the number of bullet screens sent at the target time, and the gift-giving data can refer to the value of the gifts given at the target time, as well as the number of gifts corresponding to each gift value.

[0051] Example, Figure 2a This is a bar chart of bullet screen data provided in one embodiment of this application, such as... Figure 2a As shown, a bar chart of bullet screen data for a video to be extracted is presented. The horizontal axis represents each moment of the video to be extracted, and the vertical axis represents the number of bullet screens corresponding to each moment of the video to be extracted.

[0052] Step 104: Fuzzyen the gift-giving data at the target time to obtain fuzzy gift data at the target time.

[0053] It's important to note that gift-giving data is generally sensitive and cannot be directly disclosed to users. Therefore, gift-giving data at a specific time can be obfuscated to obtain obfuscated gift data for that time. This obfuscated data is not the actual gift-giving data, but it represents the characteristics of real gift-giving data. This data can be used to analyze users' preferences for different moments in the extracted video, thereby identifying and extracting high-energy segments. In this way, obfuscating the gift-giving data at the target time blurs the actual value of the gifts while still representing statistical significance. This de-identification process for sensitive gift-giving data improves its security.

[0054] In one optional implementation of this embodiment, the gift-giving data includes the gift value and the corresponding quantity of gifts; the gift-giving data at the target time is fuzzified to obtain fuzzy gift data at the target time, including:

[0055] The value of the gift is divided according to at least one preset value range;

[0056] Based on the value of each gift within the first value interval and the corresponding quantity of gifts, determine the fuzzy value corresponding to the first value interval, wherein the first value interval is any one of at least one value interval;

[0057] Based on the fuzzy values ​​corresponding to each value range, the fuzzy gift data at the target time is determined.

[0058] Specifically, gift-giving data can include the value of each gift given by the user, as well as the number of gifts corresponding to each gift value.

[0059] In practical applications, at least one value range can be preset. Based on the preset value range, the value of each gift given by the user at the target time can be divided. Then, for each value range, the value of each gift and the corresponding number of gifts can be determined. Based on the fuzzy value of each value range, the fuzzy gift data at the target time can be determined.

[0060] For example, at least one preset value range is greater than or equal to 0 and less than 1, greater than or equal to 1 and less than 10, greater than or equal to 10 and less than 20, greater than or equal to 20 and less than 30, greater than or equal to 30 and less than 40, greater than or equal to 40 and less than 50, and greater than or equal to 50. Assume the gift values ​​at the target time are 0.3, 0.5, 2, 5, 7, 13, 16, 19, 25, 27, 36, 47, and 58, respectively, and the number of gifts corresponding to 0.3 is 10, 0.5 is 5, 2 is 7, 5 is 3, 7 is 2, 13 is 3, 16 is 2, 19 is 2, 25 is 1, 27 is 1, 36 is 1, 47 is 1, and 58 is 1. Gifts with values ​​of 0.3 and 0.5 fall within the value range of 0 to 1; gifts with values ​​of 2, 5, and 7 fall within the value range of 1 to 10; gifts with values ​​of 13, 16, and 19 fall within the value range of 10 to 20; gifts with values ​​of 25 and 27 fall within the value range of 20 to 30; gifts with values ​​of 36 fall within the value range of 30 to 40; gifts with values ​​of 47 fall within the value range of 40 to 50; and gifts with values ​​of 38 fall within the value range of 50. Based on the gift values ​​and corresponding quantities within each of these value ranges, the fuzzy gift data corresponding to each value range can be determined.

[0061] In this embodiment, statistical calculations can be performed on each value range to determine the fuzzy gift data for each value range. While fuzzing the actual value of the gift, it can also represent the statistical significance of the actual value. The more sensitive gift-giving data has been desensitized, thus improving the security of sensitive data.

[0062] In one optional implementation of this embodiment, the fuzzy value corresponding to the first value range is determined based on the value of each gift within the first value range and the corresponding quantity of gifts, including:

[0063] Determine the average value of each gift within the first value range, and determine the total number of gifts within the first value range;

[0064] Multiply the average value by the total number of gifts to obtain the fuzzy value corresponding to the first value range.

[0065] It should be noted that the fuzzy value corresponding to the first value interval can be determined based on the average value of each gift within the first value interval and the total number of gifts within the first value interval. That is, by determining the average value of each gift within the first value interval and the total number of gifts within the first value interval, the average value is multiplied by the total number of gifts to obtain the fuzzy value corresponding to the first value interval. In this way, statistical calculations are performed on each value interval to determine the fuzzy gift data for each value interval. While fuzzing the actual value of the gifts, it still represents the statistical significance of the actual value. Sensitive gift-giving data has been anonymized, improving the security of sensitive data.

[0066] Continuing with the previous example, let's define the value range of 0 to 1 as the first value range. The average value of each gift within this range is 0.4, and the total number of gifts is 15. The fuzzy value corresponding to this range is 6. Let's define the value range of 1 to 10 as the first value range. The average value of each gift within this range is 4.67, and the total number of gifts is [missing value]. The fuzzy value corresponding to this range is 56.04. Let's define the value range of 10 to 20 as the first value range. The average value of each gift within this range is 16, and the total number of gifts is 7. The fuzzy value corresponding to this range is 112. Let's define the value range of 20 to 30 as the first value range. The average value of each gift within this range is 0.4, and the total number of gifts is [missing value]. The fuzzy value corresponding to this range is 112. The average value of each gift is 26, and the total number of gifts is 2. Therefore, the fuzzy value corresponding to the value range of 20 or more and less than 30 is 52. Taking the value range of 30 or more and less than 40 as the first value range, the average value of each gift within this first value range is 36, and the total number of gifts is 1. Therefore, the fuzzy value corresponding to the value range of 30 or more and less than 40 is 36. Taking the value range of 40 or more and less than 50 as the first value range, the average value of each gift within this first value range is 47, and the total number of gifts is 1. Therefore, the fuzzy value corresponding to the value range of 40 or more and less than 50 is 47. Taking the value range of 50 or more as the first value range, the average value of each gift within this first value range is 58, and the total number of gifts is 1. Therefore, the fuzzy value corresponding to the value range of 50 or more is 58.

[0067] From the above, we can obtain the fuzzy value table corresponding to each value interval as shown in Table 1 below:

[0068] Table 1. Fuzzy value tables corresponding to each value range.

[0069]

[0070]

[0071] In one optional implementation of this embodiment, the fuzzy gift data at the target time is determined based on the fuzzy value corresponding to each value interval, including:

[0072] Obtain the value weight value corresponding to each value range;

[0073] Based on the fuzzy value and value weight value corresponding to each value interval, the fuzzy gift data at the target time is determined.

[0074] It should be noted that after determining the fuzzy values ​​corresponding to each value interval, the fuzzy values ​​corresponding to each value interval can be directly summed to obtain the fuzzy gift data at the target time. In practical applications, since there are many differences in gift giving across different value intervals, generally, lower-value gifts are given to more people, while higher-value gifts are given to fewer people. Therefore, it is also possible to pre-set value weights for different value intervals, and then perform a weighted summation of the fuzzy values ​​corresponding to each value interval to obtain the fuzzy gift data at the target time.

[0075] Example, Figure 2b This is a bar chart of fuzzy gift data provided in one embodiment of this application, such as... Figure 2b As shown, a bar chart is presented to display fuzzy gift data at various moments in a video to be extracted. The horizontal axis represents each moment in the video to be extracted, and the vertical axis represents the fuzzy gift data corresponding to each moment in the video to be extracted.

[0076] Step 106: Determine the high-energy value at the target time based on the fuzzy gift data and bullet screen data at the target time.

[0077] It should be noted that after determining the fuzzy gift data for the target moment, the fuzzy gift data and the bullet screen data can be combined to jointly determine the high-energy value for the target moment. This takes advantage of the characteristics of live streaming, has extremely low cost, and is highly versatile. This logic can be applied to any type of live stream video to be extracted.

[0078] In one optional implementation of this embodiment, the high-energy value at the target time is determined based on the fuzzy gift data and bullet screen data at the target time, including:

[0079] Determine the first weight corresponding to the fuzzy gift data and the second weight corresponding to the bullet screen data respectively;

[0080] Based on the fuzzy gift data and first weight at the target time, as well as the bullet screen data and second weight at the target time, the high-energy value at the target time is determined.

[0081] It should be noted that corresponding weights can be pre-set for the gift data and the bullet screen data, and then the fuzzy gift data and bullet screen data can be weighted and summed to obtain the high-energy value at the target moment. This process can be repeated to obtain the high-energy values ​​for each moment in the video to be extracted.

[0082] Example, Figure 2c This is a bar chart of high-energy values ​​provided in one embodiment of this application, such as... Figure 2c As shown, a bar chart is presented to display the high-energy values ​​at various times in the video to be extracted. The horizontal axis represents the various times in the video to be extracted, and the vertical axis represents the high-energy values ​​corresponding to each time in the video to be extracted.

[0083] In this embodiment, the feature of users sending bullet comments and gifts during video viewing is utilized. The cost is extremely low, only requiring the collection of historical data of the video to be extracted. This can save a lot of processing resources and has a high return on investment. It can quickly generate high-energy segments of the video to be extracted, and the extraction efficiency of high-energy segments is high, which is convenient for users to carry out secondary creations. In addition, the more sensitive gift-giving data has been de-identified, which improves the security of sensitive data.

[0084] In an optional implementation of this embodiment, the target interaction data further includes progress jump data; determining the high-energy value at the target time based on the fuzzy gift data and bullet screen data at the target time includes:

[0085] Based on the fuzzy gift data, bullet screen data, and progress jump data at the target time, the high-energy value at the target time is determined.

[0086] It should be noted that users can jump to the moment they are interested in by clicking the progress bar. Therefore, the progress jump data can indicate whether the user is interested in the current moment. Thus, in addition to combining the bullet screen data and gift-giving data to determine the high-energy value of the target moment, the progress jump data corresponding to the target moment can also be combined. The progress jump data refers to the operation data related to the user clicking the progress bar to jump to the moment they are interested in. For example, the progress jump data can include the first number of times to jump from other moments to play the target moment, and the second number of times to jump from the target moment to play other moments.

[0087] In one optional implementation of this embodiment, the progress jump data includes a first number of times the playback jumps from other times to the target time, and a second number of times the playback jumps from the target time to other times; based on the fuzzy gift data, bullet screen data, and progress jump data of the target time, the high-energy value of the target time is determined, including:

[0088] Based on the fuzzy gift data and bullet screen data at the target time, determine the initial high-energy value at the target time;

[0089] The first value is added to the initial high-energy value, and the second value is subtracted to determine the high-energy value at the target time.

[0090] Specifically, the progress jump data can include the first number of times the user jumps from other times to play the target time, and the second number of times the user jumps from the target time to play other times. It should be noted that jumping from other times to play the target time indicates that the user is interested in the target time, which suggests that the video content at the target time is relatively exciting; while jumping from the target time to play other times suggests that the user is not interested in the target time, which suggests that the video content at the target time may not be relatively exciting.

[0091] Therefore, in practical applications, after determining the fuzzy gift data at the target time, the initial high-energy value at the target time can be determined based on the fuzzy gift data and the barrage data at the target time. Then, the first number can be added to the initial high-energy value, and the second number can be subtracted to determine the high-energy value at the target time.

[0092] In this embodiment, the high-energy value of the target time can be determined by combining the first number of times the target time is jumped from other times to play the target time, the second number of times the target time is jumped to play other times, and the bullet screen data and fuzzy gift data of the target time, thereby improving the accuracy of the high-energy value.

[0093] Step 108: Extract high-energy segments from the video based on the high-energy values ​​at various times in the video to be extracted.

[0094] It should be noted that after determining the high-energy values ​​at various moments in the video to be extracted, high-energy segments can be extracted from the video based on these values. These high-energy segments can be viewed or edited by users; that is, the extracted high-energy segments can be provided for users to create secondary content or publish. For example, if the video to be extracted is a live stream, high-energy segments can be extracted from the live stream based on the high-energy values ​​at various moments. These extracted high-energy segments can then be edited and combined to output on-demand content.

[0095] In one optional implementation of this embodiment, high-energy segments are extracted from the video based on the high-energy values ​​at various times in the video to be extracted, including:

[0096] Identify candidate high-energy values ​​that exceed the median at various time points;

[0097] Determine the average value of each candidate high-energy value, and identify the target high-energy value that exceeds the average value among the candidate high-energy values. Take the time corresponding to the target high-energy value as the high-energy time.

[0098] Based on high-energy moments, extract high-energy segments from the video to be extracted.

[0099] In practical applications, we can identify candidate high-energy values ​​that exceed the median at various times, discarding those below the median. Then, we can determine the average of these candidate high-energy values ​​and filter out target high-energy values ​​that exceed the average. The time corresponding to this target high-energy value is designated as the high-energy time. Based on these selected high-energy times, high-energy segments are extracted from the video. This leverages the user's habit of sending comments and gifts while watching videos, achieving extremely low cost—only requiring historical data analysis of the video to be extracted. This saves significant processing resources, resulting in a high return on investment. It can quickly generate high-energy segments from the video, with high extraction efficiency, facilitating subsequent user-generated content. Furthermore, sensitive gift-giving data has been anonymized, enhancing the security of sensitive data.

[0100] In one optional implementation of this embodiment, extracting high-energy segments from the video to be extracted based on high-energy moments includes:

[0101] Set the length limit for obtaining high-energy fragments;

[0102] The smoothing length of the high-energy moments is determined based on the set length limit and the number of high-energy moments;

[0103] Before and after the high-energy moment, extract video content of smooth length and merge it into a high-energy segment.

[0104] It should be noted that if only the selected high-energy moments are extracted, it may result in fragmented content. Therefore, after selecting the high-energy moments from the video to be extracted, for each high-energy moment, video content of smooth length can be extracted before and after the high-energy moment as transition content to synthesize the final high-energy segment.

[0105] In practical applications, the smoothing length can be determined based on a set length limit, specifically the length of video content to be extracted before and after the high-energy moment as transition content, based on the length limit of the high-energy segment. Specifically, the pre-set length limit of the high-energy segment can be obtained first. Then, based on the set length limit and the number of high-energy moments, the smoothing length of each high-energy moment can be determined. Finally, video content of the smoothing length is extracted before and after each high-energy moment and merged into a single high-energy segment. In this way, by extracting video content of the smoothing length before and after the high-energy moment, discontinuities in the extracted high-energy segment can be avoided, ensuring the smoothness of the extracted high-energy segment.

[0106] For example, the set length of the high-energy segment is limited to 12 minutes. Assuming that 3 high-energy moments are selected, the smooth length can be determined to be 2 minutes. That is, 2 minutes of video content is extracted before and after each high-energy moment. The video content extracted from each high-energy moment can be merged into a high-energy segment.

[0107] In one optional implementation of this embodiment, high-energy segments are extracted from the video based on the high-energy values ​​at various times in the video to be extracted, including:

[0108] The high-energy values ​​at each moment in the video to be extracted are sorted.

[0109] Obtain the set length limit of the high-energy fragment, and determine the number of high-energy moments to be extracted based on the set length limit;

[0110] Based on the sorting results, the number of high-energy moments to be extracted from each moment is determined, and the high-energy video frames corresponding to the number of high-energy moments are merged into a high-energy segment.

[0111] It should be noted that the set length limit for high-energy clips indicates the final length of the high-energy clip to be synthesized. Based on this set length limit, the number of high-energy moments required to synthesize the high-energy clip can be determined. In practical applications, one high-energy moment corresponds to one high-energy video frame, i.e., one still image. Therefore, when determining the number of high-energy moments to be extracted based on the set length limit, it can also be determined in conjunction with the unit duration of the video frames corresponding to the required high-energy clip. Specifically, the set length limit of the high-energy video can be converted into a unit of measurement with the same unit duration, and then divided by the unit duration to determine how many high-energy video frames are needed for the high-energy clip, i.e., the number of high-energy moments to be extracted. Then, according to the sorting results of each moment, the number of moments with higher high-energy values ​​are selected as high-energy moments. The high-energy video frames corresponding to each high-energy moment last for the unit duration, thus obtaining the corresponding high-energy clip.

[0112] For example, the video to be extracted contains 1,000,000 moments. Based on the high-energy values ​​of each moment, the 1,000,000 moments are sorted from high to low. Assuming the set length limit for the high-energy segment is 12 minutes and the unit duration of the video frame is 1 second, the set length limit of 12 minutes is 720 seconds. Dividing 720 seconds by 1 second, we can determine that the number of high-energy moments to be extracted is 720. At this time, the top 720 moments in the sorted list can be selected as high-energy moments. The high-energy video frames corresponding to these 720 high-energy moments each last for 1 second, and they are merged to obtain the high-energy segment.

[0113] In addition, when determining the number of high-energy moments to be extracted based on the set length limit, besides determining it by combining the unit duration of the video frames corresponding to the required high-energy segment, it is also possible to combine it with the number of frames transmitted per second of the required high-energy segment. That is, convert the set length limit of the high-energy video into seconds, and then multiply it by the number of frames transmitted per second to determine how many high-energy video frames are needed for the high-energy segment, that is, the number of high-energy moments to be extracted. Then, according to the sorting results of each moment, select the number of moments with higher high-energy values ​​as high-energy moments, and merge the corresponding high-energy video frames to obtain the corresponding high-energy segment.

[0114] For example, the video to be extracted contains 1,000,000 moments. Based on the high-energy values ​​of each moment, the 1,000,000 moments are sorted from high to low. Assuming the set length limit for the high-energy segment is 12 minutes, the required transmission frame rate for the high-energy segment is 30 fps, and the set length limit of 12 minutes is 720 seconds, 720 seconds multiplied by 30, we can determine that the number of segments to be extracted is 21,600. At this time, the top 21,600 moments can be selected from the 1,000,000 moments as high-energy moments, and the high-energy video frames corresponding to these 21,600 high-energy moments are merged to obtain the high-energy segment.

[0115] It's important to note that the high-energy moments in the video to be extracted can be sorted based on their high-energy values. Then, a set length limit for the high-energy segments can be obtained, and the number of high-energy moments to be extracted can be determined based on this limit. According to the sorting results, the number of high-energy moments to be extracted is determined from each moment, and the corresponding high-energy video frames are merged into a single high-energy segment. In this way, high-energy segments can be extracted from multiple high-energy moments in the video. The extraction of high-energy segments is highly flexible, and the extracted high-energy segments only include the video frames corresponding to the high-energy moments. This means that only content of interest to the user is extracted as high-energy segments, excluding other redundant content, thus improving the relevance between the extracted high-energy segments and the user's interests, and ultimately enhancing the user experience.

[0116] The video content extraction method provided in this application can obtain interactive data such as bullet screen data and gift-giving data from the video to be extracted. Since gift-giving data may be sensitive and cannot be directly used or displayed, it can be blurred to obtain blurred gift data. This blurred gift data is then combined with the bullet screen data and the blurred gift data to determine high-energy values. Based on these high-energy values ​​at various moments in the video, the most exciting high-energy segments are automatically extracted. This method leverages the characteristics of users sending bullet screens and gifts while watching videos, resulting in extremely low costs. It only requires statistical analysis of the historical data of the video to be extracted, saving significant processing resources and achieving a high return on investment. It can quickly generate high-energy segments from the video to be extracted, with high extraction efficiency, facilitating subsequent secondary creation by users. Furthermore, the sensitive gift-giving data has been anonymized, improving the security of sensitive data.

[0117] The following is in conjunction with the appendix Figure 3 Taking the application of the video content extraction method provided in this application in a live video scenario as an example, the video content extraction method will be further explained. Figure 3 This application provides a flowchart illustrating a video content extraction method for live video scenarios according to an embodiment of the present application, which specifically includes the following steps:

[0118] Step 302: Obtain the live video of the historical live stream, as well as the corresponding bullet screen data, gift-giving data and progress jump data. The gift-giving data includes the value of the gift and the corresponding number of gifts. The progress jump data includes the first number of times to jump from other times to play the target time, and the second number of times to jump from the target time to play other times.

[0119] Step 304: Determine the target time from the live video, and then determine the bullet screen data and gift-giving data corresponding to the target time in the live video. The target time is any moment in the live video.

[0120] Step 306: Divide the value of gifts according to at least one preset value range, determine the average value of each gift in the first value range, and determine the total number of gifts in the first value range. Multiply the average value by the total number of gifts to obtain the fuzzy value corresponding to the first value range, wherein the first value range is any one of at least one value range.

[0121] Step 308: Determine the fuzzy gift data at the target time based on the fuzzy value corresponding to each value interval.

[0122] Step 310: Determine the first weight corresponding to the fuzzy gift data and the second weight corresponding to the bullet screen data, and determine the initial high-energy value at the target time based on the fuzzy gift data and the first weight at the target time, as well as the bullet screen data and the second weight at the target time.

[0123] Step 312: Add the first number to the initial high-energy value and subtract the second number to determine the high-energy value at the target time.

[0124] Step 314: After determining the high-energy values ​​at each time, select the candidate high-energy values ​​that exceed the median at each time. Then, determine the average value of each candidate high-energy value and determine the target high-energy value that exceeds the average value among each candidate high-energy value. Take the time corresponding to the target high-energy value as the high-energy time.

[0125] Step 316: Obtain the set length limit of the high-energy segment, and determine the smooth length of the high-energy moment based on the set length limit and the number of high-energy moments. Before and after the high-energy moment, extract the video content of the smooth length and merge them as a high-energy segment.

[0126] The video content extraction method provided in this application can obtain interactive data such as bullet screen data and gift-giving data from live videos. Since gift-giving data may be sensitive and cannot be directly used or displayed, it can be obfuscated to obtain obfuscated gift data. This obfuscated gift data is then combined with bullet screen data and obfuscated gift data to determine high-energy values. Based on these high-energy values ​​at various moments in the live video, the most exciting high-energy segments are automatically extracted. This method leverages the characteristics of users sending bullet screens and gifts while watching live videos, with extremely low cost—only requiring the collection of historical data from the live video—saving significant processing resources and achieving a high return on investment. It can quickly generate high-energy segments from live videos, with high extraction efficiency, facilitating subsequent secondary creation by users. Furthermore, the sensitive gift-giving data has been anonymized, improving the security of sensitive data.

[0127] Corresponding to the above method embodiments, this application also provides embodiments of a video content extraction device. Figure 4 A schematic diagram of a video content extraction device according to an embodiment of this application is shown. Figure 4 As shown, the device includes:

[0128] The first determining module 402 is configured to determine the target interaction data corresponding to the target time in the video to be extracted, wherein the target time is any time in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data.

[0129] Processing module 404 is configured to perform fuzzing processing on the gift-giving data at the target time to obtain fuzzy gift data at the target time;

[0130] The second determining module 406 is configured to determine the high-energy value at the target time based on the fuzzy gift data and bullet screen data at the target time.

[0131] Extraction module 408 is configured to extract high-energy segments from the video to be extracted based on the high-energy values ​​at various times in the video.

[0132] Optionally, the gift-giving data includes the gift value and the corresponding number of gifts; processing module 404 is further configured to:

[0133] The value of the gift is divided according to at least one preset value range;

[0134] Based on the value of each gift within the first value interval and the corresponding quantity of gifts, determine the fuzzy value corresponding to the first value interval, wherein the first value interval is any one of at least one value interval;

[0135] Based on the fuzzy values ​​corresponding to each value range, the fuzzy gift data at the target time is determined.

[0136] Optionally, processing module 404 is further configured to:

[0137] Determine the average value of each gift within the first value range, and determine the total number of gifts within the first value range;

[0138] Multiply the average value by the total number of gifts to obtain the fuzzy value corresponding to the first value range.

[0139] Optionally, the second determining module 406 is further configured to:

[0140] Determine the first weight corresponding to the fuzzy gift data and the second weight corresponding to the bullet screen data respectively;

[0141] Based on the fuzzy gift data and first weight at the target time, as well as the bullet screen data and second weight at the target time, the high-energy value at the target time is determined.

[0142] Optionally, the extraction module 408 is further configured as follows:

[0143] Identify candidate high-energy values ​​that exceed the median at various time points;

[0144] Determine the average value of each candidate high-energy value, and identify the target high-energy value that exceeds the average value among the candidate high-energy values. Take the time corresponding to the target high-energy value as the high-energy time.

[0145] Based on high-energy moments, extract high-energy segments from the video to be extracted.

[0146] Optionally, the extraction module 408 is further configured as follows:

[0147] Set the length limit for obtaining high-energy fragments;

[0148] The smoothing length of the high-energy moments is determined based on the set length limit and the number of high-energy moments;

[0149] Before and after the high-energy moment, extract video content of smooth length and merge it into a high-energy segment.

[0150] Optionally, the extraction module 408 is further configured as follows:

[0151] The high-energy values ​​at each moment in the video to be extracted are sorted.

[0152] Obtain the set length limit of the high-energy fragment, and determine the number of high-energy moments to be extracted based on the set length limit;

[0153] Based on the sorting results at each time point, a number of high-energy moments are selected from each time point, and the high-energy video frames corresponding to the selected number of high-energy moments are merged into a high-energy segment.

[0154] Optionally, the target interaction data also includes progress jump data; the second determining module 406 is further configured to:

[0155] Based on the fuzzy gift data, bullet screen data, and progress jump data at the target time, the high-energy value at the target time is determined.

[0156] Optionally, the progress jump data includes a first number of times the playback jumps from other times to the target time, and a second number of times the playback jumps from the target time to other times; the second determining module 406 is further configured to:

[0157] Based on the fuzzy gift data and bullet screen data at the target time, determine the initial high-energy value at the target time;

[0158] The first value is added to the initial high-energy value, and the second value is subtracted to determine the high-energy value at the target time.

[0159] The video content extraction device provided in this application can acquire interactive data such as bullet screen data and gift-giving data from the video to be extracted. Since gift-giving data may be sensitive and cannot be directly used or displayed, it can be blurred to obtain blurred gift data. This blurred gift data is then combined with the bullet screen data and the blurred gift data to determine high-energy values. Based on these high-energy values ​​at various moments in the video, the device automatically extracts the most exciting high-energy segments. This utilizes the characteristics of users sending bullet screens and gifts while watching videos, resulting in extremely low costs. Only historical data of the video to be extracted needs to be collected, saving significant processing resources and achieving a high return on investment. It can quickly generate high-energy segments from the video, with high extraction efficiency, facilitating subsequent secondary creation by users. Furthermore, the sensitive gift-giving data has been anonymized, improving the security of sensitive data.

[0160] The above is an illustrative scheme of a video content extraction device according to this embodiment. It should be noted that the technical solution of this video content extraction device and the technical solution of the video content extraction method described above belong to the same concept. For details not described in detail in the technical solution of the video content extraction device, please refer to the description of the technical solution of the video content extraction method described above.

[0161] Figure 5 A structural block diagram of a computing device according to an embodiment of this application is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0162] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Controller (NIC)), such as an IEEE 802.11 Wireless Local Area Networks (WLAN) wireless interface, Wi-MAX (Worldwide Interoperability for Microwave Access) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, and so on.

[0163] In one embodiment of this application, the aforementioned components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0164] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 500 can also be a mobile or stationary server.

[0165] The processor 520 is configured to execute the following computer-executable instructions to implement the following method:

[0166] Determine the target interaction data corresponding to the target moment in the video to be extracted. The target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data.

[0167] The gift-giving data at the target time is fuzzed to obtain fuzzy gift data at the target time.

[0168] Based on the fuzzy gift data and bullet screen data at the target time, determine the high-energy value at the target time;

[0169] High-energy segments are extracted from the video based on the high-energy values ​​at various times in the video.

[0170] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the video content extraction method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the video content extraction method described above.

[0171] An embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement steps of an arbitrary video content extraction method.

[0172] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the video content extraction method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the video content extraction method described above.

[0173] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0174] Computer instructions include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0175] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0176] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0177] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A method for extracting video content, characterized in that, include: The target interaction data corresponding to the target moment in the video to be extracted is determined by statistically analyzing the historical interaction data of the video to be extracted. The target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data. The gift-giving data at the target time is fuzzed to obtain fuzzy gift data at the target time; Based on the fuzzy gift data and bullet screen data at the target time, the high-energy value at the target time is determined; Based on the high-energy values ​​at various times in the video to be extracted, high-energy segments are extracted from the video to be extracted. The gift-giving data includes the value of the gifts and the corresponding quantity of gifts; the process of fuzzifying the gift-giving data at the target time to obtain fuzzy gift data at the target time includes: The value of the gift is divided according to at least one preset value range; Based on the value of each gift within the first value interval and the corresponding number of gifts, a fuzzy value corresponding to the first value interval is determined, wherein the first value interval is any one of the at least one value interval. Obtain the value weight value corresponding to each value range; Based on the fuzzy value and value weight value corresponding to each value interval, the fuzzy gift data at the target time is determined; The target interaction data also includes progress jump data; determining the high-energy value at the target time based on the fuzzy gift data and bullet screen data at the target time includes: Based on the fuzzy gift data, bullet screen data, and progress jump data at the target time, the high-energy value at the target time is determined; The step of determining the fuzzy value corresponding to the first value interval based on the value of each gift within the first value interval and the corresponding quantity of gifts includes: Determine the average value of each gift within the first value range, and determine the total number of gifts within the first value range; Multiply the average value by the total amount of gifts to obtain the fuzzy value corresponding to the first value range.

2. The video content extraction method according to claim 1, characterized in that, The step of determining the high-energy value at the target time based on the fuzzy gift data and bullet screen data at the target time includes: Determine the first weight corresponding to the fuzzy gift data and the second weight corresponding to the bullet screen data respectively; Based on the fuzzy gift data and the first weight at the target time, and the bullet screen data and the second weight at the target time, the high-energy value at the target time is determined.

3. The video content extraction method according to claim 1, characterized in that, The step of extracting high-energy segments from the video to be extracted based on the high-energy values ​​at various times in the video to be extracted includes: The candidate high-energy values ​​that exceed the median at each of the aforementioned time points are identified; Determine the average value of each candidate high-energy value, and determine the target high-energy value among the candidate high-energy values ​​that exceeds the average value. Take the time corresponding to the target high-energy value as the high-energy time. Based on the high-energy moments, the high-energy segments are extracted from the video to be extracted.

4. The video content extraction method according to claim 3, characterized in that, The step of extracting the high-energy segment from the video to be extracted based on the high-energy moment includes: Obtain the set length limit of the high-energy fragment; The smooth length of the high-energy moments is determined based on the set length limit and the number of high-energy moments; Before and after the high-energy moment, video content of the smooth length is extracted and merged into the high-energy segment.

5. The video content extraction method according to claim 1, characterized in that, The step of extracting high-energy segments from the video to be extracted based on the high-energy values ​​at various times in the video to be extracted includes: The high-energy values ​​at each moment in the video to be extracted are sorted. Obtain the set length limit of the high-energy fragment, and determine the number of high-energy moments to be extracted based on the set length limit; Based on the sorting results of each time moment, the number of high-energy moments to be extracted are determined from each time moment, and the high-energy video frames corresponding to the number of high-energy moments to be extracted are merged into the high-energy segment.

6. The video content extraction method according to claim 1, characterized in that, The progress jump data includes a first number of jumps from other times to play the target time, and a second number of jumps from the target time to play other times; determining the high-energy value of the target time based on the fuzzy gift data, bullet screen data, and progress jump data of the target time includes: Based on the fuzzy gift data and bullet screen data at the target time, determine the initial high-energy value at the target time; The first number of times is added to the initial high-energy value, and the second number of times is subtracted to determine the high-energy value at the target time.

7. A video content extraction device, characterized in that, include: The first determining module is configured to determine the target interaction data corresponding to the target time in the video to be extracted by statistically analyzing the historical interaction data of the video to be extracted, wherein the target time is any time in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data. The processing module is configured to perform fuzzing processing on the gift-giving data at the target time to obtain fuzzy gift data at the target time; The second determining module is configured to determine the high-energy value at the target time based on the fuzzy gift data and bullet screen data at the target time. The extraction module is configured to extract high-energy segments from the video to be extracted based on the high-energy values ​​at various times in the video to be extracted. The gift-giving data includes the gift value and the corresponding gift quantity. The processing module is further configured to divide the gift value according to at least one preset value range; determine the fuzzy value corresponding to the first value range according to the gift value and the corresponding gift quantity within the first value range, wherein the first value range is any one of the at least one value range; obtain the value weight value corresponding to each value range; and determine the fuzzy gift data at the target time based on the fuzzy value and value weight value corresponding to each value range. The target interaction data also includes progress jump data; the second determining module is further configured to determine the high energy value at the target time based on the fuzzy gift data, bullet screen data and progress jump data at the target time. The step of determining the fuzzy value corresponding to the first value interval based on the value of each gift within the first value interval and the corresponding quantity of gifts includes: Determine the average value of each gift within the first value range, and determine the total number of gifts within the first value range; Multiply the average value by the total amount of gifts to obtain the fuzzy value corresponding to the first value range.

8. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the following method: The target interaction data corresponding to the target moment in the video to be extracted is determined by statistically analyzing the historical interaction data of the video to be extracted. The target moment is any moment in the video to be extracted, and the target interaction data includes bullet screen data and gift-giving data. The gift-giving data at the target time is fuzzed to obtain fuzzy gift data at the target time; Based on the fuzzy gift data and bullet screen data at the target time, the high-energy value at the target time is determined; Based on the high-energy values ​​at various times in the video to be extracted, high-energy segments are extracted from the video to be extracted. The gift-giving data includes the value of the gifts and the corresponding quantity of gifts; the process of fuzzifying the gift-giving data at the target time to obtain fuzzy gift data at the target time includes: The value of the gift is divided according to at least one preset value range; Based on the value of each gift within the first value interval and the corresponding number of gifts, a fuzzy value corresponding to the first value interval is determined, wherein the first value interval is any one of the at least one value interval. Obtain the value weight value corresponding to each value range; Based on the fuzzy value and value weight value corresponding to each value interval, the fuzzy gift data at the target time is determined; The target interaction data also includes progress jump data; the second determining module is further configured to determine the high energy value at the target time based on the fuzzy gift data, bullet screen data and progress jump data at the target time. The step of determining the fuzzy value corresponding to the first value interval based on the value of each gift within the first value interval and the corresponding quantity of gifts includes: Determine the average value of each gift within the first value range, and determine the total number of gifts within the first value range; Multiply the average value by the total amount of gifts to obtain the fuzzy value corresponding to the first value range.

9. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the video content extraction method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the video content extraction method according to any one of claims 1 to 6.

Images