A video clip method, device, equipment and storage medium

Abstract

The application discloses a video clip method, device, equipment and storage medium, the method comprises the following steps: acquiring original video data for promoting a business object; taking a video scene as a cutting node, and cutting the original video data into a plurality of candidate clips; calculating a segment highlight value representing the highlight degree in vision for each candidate clip; selecting part of the candidate clips as target clips according to the segment highlight value; and splicing the target clips into target video data in sequence. In the case of getting rid of the subjective judgment of an art personnel, the candidate clips are cut according to the video scenes, the completeness of the plot is ensured, the segment highlight value is taken as a reference to screen the candidate clips, the quality of the screened candidate clips is ensured, the quality of the target video data is ensured, and the overall clip efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of multimedia technology, and more particularly to a video editing method, apparatus, device, and storage medium. Background Technology

[0002] In scenarios where games, electronic products, and other business entities are promoted, video data is often used to introduce these entities. Video data presents information about these business entities through visuals and sound, making it convenient for users to read.

[0003] After recording the raw video data, the art staff mainly use professional video editing tools to edit the video data. That is, they constantly drag the playback progress of the video data, quickly browse the video data and make edits. The editing location mainly depends on the art staff browsing the content of the screen, repeatedly switching and comparing different positions on the timeline to determine the editing position, which results in low editing efficiency. Summary of the Invention

[0004] This invention provides a video editing method, apparatus, device, and storage medium to address the problem of how to improve the efficiency of video data editing.

[0005] According to one aspect of the present invention, a video editing method is provided, comprising:

[0006] Obtain the raw video data used to promote business targets;

[0007] Using video scenes as segmentation nodes, the original video data is segmented into multiple candidate segments;

[0008] For each candidate segment, a segment brilliance value is calculated that visually represents the degree of brilliance.

[0009] Based on the segment's brilliance value, select a portion of the candidate segments as target segments;

[0010] The target segments are spliced ​​together in sequence to form the target video data.

[0011] According to another aspect of the present invention, a video editing apparatus is provided, comprising:

[0012] The raw video data acquisition module is used to acquire raw video data for promotional business targets;

[0013] The candidate segment segmentation module is used to segment the original video data into multiple candidate segments, using video scenes as segmentation nodes.

[0014] The highlight value calculation module is used to calculate a segment highlight value, which visually represents the degree of highlight, for each of the candidate segments;

[0015] The target segment selection module is used to select a portion of the candidate segments as target segments based on the segment excellence value.

[0016] The target video data splicing module is used to splice the target segments into target video data in sequence.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video editing method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program configured to cause a processor to execute and implement the video editing method according to any embodiment of the present invention.

[0022] In this embodiment, the original video data for promoting the business object is acquired; the original video data is divided into multiple candidate segments using video scenes as segmentation nodes; a segment brilliance value, which visually represents the level of brilliance, is calculated for each candidate segment; some candidate segments are selected as target segments based on the segment brilliance value; and the target segments are spliced ​​together in sequence to form the target video data. By eliminating the subjective judgment of artists, segmenting candidate segments according to video scenes ensures the integrity of the plot, and using segment brilliance values ​​as a reference for selecting candidate segments ensures the quality of the selected candidate segments, thereby guaranteeing the quality of the target video data. Furthermore, the original video data is reusable, allowing for the creation of numerous target video data sets with diverse content, reducing the workload of editing target video data, shortening the production cycle, and significantly reducing manpower and time costs, thus improving overall editing efficiency. The shorter production cycle meets the time-sensitive requirements of business objects with short promotion cycles, such as games.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a video editing method provided in Embodiment 1 of the present invention;

[0026] Figure 2 This is an example diagram of video data editing according to Embodiment 1 of the present invention;

[0027] Figure 3 This is a schematic diagram of the structure of a video editing device according to Embodiment 2 of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Example 1

[0032] Figure 1This is a flowchart of a video editing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where video data is edited according to the level of interest in a video scene. The method can be executed by a video editing device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0033] Step 101: Obtain the raw video data for the target audience of the promotion business.

[0034] In different business scenarios, there are objects that possess the business characteristics of that business scenario, referred to as business objects. Business objects can be physical items, such as mobile phones, tablets, smartwatches, etc., or they can be virtual data, mostly third-party applications, such as games, game distribution applications, short video applications, shopping applications, etc. This embodiment does not impose any restrictions on them.

[0035] To enable those skilled in the art to better understand the present invention, in this embodiment, a game is used as an example of a business object for illustration.

[0036] The types of games can include MOBA (Multiplayer Online Battle Arena), RPG (Role-playing game), SLG (Simulation Game), etc., and this embodiment does not limit them.

[0037] For a given target audience, promotion may be carried out on different channels. Different channels have differences in terms of duration, content, etc. In order to facilitate subsequent promotion, the art staff can pre-create one or more video data that can cover different channels, referred to as raw video data. This raw video data can be edited according to the channel specifications. For example, the raw video data may be longer than the duration limit of all channels, allowing the art staff to edit it for specific channels. The raw video data may not have background music, allowing the art staff to add background music for specific channels, and so on.

[0038] Furthermore, the content of the raw video data (including images and sound) is related to the business object and can be used to introduce and promote the business object.

[0039] Taking games as an example, the content of raw video data can be divided into two main forms: game content and real-life storylines. Game content can include introductions to the user's control of the game, introductions by a spokesperson, or introductions by a spokesperson wearing in-game costumes. Storylines can be further divided into the following categories:

[0040] 1. Pseudo-food sharing

[0041] The raw video data included some food-related materials to attract users' attention and incorporate gameplay elements of eating food while playing games.

[0042] 2. Themes closely related to users' daily lives

[0043] The content of the raw video data closely reflects users' current lifestyles, integrating games into various aspects of life, such as playing games while eating or buying snacks. The first half of this type of material primarily features dialogue between two people, while the second half includes segments showcasing the game integration.

[0044] 3. Exaggerated situational drama

[0045] The raw video data contains material from sitcoms, some of which features exaggerated storylines designed to attract users' attention.

[0046] Of course, the above-described raw video data is merely an example. When implementing this embodiment, other raw video data can be set according to actual circumstances, and this embodiment does not impose any limitations on this. Furthermore, in addition to the above-described raw video data, those skilled in the art can use other raw video data as needed, and this embodiment does not impose any limitations on this either.

[0047] In practical applications, the original video data contains multiple frames of image data. In order to promote business targets, information such as icons (logos), banners, and ending cards (ECs) are usually configured in different image data.

[0048] The icon (Logo) is the identifier of the business object itself, and it can be a text icon (containing the name of the business object) or a graphic icon.

[0049] Banner information is generally rectangular and is usually located at the top and / or bottom of image data. It can record information about the business object itself (such as a scene in a game, a character in the game, or a name) or information to attract users to purchase or download the business object (such as a gift code).

[0050] The end segment EC contains the identifier of the download business object, such as the business object's own information (e.g., in-game graphics, characters, and names), and the method of purchasing or downloading the business object (e.g., the icon of the application distribution platform, the name and icon of the application distribution platform, the name and icon of the shopping platform, etc.).

[0051] Step 102: Using video scenes as the segmentation nodes, the original video data is segmented into multiple candidate segments.

[0052] In this embodiment, command-line tools, library files, and other methods can be used to detect video scenes in the raw video data. A video scene can refer to a scene formed in the raw video data, which can reflect a relatively independent scenario in the raw video data.

[0053] Therefore, by using video scenes as the segmentation nodes (i.e., segmenting the time points between two adjacent video scenes), the original video data is divided into multiple segments, which are denoted as candidate segments. Then, each candidate segment contains one or more independent video scenes.

[0054] In the specific implementation, the scene is detected on the original video data, and the segmentation points of video scene switching are represented on the original video data.

[0055] The detection methods include at least one of the following:

[0056] 1. Threshold mode

[0057] The threshold mode is suitable for raw video data with obvious video scene boundaries. It compares the color value of a specified channel of each frame of image data with a set first threshold (the first threshold is used to characterize the color value of video scene boundaries such as black and white). If the color value of the specified channel of the image data is greater than the first threshold, the boundaries of video scenes such as fade-in, fade-out, and cut to black can be detected. At this time, the frame of image data is set as the first target frame, and the time point of the first target frame is set as the cutting point. That is, the cutting point is the time point of the first target frame, and the first target frame is the image data with the color value of the specified channel greater than the first threshold.

[0058] 2. Content Format

[0059] Content mode is suitable for raw video data that switches quickly between video scenes. It compares the amount of content change between adjacent frames of image data. If the amount of change is greater than a second threshold, it means that the content change between two adjacent frames of image data is large, which belongs to the video scene switch. At this time, the two adjacent frames of image data are set as the second target frames, and the time point between the two second target frames is set as the split point. That is, the split point is the time point between the two second target frames. The two second target frames are two frames of image data that are adjacent in position and whose content change is greater than the second threshold.

[0060] When traversing the entire original video data, the original video data is segmented at each segmentation point to obtain multiple candidate segments.

[0061] Generally, data containing a single video scene can be segmented into a candidate segment. However, considering that some candidate segments containing a single video scene are short, editing them separately may result in a significant break in the continuity of the content, affecting the user's viewing experience.

[0062] Therefore, candidate segments can be sorted in chronological order, and each candidate segment can be traversed in the sorted order. During the traversal, the duration of the current candidate segment is queried in turn, and the duration is compared with a preset third threshold. If the duration is less than or equal to the third threshold, it means that the duration of the candidate segment is too short. In this case, the current candidate segment can be merged into other adjacent candidate segments (i.e., the candidate segment ranked above or below), thereby reducing the fragmentation of content during editing.

[0063] Step 103: Calculate the segment brilliance value, which visually represents the brilliance of each candidate segment.

[0064] For each candidate segment, deep learning and business rules (such as setting multiple targets and their scores in the business, and increasing or decreasing them according to their scores when a target is detected on the screen, and finally obtaining the segment's highlight value) can be used to quantify its highlight level (also known as importance) in the overall visual semantics (i.e. the content of the screen) to obtain the segment's highlight value.

[0065] Generally, the overall visual semantic appeal of a candidate segment is positively correlated with its segment appeal value. The higher the segment appeal value, the higher the overall visual semantic appeal of the candidate segment; conversely, the lower the segment appeal value, the lower the overall visual semantic appeal of the candidate segment.

[0066] In one approach, a summarization network (A Flexible Detect to Summarize Network for Video Summarization, DSNet) can be loaded into memory and run. The summarization network can extract the main parts of video data to generate segments, which are then used to summarize the content of the video data. The summarization network includes two network frameworks: an anchor-based method and an anchor-free method.

[0067] In the anchor-based method, a multi-scale interval of proposals (candidate boxes) is provided for dense sampling to extract their long-term, time-dependent features for proposal location regression and importance prediction. Here, positive and negative samples are assigned to generate correctness and completeness information for the summary.

[0068] In the anchor-free method, the importance of each frame of image data and segment position in the video data is directly predicted.

[0069] In this approach, each frame of image data from the candidate segments is input into a summarization network for processing to obtain a frame brilliance value that visually represents the brilliance of each frame of image data.

[0070] Generally, the visual semantic appeal of image data is positively correlated with the frame appeal value. The higher the frame appeal value, the higher the visual semantic appeal of the image data, and vice versa.

[0071] The frame highlights of all image data in the candidate segment are merged into the segment highlights, which visually represent the highlights of the candidate segment, by averaging and weighted summation.

[0072] Furthermore, such as Figure 2 As shown, a summary generation network can be preloaded and each frame of image data in the original video data can be input into the summary generation network for processing to obtain the frame highlight value, which visually represents the highlight level of each frame of image data. The frame highlight value is recorded in a highlight coordinate system 210, where the horizontal axis of the highlight coordinate system 210 is the frame ID and the vertical axis is the frame highlight value (frame_score).

[0073] When calculating the segment brilliance value that visually represents the brilliance of each candidate segment, the frame brilliance value that visually represents the brilliance of each frame image data in each candidate segment is read from the cache, and the frame brilliance values ​​of all image data in the candidate segment are merged into the segment brilliance value that visually represents the brilliance of the candidate segment, thereby improving the efficiency of calculating the segment brilliance value.

[0074] Step 104: Select some candidate segments as target segments based on the segment excellence value.

[0075] In practical applications, such as Figure 2As shown, there are many candidate segments from the original video data, and a large number of candidate sets are selected from some of the candidate segments. At this time, the segment highlights value can be used as one of the indicators to evaluate each candidate set, and one or more suitable candidate sets can be selected. The candidate segments in each candidate set are respectively denoted as target segments 220.

[0076] In one embodiment of the present invention, step 104 may include the following steps:

[0077] Step 1041: Set the total duration and total highlights as filter criteria.

[0078] Step 1042: Select some candidate segments as target segments according to the screening criteria, so as to maximize the total highlight value under the constraint of total duration.

[0079] In this embodiment, two variables can be set as filtering conditions. One variable is the total duration, which is the sum of the durations of the target segments. The other variable is the total highlight value, which is the sum of the highlight values ​​of the target segments.

[0080] Generally, the total duration can be set according to business needs (such as the limitations of the channels to be promoted). The total duration can be constrained to meet certain conditions, such as being less than a certain time value, or within a certain time range (such as 10-15 seconds, 25-30 seconds, etc.).

[0081] Therefore, we can consider the duration as the weight of a single element (i.e., a candidate segment) in the knapsack, the total duration as the weight of all elements (i.e., candidate segments) in the knapsack, the knapsack itself has a fixed capacity target (i.e., a condition constraining the total duration), the segment highlight value as the value of a single element (i.e., a candidate segment) in the knapsack, and the total highlight value as the total value of all elements (i.e., candidate segments) in the knapsack. Thus, selecting suitable candidate segments as target segments from the candidate segments is considered a knapsack problem (especially a 0-1 knapsack problem). When the total weight does not exceed the upper limit of the knapsack capacity, the knapsack problem is solved by using two loops (outer loop for candidate segments, inner loop for total highlight value) to make the total highlight value tend to a maximum value (the maximum value is the maximum value of the total highlight values ​​corresponding to all candidate sets), that is, to make the total highlight value as large as possible.

[0082] In another embodiment of the present invention, step 104 may further include the following steps:

[0083] Step 1043: If the candidate fragment contains a business scenario in the business object, then set the candidate fragment as a business fragment.

[0084] In practical applications, due to the diversity of raw video data, some content in the raw video data has a weak correlation with the business object. Furthermore, the visual appeal of the raw video data also has a weak correlation with the business object. Therefore, under the constraint of total duration, the target segments selected based on the total appeal value may have a weak connection with the business object, affecting the effectiveness of promoting the business object.

[0085] Therefore, in this embodiment, the total highlight value can be dynamically adjusted according to the characteristics of the business object, thereby improving the connection between the target segments selected based on the total highlight value and the business object under the constraint of the total duration, and ensuring the effectiveness of promoting the business object.

[0086] In practical implementation, scenario detection can be performed on candidate fragments to detect whether they contain business scenarios within business objects. Here, a business scenario can refer to a scenario constituted within a business object, which can reflect a relatively independent situation within the business object.

[0087] If a business object contains one or more business scenarios within a business object, then a candidate fragment can be set as a business fragment.

[0088] Furthermore, both video scenarios and business scenarios belong to a certain object. The object for detecting video scenarios is the original video data, while the object for detecting business scenarios is the candidate segments.

[0089] For business objects such as games, artists may use a collection of images from various business scenes in the game (such as equipment slots, character skill triggers, team dungeon runs, monster drops, etc.), each scene lasting only tens to hundreds of milliseconds, which is relatively short.

[0090] For raw video data, the duration varies from tens of seconds to 1 minute due to plot considerations. In addition to camera cuts, there are also ending segments (ECs). Therefore, a more lenient approach is usually used to segment the raw video data into video scenes, such as threshold mode. This results in a larger granularity of segmentation and a higher probability of segmenting the collection of images from various business scenarios of the game into the same video scene.

[0091] Therefore, in this embodiment, the video can first be segmented into candidate segments according to the video scene. Then, the candidate segments can be further segmented into smaller business scenes within the business object using a more detailed method (such as content mode).

[0092] In one embodiment of the present invention, step 1043 may further include the following steps:

[0093] Step 10431: Select candidate segments whose duration is outside the preset range and use them as reference segments.

[0094] In one design approach, artists edit business scenarios from a business object into short segments and splice these segments into a new segment. The new segment's footage may be directly set as the footage from the original video data and presented directly to the user, or the footage from the new segment may be embedded into the footage from the original video data in a picture-in-picture manner and introduced by actors.

[0095] Taking games as an example, artists edit out the game's equipment bar, character skill activation, team dungeon runs, monster drops, and other business scenarios separately. Each business scenario may be allocated a few milliseconds to hundreds of milliseconds in order to display the game's content as much as possible within a limited time.

[0096] Furthermore, these business scenarios can be displayed as original images, that is, the business scenarios are directly used as frames of the original video data, or they can be displayed as picture-in-picture, that is, the above business scenarios are displayed with rectangular (such as mobile phones) or circular (such as magic cave) borders, and explained by actors (such as spokespersons).

[0097] Although these segments are short and the business scenarios are simple, they contain a large amount of information about the business objects, showcasing all aspects of the business objects and giving users a macro-level initial impression. They are especially suitable for presenting the content of business objects when time is limited.

[0098] For the original image display business scenario, since there is switching between different business scenarios, when detecting video scenarios, one or more business scenarios may be segmented into candidate segments with shorter durations.

[0099] For picture-in-picture display scenarios, since most of the content on the screen does not change significantly, multiple scenarios may be segmented into longer candidate segments when detecting video scenes.

[0100] For this situation, a time range can be preset. If the candidate segment is less than the lower limit of the range, it may meet the business scenario of displaying the original image. If the candidate segment is greater than the upper limit of the range, it may meet the business scenario of displaying a picture-in-picture.

[0101] Then, the duration of all candidate segments is compared with this range in turn, and candidate segments whose duration is outside the range are selected from all candidate segments and recorded as reference segments.

[0102] Step 10432: Extract image data from the reference segment.

[0103] In this embodiment, all image data can be extracted from the reference segment, or partial image data can be extracted from the reference segment using sampling methods such as frame skipping at equal intervals.

[0104] Step 10433: Extract the image data containing the content of the business object.

[0105] In this embodiment, semantic recognition can be performed on each extracted frame of image data. In each frame of image data, it can be detected whether there is a scene of a business object. If there is, a scene of the business object is extracted from the frame of image data. This scene can be a part of the image data or the entire area of ​​the image data.

[0106] In the specific implementation, if the duration of the reference segment is less than the lower limit of the range, multiple regions are uniformly sampled in the image data and denoted as the first region. For example, a rectangular region is extracted from the top left corner, top right corner, bottom left corner, bottom right corner, and middle of the image data and denoted as the first region.

[0107] Each first region is input into a classifier for classification. This classifier can be a binary classifier, and the classification is either virtual world (especially the virtual world of a game) or real world. Therefore, the structure of the classifier can be a simple convolutional layer and a fully connected layer (FC).

[0108] Considering that simple classifiers may make some misjudgments, especially when actors are wearing game props (there are few samples of actors wearing game props, which makes the classifier training insufficient), the probability of misjudgment may be higher. For cases where the scene that matches the whole frame image data is the business object, we can count the proportion of the first region classified as virtual world among all first regions.

[0109] If the proportion of the first region classified as virtual world among all first regions is greater than or equal to the sixth threshold, it means that the proportion of the first region classified as virtual world among all first regions is relatively high. Since more regions belong to the virtual world, it can be determined that the content of the entire frame image data is the picture of the business object. Confirming the picture of the business object by the proportion can reduce the impact of classifier misjudgment.

[0110] If the duration of the reference segment exceeds the upper limit of the range, algorithms such as Roberts and Canny can be used to detect edges in the image data, extract a second region with edges of a specified shape (such as rectangle, circle, etc.) from the image data, and sample a third region outside the second region.

[0111] Considering that actors generally do not wear game props in this situation, the probability of misclassification by the classifier is low. In order to reduce the computational load of the classifier, the second and third regions can be input into the classifier for classification. This classifier can be a binary classification classifier, and the classification is either virtual world (especially the virtual world of the game) or real world. Therefore, the structure of the classifier can be a simple convolutional layer and a fully connected layer.

[0112] Iterate through the categories of the second region and the categories of the third region.

[0113] If the second region is classified as the virtual world and the third region as the real world, it means that the area within the specified shape at the edge of the image data frame is the virtual world and the area outside the specified shape at the edge is the real world. This matches the situation where the actor introduces the business object, so the content of the second region is determined to be the image of the business object.

[0114] Step 10434: Calculate the similarity between two adjacent frames.

[0115] In this embodiment, the similarity between two adjacent frames can be calculated using methods such as SSIM (structural similarity measure), cosine similarity, or histogram-based methods.

[0116] Step 10435: If the similarity is greater than or equal to the fourth threshold, then accumulate the second number of business scenarios for the reference fragment.

[0117] In this embodiment, a variable can be set for the reference segment, denoted as the second number of business scenarios. The second number of business scenarios is initialized to 1, indicating that the reference segment initially has at least one business scenario. Each time a similarity greater than or equal to the fourth threshold is traversed, the second number of business scenarios can be incremented by 1.

[0118] Step 10436: If the second quantity is greater than or equal to the fifth threshold, then set the reference segment as the business segment.

[0119] When traversing all the reference fragments containing business objects, if the second quantity is greater than or equal to the fifth threshold, it indicates that the main content of the reference fragment is the business scenario within the business object, and the reference fragment can be marked as a business fragment.

[0120] Step 1044: Count the first number of business segments in the candidate set.

[0121] Different candidate segments can form multiple candidate sets under the constraint of total duration. Then, a candidate set contains multiple candidate segments, and a candidate set is selected as the target segment when it meets the filtering conditions.

[0122] Iterate through each candidate set and count the first number of business segments in each candidate set.

[0123] Step 1045: Adjust the total brilliance value corresponding to the candidate set based on the first quantity.

[0124] Using the first number of business segments in each candidate set as a reference, the total highlight value corresponding to the candidate set is adjusted, thereby selecting some candidate segments as target segments according to the screening conditions, so as to maximize the adjusted total highlight value under the constraint of total duration.

[0125] In the specific implementation, if the first number of business segments in the candidate set is zero, the total brilliance value corresponding to the candidate set remains unchanged.

[0126] If the first number of business segments in the candidate set is one, then the second number of business scenarios in the business segments is counted, and the total excellence value of the candidate set is increased according to the second number.

[0127] Among them, the first magnitude of the total exciting value corresponding to the upward adjustment of the candidate set is positively correlated with the second quantity. That is, the more second quantities of business scenarios in the business segment, the greater the first magnitude of the total exciting value corresponding to the upward adjustment of the candidate set; conversely, the fewer second quantities of business scenarios in the business segment, the lower the first magnitude of the total exciting value corresponding to the upward adjustment of the candidate set.

[0128] If the first number of business segments in the candidate set is greater than one, then the total excellence value corresponding to the candidate set is increased according to the first number.

[0129] Among them, the second magnitude of the total wonderful value corresponding to the upward adjustment of the candidate set is positively correlated with the first quantity. That is, the more business segments there are in the candidate set, the greater the second magnitude of the total wonderful value corresponding to the upward adjustment of the candidate set; conversely, the fewer the business segments there are in the candidate set, the lower the first magnitude of the total wonderful value corresponding to the upward adjustment of the candidate set.

[0130] Furthermore, when the first number of business segments in the candidate set is one, the increase in the total performance value corresponding to the candidate set is a slight increase; when the first number of business segments in the candidate set is greater than one, the increase in the total performance value corresponding to the candidate set is a moderate increase. That is, the first magnitude of the increase in the total performance value corresponding to the candidate set is less than the second magnitude of the increase in the total performance value corresponding to the candidate set.

[0131] In this embodiment, business segments and business scenarios are characteristic representations of business objects. By increasing the total excellence value corresponding to the candidate set based on business segments and business scenarios, the probability of selecting a candidate set that is more relevant to the business object can be increased, thereby improving the relevance between the target segment and the business object and ensuring the effectiveness of promoting the business object.

[0132] Step 105: Assemble the target segments into target video data in sequence.

[0133] In this embodiment, as Figure 2 As shown, each target segment 220 is sorted according to its time on the timeline of the original video data, and each target segment is spliced ​​together in the sorting order. That is, the beginning of the next target segment is spliced ​​to the end of the previous target segment. When the splicing is completed, new video data is obtained, which is denoted as target video data 230.

[0134] In practical applications, the selected target segments may be continuous or discontinuous (i.e., other candidate segments are interspersed between the target segments on the timeline of the original video data). In order to ensure the quality of editing, each target video data can be presented to the art department for browsing and comparison, and the art department can further adjust the target video data.

[0135] The target video data contains information related to the business object. Subsequently, the target video data can be published on designated channels (such as news, short videos, novel reading, sports and health, etc.) so that when the client accesses the channel, the target video data is pushed to the client for playback. When users are interested in the business object, they can search for the business object through the information in the target video data. For example, they can search for and download games from a game distribution platform, etc.

[0136] In this embodiment, the original video data for promoting the business object is acquired; the original video data is divided into multiple candidate segments using video scenes as segmentation nodes; a segment brilliance value, which visually represents the level of brilliance, is calculated for each candidate segment; some candidate segments are selected as target segments based on the segment brilliance value; and the target segments are spliced ​​together in sequence to form the target video data. By eliminating the subjective judgment of artists, segmenting candidate segments according to video scenes ensures the integrity of the plot, and using segment brilliance values ​​as a reference for selecting candidate segments ensures the quality of the selected candidate segments, thereby guaranteeing the quality of the target video data. Furthermore, the original video data is reusable, allowing for the creation of numerous target video data sets with diverse content, reducing the workload of editing target video data, shortening the production cycle, and significantly reducing manpower and time costs, thus improving overall editing efficiency. The shorter production cycle meets the time-sensitive requirements of business objects with short promotion cycles, such as games.

[0137] Example 2

[0138] Figure 3 This is a schematic diagram of the structure of a video editing device provided in Embodiment 2 of the present invention. Figure 3 As shown, the device includes:

[0139] The raw video data acquisition module 301 is used to acquire raw video data for promotional business objects;

[0140] The candidate segment segmentation module 302 is used to segment the original video data into multiple candidate segments, using video scenes as segmentation nodes;

[0141] The highlight value calculation module 303 is used to calculate a segment highlight value that visually represents the degree of highlight for each of the candidate segments;

[0142] The target segment selection module 304 is used to select a portion of the candidate segments as target segments based on the segment excellence value.

[0143] The target video data splicing module 305 is used to splice the target segments into target video data in sequence.

[0144] In one embodiment of the present invention, the candidate segmentation module 302 is further configured to:

[0145] The segmentation point of video scene switching is characterized on the original video data. The segmentation point is the time point of the first target frame and / or the time point between two second target frames. The first target frame is image data with a color value of a specified channel greater than a first threshold. The two second target frames are two image data frames that are adjacent in position and have a change in content greater than a second threshold.

[0146] The original video data is segmented at the segmentation point to obtain multiple candidate segments.

[0147] In one embodiment of the present invention, the candidate segmentation module 302 is further configured to:

[0148] Query the duration of the currently selected candidate segment;

[0149] If the duration is less than or equal to the third threshold, the current candidate segment is merged into other adjacent candidate segments.

[0150] In one embodiment of the present invention, the brilliance value calculation module 303 is further configured to:

[0151] Load the summary to generate the network;

[0152] Each frame of image data in the candidate segment is input into the summary generation network for processing to obtain the frame brilliance value, which visually represents the brilliance of the image data.

[0153] The frame highlights of all the image data in the candidate segment are fused into a segment highlights value that visually represents the highlights of the candidate segment.

[0154] In one embodiment of the present invention, the target segment selection module 304 is further configured to:

[0155] Set the total duration and total highlights as the filtering conditions, where the total duration is the sum of the durations of the target segments, and the total highlights value is the sum of the highlights values ​​of the target segments;

[0156] The candidate segments are selected as target segments according to the filtering conditions, so as to maximize the total highlight value while constraining the total duration.

[0157] In one embodiment of the present invention, the target segment selection module 304 is further configured to:

[0158] If the candidate fragment contains a business scenario in the business object, then the candidate fragment is set as a business fragment;

[0159] The first number of the business segments is counted in the candidate set, wherein the candidate set contains multiple candidate segments, and the candidate set is selected as the target segment when it meets the filtering conditions;

[0160] The total brilliance value corresponding to the candidate set is adjusted based on the first quantity.

[0161] In one embodiment of the present invention, the target segment selection module 304 is further configured to:

[0162] Candidate segments whose duration falls outside the preset range are selected as reference segments;

[0163] Extract image data from the reference segment;

[0164] Extract the image data to identify the content of the business object;

[0165] Calculate the similarity between two adjacent frames of the image;

[0166] If the similarity is greater than or equal to the fourth threshold, then the second number of business scenarios is accumulated for the reference fragment;

[0167] If the second quantity is greater than or equal to the fifth threshold, then the reference segment is set as a business segment.

[0168] In one embodiment of the present invention, the target segment selection module 304 is further configured to:

[0169] If the duration of the reference segment is less than the lower limit of the range, then multiple first regions are uniformly sampled in the image data;

[0170] Each of the first regions is input into a classifier for classification.

[0171] If the proportion of the first region classified as virtual world is greater than or equal to the sixth threshold, then the content of the entire frame of image data is determined to be the image of the business object;

[0172] If the duration of the reference segment is greater than the upper limit of the range, then a second region with a specified shape is extracted from the image data, and a third region is sampled outside the second region;

[0173] The second region and the third region are respectively input into the classifier for classification;

[0174] If the second region is classified as the virtual world and the third region is classified as the real world, then the content of the second region is determined to be the image of the business object.

[0175] In one embodiment of the present invention, the target segment selection module 304 is further configured to:

[0176] If the first quantity is zero, then the total brilliance value corresponding to the candidate set is maintained;

[0177] If the first quantity is one, then the second quantity of the business scenario in the business segment is counted, and the total wonderful value corresponding to the candidate set is increased according to the second quantity, wherein the first increase is positively correlated with the second quantity;

[0178] If the first quantity is greater than one, then the total brilliance value corresponding to the candidate set is increased according to the first quantity, wherein the second increase is positively correlated with the first quantity.

[0179] The video editing apparatus provided in this embodiment of the invention can execute the video editing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the video editing method.

[0180] Example 3

[0181] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0182] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0183] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0184] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as video editing methods.

[0185] In some embodiments, the video editing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the video editing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the video editing method by any other suitable means (e.g., by means of firmware).

[0186] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0187] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0188] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0189] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0190] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0191] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0192] Example 4

[0193] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the video editing method provided in any embodiment of this invention.

[0194] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0195] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0196] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A video editing method, characterized in that, include: Obtain the raw video data used to promote business targets; Using video scenes as segmentation nodes, the original video data is segmented into multiple candidate segments; For each candidate segment, a segment brilliance value is calculated that visually represents the degree of brilliance. Based on the segment's brilliance value, select a portion of the candidate segments as target segments; The target segments are spliced ​​together into target video data in sequence; The step of selecting a portion of the candidate segments as target segments based on the segment highlights value includes: Set the total duration and total highlights as the filtering conditions, where the total duration is the sum of the durations of the target segments, and the total highlights value is the sum of the highlights values ​​of the target segments; Selecting some of the candidate segments as target segments according to the filtering conditions, so as to maximize the total highlight value under the constraint of the total duration; The step of selecting a portion of the candidate segments as target segments based on the segment highlights value further includes: If the candidate fragment contains a business scenario in the business object, then the candidate fragment is set as a business fragment; The first number of the business segments is counted in the candidate set, wherein the candidate set contains multiple candidate segments, and the candidate set is selected as the target segment when it meets the filtering conditions; The total brilliance value corresponding to the candidate set is adjusted based on the first quantity; Wherein, adjusting the total brilliance value corresponding to the candidate set based on the first quantity includes: If the first quantity is zero, then the total brilliance value corresponding to the candidate set is maintained; If the first quantity is one, then the second quantity of the business scenario in the business segment is counted, and the total wonderful value corresponding to the candidate set is increased according to the second quantity, wherein the first increase is positively correlated with the second quantity; If the first quantity is greater than one, then the total brilliance value corresponding to the candidate set is increased according to the first quantity, wherein the second increase is positively correlated with the first quantity.

2. The method according to claim 1, characterized in that, The method of using video scenes as segmentation nodes to segment the original video data into multiple candidate segments includes: The segmentation point of video scene switching is characterized on the original video data. The segmentation point is the time point of the first target frame and / or the time point between two second target frames. The first target frame is image data with a color value of a specified channel greater than a first threshold. The two second target frames are two image data frames that are adjacent in position and have a change in content greater than a second threshold. The original video data is segmented at the segmentation point to obtain multiple candidate segments.

3. The method according to claim 2, characterized in that, The method of segmenting the original video data into multiple candidate segments using video scenes as segmentation nodes also includes: Query the duration of the currently selected candidate segment; If the duration is less than or equal to the third threshold, the current candidate segment is merged into other adjacent candidate segments.

4. The method according to claim 1, characterized in that, The calculation of a segment brilliance value, which visually represents the brilliance of each candidate segment, includes: Load the summary to generate the network; Each frame of image data in the candidate segment is input into the summary generation network for processing to obtain the frame brilliance value, which visually represents the brilliance of the image data. The frame highlights of all the image data in the candidate segment are fused into a segment highlights value that visually represents the highlights of the candidate segment.

5. The method according to claim 1, characterized in that, If the candidate fragment contains a business scenario from the business object, then setting the candidate fragment as a business fragment includes: Candidate segments whose duration falls outside the preset range are selected as reference segments; Extract image data from the reference segment; Extract the image data to identify the content of the business object; Calculate the similarity between two adjacent frames of the image; If the similarity is greater than or equal to the fourth threshold, then the second number of business scenarios is accumulated for the reference fragment; If the second quantity is greater than or equal to the fifth threshold, then the reference segment is set as a business segment.

6. The method according to claim 5, characterized in that, The step of extracting the image data containing the content of the business object includes: If the duration of the reference segment is less than the lower limit of the range, then multiple first regions are uniformly sampled in the image data; Each of the first regions is input into a classifier for classification. If the proportion of the first region classified as virtual world is greater than or equal to the sixth threshold, then the content of the entire frame of image data is determined to be the image of the business object; If the duration of the reference segment is greater than the upper limit of the range, then a second region with a specified shape is extracted from the image data, and a third region is sampled outside the second region; The second region and the third region are respectively input into the classifier for classification; If the second region is classified as the virtual world and the third region is classified as the real world, then the content of the second region is determined to be the image of the business object.

7. A video editing device, characterized in that, include: The raw video data acquisition module is used to acquire raw video data for promotional business targets; The candidate segment segmentation module is used to segment the original video data into multiple candidate segments, using video scenes as segmentation nodes. The highlight value calculation module is used to calculate a segment highlight value, which visually represents the degree of highlight, for each of the candidate segments; The target segment selection module is used to select a portion of the candidate segments as target segments based on the segment excellence value. The target video data splicing module is used to splice the target segments into target video data in sequence. The target segment selection module is also used for: Set the total duration and total highlights as the filtering conditions, where the total duration is the sum of the durations of the target segments, and the total highlights value is the sum of the highlights values ​​of the target segments; Selecting some of the candidate segments as target segments according to the filtering conditions, so as to maximize the total highlight value under the constraint of the total duration; The target segment selection module is also used for: If the candidate fragment contains a business scenario in the business object, then the candidate fragment is set as a business fragment; The first number of the business segments is counted in the candidate set, wherein the candidate set contains multiple candidate segments, and the candidate set is selected as the target segment when it meets the filtering conditions; The total brilliance value corresponding to the candidate set is adjusted based on the first quantity; The target segment selection module is also used for: If the first quantity is zero, then the total brilliance value corresponding to the candidate set is maintained; If the first quantity is one, then the second quantity of the business scenario in the business segment is counted, and the total wonderful value corresponding to the candidate set is increased according to the second quantity, wherein the first increase is positively correlated with the second quantity; If the first quantity is greater than one, then the total brilliance value corresponding to the candidate set is increased according to the first quantity, wherein the second increase is positively correlated with the first quantity.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video editing method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the video editing method of any one of claims 1-6.

Images