Target video generation method and device

Abstract

The embodiments of this specification provide a target video generation method and device, wherein the target video generation method includes obtaining an initial video containing a target object; segmenting the initial video based on the target object, and segmenting the initial video into video segments containing the target object; extracting target segments corresponding to the category of the target object from the video segments according to the category of the target object; and generating a target video based on the target segments. After receiving the initial video containing the target object, the target video generation method can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content from each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a method for generating a target video. One or more embodiments of this specification also relate to a target video processing device, a computing device, and a computer-readable storage medium. Background Art

[0002] At present, with the growing growth of e-commerce live streaming, it is playing an increasingly important role in the e-commerce field. After each live broadcast, the live video can be used in two places: 1) to decorate the merchant's store; 2) to publish the video to other platforms or to the public domain to attract traffic for the store. However, the entire live video is usually quite long (for example, 4 hours, 5 hours, etc.), and a live video will contain multiple product introductions, and the process of each product introduction is relatively redundant. Therefore, if you want to apply a product in a live video to other places, you need to split the products in the entire live video, and then make a collection of the products, and then use them for other applications. However, most users currently do not have the ability to make collections of products.

[0003] Therefore, there is an urgent need to provide a target video generation method that can solve the above technical problems. Summary of the invention

[0004] In view of this, the present specification provides a method for generating a target video. One or more embodiments of the present specification also relate to a target video processing device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a target video generation method is provided, including:

[0006] Obtain an initial video containing a target object;

[0007] Segmenting the initial video based on the target object into video segments containing the target object;

[0008] Extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0009] A target video is generated based on the target segment.

[0010] According to a second aspect of the embodiments of this specification, a method for generating a target video is provided, including:

[0011] Displaying an image input interface to the user based on a call request from the user;

[0012] receiving an initial video including a target object and sent by the user based on the image input interface;

[0013] Segmenting the initial video based on the target object into video segments containing the target object;

[0014] Extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0015] A target video is generated based on the target segment and returned to the user.

[0016] According to a third aspect of the embodiments of this specification, a target video generation method is provided, including:

[0017] Receiving a call request sent by a user, wherein the call request carries an initial video containing a target object;

[0018] Segmenting the initial video based on the target object into video segments containing the target object;

[0019] Extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0020] A target video is generated based on the target segment and returned to the user.

[0021] According to a fourth aspect of the embodiments of this specification, a target video generating device is provided, including:

[0022] A first video receiving module is configured to obtain an initial video containing a target object;

[0023] A first video segmentation module is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object;

[0024] A first segment extraction module is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0025] The first video generation module is configured to generate a target video based on the target segment.

[0026] According to a fifth aspect of the embodiments of this specification, a target video generating device is provided, including:

[0027] An interface display module, configured to display an image input interface to the user based on a user's call request;

[0028] A second video receiving module is configured to receive an initial video including a target object and sent by the user based on the image input interface;

[0029] A second video segmentation module is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object;

[0030] A second segment extraction module is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0031] The second video generation module is configured to generate a target video based on the target segment and return the target video to the user.

[0032] According to a sixth aspect of the embodiments of this specification, there is provided a target video generating device, including:

[0033] A request receiving module is configured to receive a call request sent by a user, wherein the call request carries an initial video containing a target object;

[0034] A third video segmentation module is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object;

[0035] A third segment extraction module is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0036] The third video generation module is configured to generate a target video based on the target segment and return the target video to the user.

[0037] According to a seventh aspect of an embodiment of this specification, a computing device is provided, including:

[0038] Memory and processor;

[0039] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, the steps of the target video generation method are implemented.

[0040] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the target video generation method are implemented.

[0041] An embodiment of the present specification implements a target video generation method and device, wherein the target video generation method includes obtaining an initial video containing a target object; segmenting the initial video based on the target object, and segmenting the initial video into video segments containing the target object; extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object; and generating a target video based on the target segment. After receiving the initial video containing the target object, the target video generation method can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content for each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is an example diagram of a specific application scenario of a target video generation method provided by an embodiment of this specification;

[0043] Figure 2 is a flow chart of a first target video generation method provided by an embodiment of this specification;

[0044] Figure 3A is a schematic diagram of segmenting an initial video in a target video generation method provided by an embodiment of this specification;

[0045] Figure 3B It is a schematic diagram of extracting a target segment in a target video generation method provided by an embodiment of this specification;

[0046] Figure 4 is a specific application flow chart of a target video generation method provided by an embodiment of this specification;

[0047] Figure 5 is a flow chart of a second target video generation method provided by an embodiment of this specification;

[0048] Figure 6 is a flowchart of a third target video generation method provided by an embodiment of this specification;

[0049] Figure 7 It is a schematic diagram of the structure of a first target video generating device provided by an embodiment of this specification;

[0050] Figure 8 is a schematic diagram of the structure of a second target video generating device provided by an embodiment of this specification;

[0051] Fig. 9 is a schematic diagram of the structure of a third target video generating device provided by an embodiment of this specification;

[0052] Fig.10 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0053] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0054] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0056] First, the terms involved in one or more embodiments of this specification are explained.

[0057] Live video stripping: Usually a live video will contain live displays of multiple products. The entire live video is split into multiple segments through an algorithm, which is called live video stripping. Each segment only contains a single product.

[0058] Intelligent short video generation: Videos containing a single product are usually longer and contain clips of the host introducing the product, displaying the product, chatting with the audience, and other non-action clips. The main purpose of intelligent short video generation is to select multiple clips and make them into a short video, so that the short video can introduce a single product in a relatively simple but comprehensive way.

[0059] Interactive short video generation: Users can make fine adjustments based on the algorithm-generated algorithmic short videos to make the generated videos more accurate. The algorithm that generates short videos will also improve its algorithm performance based on user fine-tuning.

[0060] Anchor: refers to a "host-type announcer". Simply put, an announcer is reading other people's words; a host is speaking one's own words. An anchor is a combination of an announcer and a host; for example, an anchor who sells goods, that is, a person who displays and introduces goods through live broadcasting in order to sell goods, can be called an anchor.

[0061] In this specification, a target video generation method is provided. One or more embodiments of this specification simultaneously relate to a target video processing device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0062] See also Figure 1 , Figure 1 An example diagram showing a specific application scenario of a target video generation method provided by an embodiment of the present specification.

[0063] Figure 1 The application scenario includes a terminal 102 and a server 104. Specifically, a user sends a live video of a product to the server through the terminal 102, wherein the live video of the product includes live introductions to multiple products. After receiving the live video of the product, the server 104 splits the live video of the product into strips, that is, splits the live video of the product into multiple independent short videos based on the products. For example, the live video of the product includes live introductions to three products. Then, after splitting the live video of the product into strips, three independent short videos can be obtained, wherein each independent short video contains one product.

[0064] Then, each independent short video is detected for product highlights. According to the different categories of products contained in each independent short video, the highlight clips for the product in each independent short video are extracted, that is, different categories of products have different definitions of highlight clips; for example, if the product contained in an independent short video is clothing, then the host's display actions (such as the action of holding the product close to the camera or the action of wearing the product and turning around in front of the camera to show the product), display angles (such as the front, back, side and neckline details of the product), and language content introducing the clothing (such as keywords when the host introduces the clothing, such as color, style, price, etc.) can be used as highlights of the product; if the product contained in the independent short video is shoes and hats, then the host's display actions, display angles, enlarged details of shoes and hats, and language content introducing shoes and hats can be used as highlights of the product.

[0065] After extracting the highlights of the product in each independent short video, the extracted highlights can be displayed to the user, and the user can manually fine-tune the boundaries to make the highlights of the product more accurate and meet the standards for publication.

[0066] Finally, discrete highlights of each product are obtained through the above method, which are integrated based on the templates in the platform's existing template library or user-defined templates, and the final highlight video of each product is generated after adding special effects.

[0067] The target video generation method provided in the embodiment of this specification can realize the stripping of live video and the detection of highlight clips without the participation of users, and can allow users to adjust the highlight clips after obtaining the highlight clips of each product in the live video, thereby improving the performance of the highlight clip extraction algorithm while meeting the user's customization needs and improving the user experience, thereby further improving the performance of product highlight video generation.

[0068] See also Figure 2 , Figure 2 A flowchart of a first target video generation method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0069] Step 202: Acquire an initial video containing a target object.

[0070] Among them, the target objects include but are not limited to physical goods, virtual goods or characters, for example, physical goods can be clothing, shoes and hats, daily necessities or skin care products, virtual goods can be game character clothing, equipment or mounts, and characters can be characters in a TV series or stars in an entertainment program.

[0071] In practical applications, the initial video containing the target object may be a live video, a recorded video, or a video on demand containing the target object, that is, the target object may be a product, and the initial video may be a live video containing the product.

[0072] For ease of understanding, the embodiments of this specification all take the target object as a physical commodity and the initial video as a live video as an example to introduce the target video generation method in detail.

[0073] In practical applications, if the physical product is clothing, then obtaining an initial video containing the target object can be understood as obtaining an initial video containing various clothing, wherein the initial video can be a live video for various clothing.

[0074] Step 204: Segment the initial video based on the target object, and divide the initial video into video segments containing the target object.

[0075] The segmentation of the initial video based on the target object may include at least two segmentation methods, which are described in detail as follows:

[0076] The segmenting the initial video based on the target object includes:

[0077] The image of the preset target object is matched with the target object in the initial video, and the initial video is segmented based on the matching result.

[0078] Taking the target object as clothing as an example, the image of the preset target object can be understood as an image of the preset clothing.

[0079] Specifically, the target object in the initial video is also clothing. Then when the initial video is segmented, the image of the preset clothing is matched with the clothing in the initial video, and the initial video is segmented based on the matching result. That is, if the image of the preset clothing matches the clothing in the initial video successfully, the target object in the initial video is segmented from the successful matching point; if the image of the preset clothing fails to match the clothing in the initial video, the initial video is not segmented.

[0080] In the embodiments of the present specification, by presetting the image of the target object and matching the image of the preset target object with the target object in the initial video, the segmentation of the initial video can be quickly and conveniently achieved, which can greatly improve the segmentation efficiency of the initial video.

[0081] In a specific implementation, matching the image of the preset target object with the target object in the initial video, and segmenting the initial video based on the matching result, includes:

[0082] Matching the image of the preset target object with the target object in the initial video, and determining the first video frame and the last video frame in the initial video that contain the target object;

[0083] The initial video is segmented based on the first video frame and the last video frame containing the target object in the initial video.

[0084] In actual applications, each initial video may contain multiple target objects. When the initial video containing multiple target objects is segmented based on the image of the preset target object, the image of each preset target object can be matched with the target object in the initial video to determine the first video frame and the last video frame of the target object in the initial video that matches the image of the preset target object, and then the initial video is segmented based on the first video frame and the last video frame.

[0085] For example, the initial video includes three garments, garment 1 appears in the first video frame and the eighth video frame of the initial video, garment 2 appears in the ninth video frame and the twentieth video frame of the initial video, and garment 3 appears in the twenty-first video frame and the thirtieth video frame of the initial video. At this time, the image of the preset target object is the image of garment 2, so the image of the preset target object is matched with the target object in the initial video. According to the matching result, it can be determined that the first video frame (the ninth video frame) and the last video frame (the twentieth video frame) of the initial video contain garment 2, and then the ninth video frame to the twentieth video frame of the initial video are segmented to achieve segmentation of garment 2 from the initial video; if the image of the preset target object is the image of garment 1 or garment 3, the video containing garment 1 or garment 3 can be segmented in this way, which will not be repeated here.

[0086] In the embodiments of the present specification, by matching the image of the preset target object with the target object in the initial video, the video segments containing each target object in the initial video can be quickly and accurately segmented. The segmentation of the video segments containing the target objects in the initial video can be achieved without using complex algorithms, which greatly saves the system's video processing efficiency and enhances the user experience.

[0087] In addition, in the case where there is no image of a preset target object, the segmentation of the initial video can also be achieved, and the specific implementation method is as follows:

[0088] The segmenting the initial video based on the target object includes:

[0089] Determining a target object in each video frame of the initial video;

[0090] The target object of each video frame is matched with the target object in the adjacent video frame, and the initial video is segmented based on the matching result.

[0091] The target object is the target object contained in each video frame in the initial video, such as clothing, shoes and hats.

[0092] Specifically, in the absence of an image of a preset target object, the initial object in each video frame of the initial video can be determined first, and the initial object includes the target object and other objects, for example, if the target object in a certain video frame is clothing, then other objects can be props such as handbags or necklaces, and then the target object is determined based on the initial object, that is, the main object on each video frame is determined. For example, if the clothing occupies the largest area among the initial objects on a certain video frame, then the clothing can be considered to be the main object, that is, the target object; the target object in each video frame of the initial video is detected through this main target principle.

[0093] Then, the target object in each video frame is matched with the target object in the adjacent video frame, and the initial video is segmented based on the matching result; wherein the matching result is a matching result of the continuity between the target object in each video frame and the target object in the adjacent video frame.

[0094] In practical applications, a target object is detected on each frame of the initial video, and then the proportion of the target object in a certain video frame is compared with the proportion of the target object in the next video frame adjacent to the video frame. If the two proportions are large, it is determined that the target objects in the two video frames are the same target object. Through this strategy, each target object in the initial video is determined, and then each target object in the initial video is segmented.

[0095] See also Figure 3A , Figure 3A A schematic diagram of segmenting an initial video in a target video generation method provided in an embodiment of the present specification is shown.

[0096] Figure 3A In the process, the target object in each frame of the initial video is detected, and then the proportion of the target object in the current video frame is compared with the proportion of the target object in the next video frame, a curve is drawn according to the comparison result, and finally the initial video is segmented according to the curve.

[0097] For example Figure 3A The curve with the highest height in the video is the video segment containing the same target object from the first frame to the last frame of the curve.

[0098] In the embodiments of the present specification, accurate and fast segmentation of the initial video can be achieved by detecting the main target object and maintaining the continuity of the proportion of the main target object in adjacent video frames, thereby improving the user experience.

[0099] In a specific implementation, matching the target object of each video frame with the target object in an adjacent video frame, and segmenting the initial video based on the matching result, includes:

[0100] Matching the target object in the i-th video frame of the initial video with the target object in the i+1-th video frame;

[0101] If it is determined that the two target objects are the same target object, i is incremented by 1, and the target object in the i-th video frame of the initial video is matched with the target object in the i+1-th video frame;

[0102] If it is determined that the two target objects are different target objects, the initial video is segmented from the i-th video frame, and i is incremented by 1, and the target object in the i-th video frame of the initial video is continuously matched with the target object in the i+1-th video frame.

[0103] Specifically, the initial video contains n video frames, where i∈[1,n], and i is a positive integer. Taking i as 1 as an example, firstly, the target object in the first video frame of the initial video is matched with the target object in the second video frame. If the difference between the proportion of the target object in the first video frame in the first video frame and the proportion of the target object in the second video frame in the second video frame is less than or equal to a preset threshold, it means that the target object in the first video frame and the target object in the second video frame are the same target object. At this time, i is incremented by 1, and the target objects in the second video frame and the third video frame are matched in the above manner. If the first video frame If the difference between the proportion of the target object in the first video frame and the proportion of the target object in the second video frame is greater than a preset threshold, it means that the target object in the first video frame and the target object in the second video frame are not the same target object. At this time, the first video frame and the second video frame can be segmented to obtain a first video frame containing a target object that is different from the target object in the second video frame. At this time, i is incremented by 1, and the target objects in the second video frame and the third video frame are matched in the above manner, and so on, until i is greater than n. In this way, the initial video is segmented into multiple video clips containing different target objects.

[0104] In the embodiments of the present specification, the segmentation of the video segments of each target object is achieved by comparing the proportions of the target objects in adjacent video frames in the initial video, without the need to use other complex algorithms and other auxiliary images, thereby greatly improving the segmentation efficiency and accuracy of the initial video.

[0105] Step 206: extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object.

[0106] Different target objects have different categories; for example, if the target object is clothing, the category of the target object is clothing category; if the target object is shoes and hats, the category of the target object is shoes and hats category, and so on.

[0107] In actual applications, the target objects are of different categories, and the target segments extracted from the target objects are also different. For example, if the target object is clothing, the target segments corresponding to the category of the target object are extracted as the host's display actions, display angles, and language content introducing the clothing, etc. as the target segments of the target object; if the target object is shoes and hats, the target segments corresponding to the category of the target object are extracted as the host's display actions, display angles, magnified detail display of shoes and hats, and language content introducing shoes and hats, etc. as the target segments of the target object, wherein the target segments here have the same meaning as the highlight segments in the above-mentioned embodiment, that is, the more outstanding video content of the target object.

[0108] Specifically, extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object includes:

[0109] Determining a target segment extraction strategy according to the category of the target object;

[0110] A target segment corresponding to the category of the target object is extracted from the video segment based on the target segment extraction strategy.

[0111] The target segment extraction strategy may be understood as extracting which video contents are used as the target segments of the target object.

[0112] In specific implementation, different target objects have different target segment extraction strategies. As mentioned above, different target objects have different target segments. For example, the target object of the clothing category needs to extract the host's display actions, display angles, and language content introducing the clothing as the target segments of the target object. The target object of the shoes and hats category needs to extract the host's display actions, display angles, enlarged details of shoes and hats, and language content introducing shoes and hats as the target segments of the target object.

[0113] In the embodiments of the present specification, a target segment extraction strategy is first determined based on the category of the target object, and then a target segment matching the target object is extracted from a video segment containing the target object based on different target segment extraction strategies. Through this personalized approach, the extraction of different wonderful target segments of different target objects is achieved, and customized extraction of target segments is achieved. Based on the differences in target objects, wonderful target segments matching the target object are extracted, which can better meet the user's needs for extracting target segments of the target object and enhance the user experience.

[0114] See also Figure 3B , Figure 3B A schematic diagram of extracting a target segment in a target video generation method provided by an embodiment of this specification is shown.

[0115] Figure 3B The target segment extraction strategy includes three dimensions to extract the target segment of the target object in the video segment, wherein the three dimensions include the language content introducing the product, the display angle of the product, and the display action of the product.

[0116] Figure 3B In the method, based on the target segment extraction strategy including these three dimensions, the highlight segments in the video segment including the target object are extracted, that is, three highlight segments for the target object, namely, the target segments, are extracted.

[0117] Optionally, determining the target segment extraction strategy according to the category of the target object includes:

[0118] In the case where the video segment includes a display object of the target object, a target segment extraction strategy is determined according to the category of the target object.

[0119] The display object of the target object can be understood as the person displaying the target object, that is, only when there is a person displaying the target object in the video clip, the target clip extraction strategy can be determined according to the category of the target object.

[0120] As mentioned above, the target segment extraction strategy includes the host's display action and the display angle of the target object, etc. At this time, if the video segment does not include people, it can be determined that the target objects are all in a static state. At this time, it is impossible to extract the wonderful target segment of the target object well. Therefore, in the embodiment of this specification, when it is determined that there is a display object of the target object in a certain video segment, the target segment extraction strategy can be determined according to the category of the target object, so as to avoid the situation where there is no display object of the target object in the video segment, and the acquisition of the wonderful target segment of the target object in the video segment based on the target segment extraction strategy cannot be carried out, thereby avoiding the waste of system processing resources.

[0121] In order to allow the user to experience the fun of human-computer interaction, the initial video containing the target object is obtained, including:

[0122] Get the initial video containing the target object sent by the user.

[0123] Specifically, obtaining the initial video sent by the user and containing the target object can reflect the interactivity between the system and the user and improve the user experience.

[0124] At the same time, in order to improve the extraction accuracy of the target segment, before generating the target video based on the target segment, the target segment can also be adjusted based on the user's needs. The specific implementation method is as follows:

[0125] After extracting the target segment corresponding to the category of the target object, the method further includes:

[0126] An adjustment instruction from a user for the target segment is received, and the target segment is adjusted based on the adjustment instruction.

[0127] Specifically, after determining the target segment in each video segment containing the target object based on the target segment extraction strategy, the target segment will be displayed to the user. Then, after receiving the user's adjustment instruction for the target segment, the target segment will be adjusted based on the adjustment instruction, such as deleting the target segment, adjusting the position, or setting the playback multiple of the target segment; wherein the adjustment instruction includes but is not limited to an instruction to delete the target segment or an instruction to adjust the position, etc.

[0128] In addition, the target segment extraction method can be optimized based on the user's adjustment results to improve the target segment extraction performance for the target object, thereby improving the generation performance of the entire target video.

[0129] Step 208: Generate a target video based on the target segment.

[0130] Specifically, generating a target video based on the target segment includes:

[0131] Determine a video generation template matching the target segment from a preset template library based on the target segment, and generate a candidate video using the target segment through the video generation template;

[0132] The candidate video is rendered according to a preset special effect to generate a target video.

[0133] Among them, there will be multiple templates in the preset template library, and the corresponding templates will be different for different target objects. For example, if the target object is clothing, then the template will contain modules such as content introducing the clothing, some display actions of the anchor, and display angles. Then, the target clip of the target object can be used to generate a candidate video based on the template, and then the preset special effects can be added to generate the target video.

[0134] Specifically, the preset special effects may be background music, bubbles, text, some cartoon patterns, and other special effects.

[0135] In the embodiments of the present specification, after determining the target segment of the target object, the target video can be quickly generated based on the video generation template matching the target segment of the target object in the preset template library and the preset special effects, thereby improving the target video generation efficiency.

[0136] In another case, if the user wants to generate a personalized target video for the target segment of the target object, the video generation template can be customized to improve the user experience. The specific implementation method is as follows:

[0137] The generating a target video based on the target segment comprises:

[0138] Receiving the user-defined video generation template, and generating a candidate video from the target segment through the video generation template;

[0139] The candidate video is rendered according to a preset special effect to generate a target video.

[0140] In actual applications, different target objects require different display behaviors, such as the display behaviors of displaying skin care products and displaying clothing are different. At this time, if the video generation template in the preset template library cannot meet the user's needs for generating target videos, then the user-defined video generation template can be received, and the target segment can be generated into a target video through this personalized video generation template and preset special effects. That is, a video generation template that can support user-specified target segments can be generated according to user needs, and then the target video can be generated in combination with preset special effects.

[0141] In an embodiment of the present specification, after receiving an initial video containing a target object, the target video generation method can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content from each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience.

[0142] See also Figure 4 Taking the application of the target video generation method provided in the embodiment of this specification in the live video of a commodity as an example, the target video generation method is further explained, which specifically includes the following steps.

[0143] Step 402: The user provides a complete e-commerce live broadcast video.

[0144] Specifically, receive e-commerce live streaming videos containing goods provided by users, where the goods include but are not limited to clothing, shoes, accessories, and skin care products.

[0145] Step 404: Unpack the product.

[0146] Specifically, the above-mentioned e-commerce live broadcast video is split into multiple independent short videos based on products.

[0147] Step 406: Detection of a single product highlight segment.

[0148] Specifically, the highlight clips of the product in each independent short video are detected and extracted; and the extracted highlight clips of each product are displayed to the user.

[0149] Step 408: Determine whether to adjust, if yes, execute step 410, if not, execute step 412.

[0150] Specifically, it is determined whether the wonderful clips of the product extracted from each independent short video are adjusted. If so, the user's adjustment instruction is received; if not, the wonderful clips are collected.

[0151] Step 410: User manual adjustment.

[0152] Specifically, an adjustment instruction from a user is received, and the highlight segments of the product extracted from each independent short video are adjusted based on the adjustment instruction from the user.

[0153] Step 412: Collection of highlights.

[0154] Specifically, the wonderful clips of each product are sorted and collected.

[0155] Step 414: Determine whether the user customizes the template, if so, execute step 416, if not, execute step 418.

[0156] Step 416: Receive the user-defined video generation template and preset special effects.

[0157] Step 418: Based on the highlight clips of the product, determine from a preset template library a video generation template and preset special effects that match the highlight clips of the product.

[0158] Step 420: The highlight clips of the product are generated into a final product highlight video through the video generation template and preset special effects.

[0159] Specifically, through algorithms and user adjustments, multiple discrete highlight clips for each product are finally obtained. The combination of these discrete highlight clips depends on certain templates and special effects. Two options are provided in the embodiments of this specification. One is to select a template corresponding to the highlight clips of the product from the existing template library of the platform to realize the generation of a highlight video of the product. The other is to adopt a user-defined method to realize the generation of a highlight video of the product based on a user-defined personalized template. It is more flexible and can greatly enhance the user experience.

[0160] The target video generation method provided in the embodiment of this specification can automatically split the e-commerce live video, detect product highlights, and perform user fine-tuning after receiving the e-commerce live video containing products. Finally, a highlight video of the products can be produced based on templates in the platform's existing template library or user-defined templates combined with special effects, thereby reducing the complexity of producing a highlight video of the products, increasing users' interest and enthusiasm in producing a highlight video of the products, and improving user experience.

[0161] See also Figure 5 , Figure 5 A flow chart of a second target video generation method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0162] Step 502: Displaying an image input interface to the user based on the user's call request.

[0163] Step 504: receiving an initial video including a target object sent by the user based on the image input interface.

[0164] Step 506: Segment the initial video based on the target object, and divide the initial video into video segments containing the target object.

[0165] Step 508: extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object.

[0166] Step 510: Generate a target video based on the target segment and return it to the user.

[0167] In an embodiment of the present specification, after receiving an initial video containing a target object, the target video generation method can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content from each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience.

[0168] The above is a schematic scheme of the second target video generation method of this embodiment. It should be noted that the technical scheme of the second target video generation method and the technical scheme of the first target video generation method are of the same concept, and the details of the technical scheme of the second target video generation method that are not described in detail can all be referred to the description of the technical scheme of the first target video generation method.

[0169] See also Figure 6 , Figure 6 A flowchart of a third target video generation method provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0170] Step 602: receiving a call request sent by a user, wherein the call request carries an initial video containing a target object.

[0171] Step 604: Segment the initial video based on the target object, and divide the initial video into video segments containing the target object.

[0172] Step 606: extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object.

[0173] Step 608: Generate a target video based on the target segment and return it to the user.

[0174] In an embodiment of the present specification, after receiving an initial video containing a target object, the target video generation method can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content from each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience.

[0175] The above is a schematic scheme of the third target video generation method of this embodiment. It should be noted that the technical scheme of the third target video generation method and the technical scheme of the first target video generation method are of the same concept, and the details not described in detail in the technical scheme of the third target video generation method can be found in the description of the technical scheme of the first target video generation method.

[0176] Corresponding to the above method embodiment, this specification also provides a target video generating device embodiment, Figure 7 FIG. 1 shows a schematic diagram of the structure of a first target video generation device provided by an embodiment of this specification. Figure 7 As shown, the device comprises:

[0177] The first video receiving module 702 is configured to obtain an initial video containing a target object;

[0178] A first video segmentation module 704 is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object;

[0179] A first segment extraction module 706 is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0180] The first video generating module 708 is configured to generate a target video based on the target segment.

[0181] Optionally, the first video segmentation module 704 is further configured to:

[0182] The image of the preset target object is matched with the target object in the initial video, and the initial video is segmented based on the matching result.

[0183] Optionally, the first video segmentation module 704 is further configured to:

[0184] Matching the image of the preset target object with the target object in the initial video, and determining the first video frame and the last video frame in the initial video that contain the target object;

[0185] The initial video is segmented based on the first video frame and the last video frame containing the target object in the initial video.

[0186] Optionally, the first video segmentation module 704 is further configured to:

[0187] Determining a target object in each video frame of the initial video;

[0188] The target object of each video frame is matched with the target object in the adjacent video frame, and the initial video is segmented based on the matching result.

[0189] Optionally, the first video segmentation module 704 is further configured to:

[0190] Matching the target object in the i-th video frame of the initial video with the target object in the i+1-th video frame;

[0191] If it is determined that the two target objects are the same target object, i is incremented by 1, and the target object in the i-th video frame of the initial video is matched with the target object in the i+1-th video frame;

[0192] If it is determined that the two target objects are different target objects, the initial video is segmented from the i-th video frame, and i is incremented by 1, and the target object in the i-th video frame of the initial video is continuously matched with the target object in the i+1-th video frame.

[0193] Optionally, the first segment extraction module 706 is further configured to:

[0194] Determining a target segment extraction strategy according to the category of the target object;

[0195] A target segment corresponding to the category of the target object is extracted from the video segment based on the target segment extraction strategy.

[0196] Optionally, the first segment extraction module 706 is further configured to:

[0197] In the case where the video segment includes a display object of the target object, a target segment extraction strategy is determined according to the category of the target object.

[0198] Optionally, the first video receiving module 702 is further configured to:

[0199] Obtaining an initial video sent by a user and containing a target object;

[0200] Accordingly, the device further includes:

[0201] The adjustment module is configured to receive an adjustment instruction from a user for the target segment, and adjust the target segment based on the adjustment instruction.

[0202] Optionally, the first video generating module 708 is further configured to:

[0203] Determine a video generation template matching the target segment from a preset template library based on the target segment, and generate a candidate video using the target segment through the video generation template;

[0204] The candidate video is rendered according to a preset special effect to generate a target video.

[0205] Optionally, the first video generating module 708 is further configured to:

[0206] Receiving the user-defined video generation template, and generating a candidate video from the target segment through the video generation template;

[0207] The candidate video is rendered according to a preset special effect to generate a target video.

[0208] Optionally, the target object is a product, and the initial video is a live video containing the product.

[0209] In an embodiment of the present specification, after receiving an initial video containing a target object, the target video generating device can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content for each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience.

[0210] The above is a schematic scheme of the first target video generation device of this embodiment. It should be noted that the technical scheme of the target video generation device and the technical scheme of the first target video generation method mentioned above are of the same concept, and the details not described in detail in the technical scheme of the target video generation device can be found in the description of the technical scheme of the first target video generation method mentioned above.

[0211] Corresponding to the above method embodiment, this specification also provides a target video generating device embodiment, Figure 8 FIG. 2 shows a schematic diagram of the structure of a second target video generating device provided by an embodiment of the present specification. Figure 8As shown, the device comprises:

[0212] The interface display module 802 is configured to display an image input interface to the user based on a user's call request;

[0213] The second video receiving module 804 is configured to receive an initial video including a target object and sent by the user based on the image input interface;

[0214] A second video segmentation module 806 is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object;

[0215] A second segment extraction module 808 is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0216] The second video generating module 810 is configured to generate a target video based on the target segment and return the target video to the user.

[0217] In an embodiment of the present specification, after receiving an initial video containing a target object, the target video generating device can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content for each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience.

[0218] The above is a schematic scheme of the second target video generation device of this embodiment. It should be noted that the technical scheme of the target video generation device and the technical scheme of the second target video generation method described above are of the same concept, and the details of the technical scheme of the target video generation device that are not described in detail can all be referred to the description of the technical scheme of the second target video generation method described above.

[0219] Corresponding to the above method embodiment, this specification also provides a target video generating device embodiment, Fig. 9 FIG. 1 shows a schematic diagram of the structure of a first target video generation device provided by an embodiment of this specification. Fig. 9 As shown, the device comprises:

[0220] The request receiving module 902 is configured to receive a call request sent by a user, wherein the call request carries an initial video containing a target object;

[0221] A third video segmentation module 904 is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object;

[0222] A third segment extraction module 906 is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object;

[0223] The third video generation module 908 is configured to generate a target video based on the target segment and return the target video to the user.

[0224] In an embodiment of the present specification, after receiving an initial video containing a target object, the target video generating device can automatically split the initial video into independent short video segments according to the target object dimension, and then extract wonderful content for each independent short video segment, and finally intelligently and adaptively generate a highlight video of the target object based on the extracted wonderful content, thereby improving the user's highlight production experience.

[0225] The above is a schematic scheme of the third target video generation device of this embodiment. It should be noted that the technical scheme of the target video generation device and the technical scheme of the third target video generation method mentioned above are of the same concept, and the details of the technical scheme of the target video generation device that are not described in detail can all be referred to the description of the technical scheme of the third target video generation method mentioned above.

[0226] Fig.10 The block diagram of a computing device 1000 according to an embodiment of the present specification is shown. The components of the computing device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and the database 1050 is used to store data.

[0227] The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0228] In one embodiment of the present specification, the above components of the computing device 1000 and Fig.10 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.10The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0229] The computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 1000 may also be a mobile or stationary server.

[0230] The processor 1020 is used to execute the following computer executable instructions, wherein the processor implements the steps of the target video generation method when executing the computer executable instructions.

[0231] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the target video generation method described above are of the same concept, and the details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the target video generation method described above.

[0232] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which implement the steps of the target video generation method when executed by a processor.

[0233] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the target video generation method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the target video generation method described above.

[0234] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0235] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0236] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0237] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0238] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A target video generation method, comprising: Acquire an initial video containing a target object, wherein the initial video contains multiple target objects; Segmenting the initial video based on the target object into video segments containing the target object, wherein each video segment contains a target object; According to the category of the target object, a target segment corresponding to the category of the target object is extracted from the video segment, wherein the target object is different and the category of the target object is also different, and the target segment extraction strategy is also different for different target objects. The step of extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object comprises: Determining a target segment extraction strategy according to the category of the target object; Extracting a target segment corresponding to the category of the target object from the video segment based on the target segment extraction strategy, wherein the target segment extraction strategy is to extract which video content as the target segment of the target object; A target video is generated based on the target segment.

2. The target video generation method according to claim 1, wherein segmenting the initial video based on the target object comprises: The image of the preset target object is matched with the target object in the initial video, and the initial video is segmented based on the matching result.

3. The target video generation method according to claim 2, wherein the step of matching the image of the preset target object with the target object in the initial video and segmenting the initial video based on the matching result comprises: Matching the image of the preset target object with the target object in the initial video, and determining the first video frame and the last video frame in the initial video that contain the target object; The initial video is segmented based on the first video frame and the last video frame containing the target object in the initial video.

4. The target video generation method according to claim 1, wherein segmenting the initial video based on the target object comprises: Determining a target object in each video frame of the initial video; The target object of each video frame is matched with the target object in the adjacent video frame, and the initial video is segmented based on the matching result.

5. The target video generation method according to claim 4, wherein the target object in each video frame is matched with the target object in an adjacent video frame, and the initial video is segmented based on the matching result, comprising: Matching the target object in the i-th video frame of the initial video with the target object in the i+1-th video frame; If it is determined that the two target objects are the same target object, i is incremented by 1, and the target object in the i-th video frame of the initial video is matched with the target object in the i+1-th video frame; If it is determined that the two target objects are different target objects, the initial video is segmented from the i-th video frame, and i is incremented by 1, and the target object in the i-th video frame of the initial video is continuously matched with the target object in the i+1-th video frame.

6. The target video generation method according to claim 1, wherein the step of determining the target segment extraction strategy according to the category of the target object comprises: In the case where the video segment includes a display object of the target object, a target segment extraction strategy is determined according to the category of the target object.

7. The target video generation method according to claim 1, wherein obtaining the initial video containing the target object comprises: Obtaining an initial video sent by a user and containing a target object; Correspondingly, after extracting the target segment corresponding to the category of the target object, the method further includes: An adjustment instruction from the user for the target segment is received, and the target segment is adjusted based on the adjustment instruction.

8. The target video generation method according to claim 7, wherein generating the target video based on the target segment comprises: Determine a video generation template matching the target segment from a preset template library based on the target segment, and generate a candidate video using the target segment through the video generation template; The candidate video is rendered according to a preset special effect to generate a target video.

9. The target video generation method according to claim 7, wherein generating the target video based on the target segment comprises: Receiving the user-defined video generation template, and generating a candidate video from the target segment through the video generation template; The candidate video is rendered according to a preset special effect to generate a target video.

10. The target video generation method according to any one of claims 1 to 9, wherein the target object is a product, and the initial video is a live video containing the product.

11. A method for generating a target video, comprising: Displaying an image input interface to the user based on a call request from the user; receiving an initial video including a target object and sent by the user based on the image input interface, wherein the initial video includes a plurality of target objects; Segmenting the initial video based on the target object into video segments containing the target object, wherein each video segment contains a target object; According to the category of the target object, a target segment corresponding to the category of the target object is extracted from the video segment, wherein the target object is different and the category of the target object is also different, and the target segment extraction strategy is also different for different target objects. The step of extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object comprises: Determining a target segment extraction strategy according to the category of the target object; Extracting a target segment corresponding to the category of the target object from the video segment based on the target segment extraction strategy, wherein the target segment extraction strategy is to extract which video content as the target segment of the target object; A target video is generated based on the target segment and returned to the user.

12. A method for generating a target video, comprising: Receiving a call request sent by a user, wherein the call request carries an initial video containing a target object, and the initial video contains multiple target objects; Segmenting the initial video based on the target object into video segments containing the target object, wherein each video segment contains a target object; According to the category of the target object, a target segment corresponding to the category of the target object is extracted from the video segment, wherein the target object is different and the category of the target object is also different, and the target segment extraction strategy is also different for different target objects. The step of extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object comprises: Determining a target segment extraction strategy according to the category of the target object; Extracting a target segment corresponding to the category of the target object from the video segment based on the target segment extraction strategy, wherein the target segment extraction strategy is to extract which video content as the target segment of the target object; A target video is generated based on the target segment and returned to the user.

13. A target video generating device, comprising: A first video receiving module is configured to obtain an initial video containing a target object, wherein the initial video contains a plurality of target objects; A first video segmentation module is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object, wherein each video segment contains a target object; The first segment extraction module is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object, wherein the target object category is different for different target objects, and the target segment extraction strategy is different for different target object categories, The step of extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object comprises: Determining a target segment extraction strategy according to the category of the target object; Extracting a target segment corresponding to the category of the target object from the video segment based on the target segment extraction strategy, wherein the target segment extraction strategy is to extract which video content as the target segment of the target object; The first video generation module is configured to generate a target video based on the target segment.

14. A target video generating device, comprising: An interface display module, configured to display an image input interface to the user based on a user's call request; A second video receiving module is configured to receive an initial video containing a target object and sent by the user based on the image input interface, wherein the initial video contains a plurality of target objects; A second video segmentation module is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object, wherein each video segment contains a target object; The second segment extraction module is configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object, wherein the target object category is different if the target object is different, and the target segment extraction strategy is different if the target object category is different, The step of extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object comprises: Determining a target segment extraction strategy according to the category of the target object; Extracting a target segment corresponding to the category of the target object from the video segment based on the target segment extraction strategy, wherein the target segment extraction strategy is to extract which video content as the target segment of the target object; The second video generation module is configured to generate a target video based on the target segment and return the target video to the user.

15. A target video generating device, comprising: A request receiving module is configured to receive a call request sent by a user, wherein the call request carries an initial video containing a target object, and the initial video contains multiple target objects; A third video segmentation module is configured to segment the initial video based on the target object, and segment the initial video into video segments containing the target object, wherein each video segment contains one target object; a third segment extraction module, configured to extract a target segment corresponding to the category of the target object from the video segment according to the category of the target object, wherein the target object category is different if the target object is different, and the target segment extraction strategy is different if the target object category is different, The step of extracting a target segment corresponding to the category of the target object from the video segment according to the category of the target object comprises: Determining a target segment extraction strategy according to the category of the target object; Extracting a target segment corresponding to the category of the target object from the video segment based on the target segment extraction strategy, wherein the target segment extraction strategy is to extract which video content as the target segment of the target object; The third video generation module is configured to generate a target video based on the target segment and return the target video to the user.

16. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, the steps of the target video generation method described in any one of claims 1-10, 11, and 12 are implemented.

17. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the target video generation method described in any one of claims 1-10, 11, and 12.

Images