Target video generation method and device, computer equipment and storage medium

Abstract

The invention relates to a target video generation method and device, computer equipment and a storage medium. The method comprises the following steps: determining to-be-processed video clips in a plurality of video clips, and determining matching adjustment duration of each to-be-processed video clip; in response to the fact that the to-be-processed video clip is a to-be-expanded video clip, determining a clustering number according to the adjustment duration, and performing clustering according to the feature information of the to-be-expanded video clip, the time information of each frame of video image in the to-be-expanded video clip and the clustering number to generate a target clip; in response to the to-be-processed video clip being a to-be-clipped video clip, determining a first video style of the to-be-clipped video clip, and determining a target clip according to a relationship among an adjustment duration matched with the to-be-clipped video clip, the first video style and a second video style of the clipping template; and generating a target video at least according to the generated target clip and the editing template. By adopting the method, the video adaptive to the editing template can be generated.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a method, device, computer equipment, and storage medium for generating a target video. Background Art

[0002] With the development of various video platforms, users usually use various editing software to edit the videos they have shot. In order to improve the efficiency of video editing and cater to the various preferences of various users, various editing templates are added to the editing software.

[0003] However, usually, each video clip in each editing template has a fixed length. If the video clip that the user puts into the editing template is too short or too long, the editing template cannot accurately process the video that is too short or too long, and thus a video suitable for the editing template cannot be generated. Summary of the invention

[0004] Based on this, it is necessary to provide a target video generation method, device, computer equipment, and storage medium to address the above technical problems.

[0005] In a first aspect, the present disclosure provides a method for generating a target video. The method comprises:

[0006] Determine, according to the positions of the multiple video clips input into the editing template, a video clip to be processed among the multiple video clips, and determine the matching adjustment time length of each of the video clips to be processed;

[0007] In response to the to-be-processed video segment being a to-be-expanded video segment, determining the number of clusters according to the adjusted duration matched by the to-be-expanded video segment, and performing clustering according to feature information of the to-be-expanded video segment, time information of each frame of video image in the to-be-expanded video segment, and the number of clusters to obtain a plurality of video sub-segments;

[0008] Determine a video expansion method between the plurality of video sub-segments according to the image features of the plurality of video sub-segments to generate a target segment;

[0009] In response to the to-be-processed video segment being a to-be-cropped video segment, determining a first video style of the to-be-cropped video segment, and determining a target segment in the to-be-cropped video segment according to a relationship between an adjusted duration matching the to-be-cropped video segment, the first video style, and a second video style of the editing template;

[0010] A target video is generated at least according to the generated target segment and the editing template.

[0011] In one of the embodiments, clustering is performed according to the feature information of the video segment to be expanded, the time information of each frame of the video image in the video segment to be expanded, and the number of clusters to obtain a plurality of video sub-segments, including:

[0012] Clustering the video images according to feature information in each frame of the video image, time information corresponding to each frame of the video image, and the number of clusters to obtain a clustering result that matches the number of clusters;

[0013] A cluster boundary between the clustering results is determined, and the video segment to be expanded is segmented according to the cluster boundary to obtain a plurality of video sub-segments.

[0014] In one embodiment, determining the clustering boundaries between the clustering results includes:

[0015] In response to the existence of a time overlapping region in the clustering results, determining a cluster boundary between the clustering results based on a time point in the time overlapping region;

[0016] The determining the clustering boundaries between the clustering results based on the time points in the time overlapping area includes:

[0017] Determine a time median point in the time overlapping area, and determine a cluster boundary between clustering results according to the time median point;

[0018] Alternatively, determining the total amount of feature information contained in the time overlapping region, and determining the first number of clustering results having the same time overlapping region;

[0019] A time point for equally dividing the total number according to the first number is determined from the time overlapping region, and a clustering boundary between clustering results is determined based on the time point.

[0020] In one embodiment, after obtaining the clustering results matching the number of clusters, the method further includes:

[0021] Calculating the silhouette coefficient of the clustering result;

[0022] In response to the silhouette coefficient being less than a preset coefficient threshold, decomposing the feature information using a principal component analysis method to obtain feature information of multiple dimensions;

[0023] Combining the feature information of the multiple dimensions with the time information corresponding to each frame of the image to obtain a modified cluster feature group;

[0024] Clustering is performed according to the modified clustering feature group and the number of clusters to obtain a modified clustering result.

[0025] In one of the embodiments, determining the first video style of the video segment to be cropped includes:

[0026] Performing color extraction on each frame of the video image in the video segment to be cropped to obtain color information;

[0027] Constructing a color spectrum of each frame of video image according to the color information, and determining the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum;

[0028] Determining the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped;

[0029] The first video style of the video segment to be cropped is determined according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0030] In one of the embodiments, determining the target segment in the video segment to be cropped according to the relationship between the adjusted duration matching the video segment to be cropped, the first video style, and the second video style of the editing template includes:

[0031] In response to the second video style being the same as the first video style, determining a first material region in each frame of the video image of the to-be-cropped video segment that has a matching relationship with the first video style;

[0032] Determining a style score of each frame of the video image according to the first material area, and determining a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold;

[0033] In response to the second video style being different from the first video style, determining a second material region in each frame of the video segment to be cropped that has a matching relationship with the second video style;

[0034] According to the second material area, a style score of each frame of the video image is determined, and according to the style score of each frame of the video image and the duration threshold, a target segment in the video segment to be cropped is determined.

[0035] In one embodiment, when the first video style is a motion style, determining the target segment in the to-be-cropped video segment according to the relationship between the duration threshold, the first video style, and the second video style includes:

[0036] In response to the second video style being the same as the first video style, determining gesture information corresponding to each frame of the video image in the video segment to be cropped;

[0037] Determine the change range of the posture information between each frame of video images according to the posture information corresponding to each frame of video images;

[0038] Sorting each frame of video image according to the change amplitude, and screening the sorted video images according to the duration threshold to obtain at least one frame of video image that meets the duration threshold;

[0039] According to at least one frame of video image that meets the duration threshold, a target segment in the video segment to be cropped is determined.

[0040] In a second aspect, the present disclosure further provides a target video generation device. The device comprises:

[0041] A video segment determination module, used to determine a video segment to be processed among the multiple video segments according to the positions of the multiple video segments input into the editing template, and determine the matching adjustment time length of each of the video segments to be processed;

[0042] a cluster processing module, configured to, in response to the to-be-processed video segment being a to-be-extended video segment, determine the number of clusters according to the adjusted duration matched with the to-be-extended video segment, and perform clustering according to feature information of the to-be-extended video segment, time information of each frame of video image in the to-be-extended video segment, and the number of clusters, to obtain a plurality of video sub-segments;

[0043] An expansion module, configured to determine a video expansion method between the plurality of video sub-segments according to the image features of the plurality of video sub-segments, and generate a target segment;

[0044] a target segment determination module, configured to determine a first video style of the video segment to be cropped in response to the video segment to be processed being the video segment to be cropped, and determine a target segment in the video segment to be cropped according to a relationship between an adjustment duration matched with the video segment to be cropped, the first video style, and a second video style of the editing template;

[0045] The target video generation module is used to generate a target video based on at least the generated target segment and the editing template.

[0046] In a third aspect, the present disclosure further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0047] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0048] In a fifth aspect, the present disclosure further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0049] In the above embodiments, in response to the video segment to be processed being a video segment to be expanded, the number of clusters is determined according to the adjusted duration matched with the video segment to be expanded, and clustering is performed according to the feature information of the video segment to be expanded, the time information of each frame of the video image in the video segment to be expanded, and the number of clusters to obtain multiple video sub-segments. The number of clusters and the feature information can be used to accurately cluster the video images and classify similar video images into one category. In addition, since the clusters formed by traditional clustering are regular in the feature space, they may be disconnected and irregular in time, which will lead to the formation of many small video segments (segments clustered by video images). Therefore, in the clustering process, time information can be used for clustering to ensure that the multiple video sub-segments obtained after clustering are continuous in time. The video expansion method between the multiple video sub-segments is determined according to the image features of the multiple video sub-segments, and the expanded video segment is generated, which can accurately expand the video segment, and the image features can be used to judge the drastic degree of video change, so as to select a suitable video expansion method, ensure the effect of video expansion, and obtain a smoother target video. In response to the video segment to be processed being a video segment to be cropped, the first video style of the video segment to be cropped is determined, and the target segment in the video segment to be cropped is determined according to the relationship between the adjustment duration matching the video segment to be cropped, the first video style, and the second video style of the editing template. The determined first video style can provide information support for subsequent processing, thereby ensuring the accuracy of subsequent target segment determination. Since the style of the editing template will also have a certain degree of influence on the target segment identification, the second video style matching the editing template can be determined first, and the target segment in the video segment to be cropped is determined according to the duration threshold, the relationship between the first video style and the second video style. Thereby, the target segment is adapted to the second video style of the editing template. Finally, a target video is generated at least according to the generated target segment and the editing template, so that the degree of adaptation of the target segment and the editing template can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A schematic diagram of an application environment of a target video generation method in an embodiment;

[0052] Figure 2 is a flow chart of a method for generating a target video in one embodiment;

[0053] Figure 3 Schematic diagram of the process of step S204 in one embodiment;

[0054] Figure 4 is a schematic diagram of a clustering result in one embodiment;

[0055] Figure 5 is a schematic diagram of a time overlap region in one embodiment;

[0056] Figure 6 A schematic diagram of a process for determining cluster boundaries in one embodiment;

[0057] Figure 7 Schematic diagram of the process after step S302 in one embodiment;

[0058] Figure 8 A schematic diagram of a part of step S208 in one embodiment;

[0059] Fig. 9 is a schematic flow chart of another part of step S208 in one embodiment;

[0060] Fig.10 This is a schematic diagram of a further part of step S208 in one embodiment;

[0061] Fig.11 is a schematic block diagram of the structure of a target video generating device in one embodiment;

[0062] Fig.12 is a schematic diagram of the internal structure of a computer device in one embodiment;

[0063] Fig.13 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0065] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0066] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0067] The present disclosure provides a method for generating a target video, which can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 can determine the video segment to be processed in the multiple video segments according to the position of the multiple video segments in the terminal 102 input into the clip template. Multiple video segments can be pre-stored in the server 104 or the terminal 102. In response to the video segment to be processed being a video segment to be expanded, the server 104 can determine the number of clusters according to the adjusted duration matched by the video segment to be expanded, and cluster according to the feature information of the video segment to be expanded, the time information of each frame of the video image in the video segment to be expanded, and the number of clusters to obtain multiple video sub-segments. The server 104 determines the video expansion mode between the multiple video sub-segments according to the image features of the multiple video sub-segments, and generates a target segment. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, panoramic cameras, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers. It should be noted that this solution can also be applied to the terminal 102 or the server 104 alone.

[0068] In one embodiment, Figure 2 As shown, a target video generation method is provided, which is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0069] S202, determining a video segment to be processed among the multiple video segments according to positions of the multiple video segments input into the editing template, and determining a matching adjustment duration of each of the video segments to be processed.

[0070] Among them, the editing template refers to a preset format or design style used in the video editing process to help editors or users quickly make and arrange videos. Editing templates usually include elements such as video clips, transition effects, text styles, sound effects, etc., which can improve the creation efficiency and consistency of videos. Many video editing software and platforms (such as Adobe PremierePro, Final Cut Pro, Filmora, etc.) provide various editing templates, and users can choose and adjust them according to their needs and styles. Using editing templates can effectively save time. Multiple video clips are usually used in editing templates, and then transition effects are added to multiple video clips. The video duration of each video clip in the editing template can be a duration threshold. The video duration of each video clip in the editing template can be the same or different. The video clips to be processed may include: video clips to be expanded and video clips to be cut. The video clip to be cut is a video clip whose video duration is greater than the duration threshold. In some embodiments of the present disclosure, the video clip to be expanded may be a video clip whose video duration needs to be expanded, that is, the duration of the video clip is expanded to be the same as the duration threshold.

[0071] Specifically, according to the positions of multiple video clips input into the editing template, the duration threshold corresponding to each current video clip can be determined. Then, according to the duration threshold and the duration of the corresponding video clip, the video clip whose duration is different from the duration threshold is determined. The video clip whose duration is different from the duration threshold can be a video clip to be processed. When the duration of the video clip is greater than the duration threshold, the video clip can be a video clip to be cropped. When the duration of the video clip is less than the duration threshold, the video clip can be a video clip to be expanded. The absolute value of the difference between the duration of the video clip to be processed and the duration threshold can be used to determine the adjustment duration.

[0072] S204, in response to the video segment to be processed being a video segment to be expanded, determining the number of clusters according to the adjusted duration that matches the video segment to be expanded, and performing clustering according to feature information of the video segment to be expanded, time information of each frame of video image in the video segment to be expanded, and the number of clusters to obtain multiple video sub-segments.

[0073] The feature information may include: color features, texture features, shape features, etc. The number of clusters may generally be the number of cluster centers. The feature information may be extracted from the video segment to be expanded by feature extraction.

[0074] Specifically, when the video segment to be processed is a video segment to be expanded, the number of clusters can be determined according to the adjustment duration that matches the video segment to be expanded. Generally, the longer the adjustment duration, the greater the number of clusters. The time information corresponding to each frame of the video image can be determined. Then, the feature information and the time information in each frame of the image are combined. Then, the video image is clustered according to the combined information and the number of clusters to obtain a clustering result that matches the number of clusters. The video image is classified according to the clustering result to obtain multiple video sub-segments.

[0075] In some exemplary embodiments, a clustering algorithm (such as K-means) may be used to cluster the combined information, and the value of K may be set according to the number of clusters during the clustering process.

[0076] S206: Determine a video expansion method between the multiple video sub-segments according to the image features of the multiple video sub-segments to generate a target segment.

[0077] Among them, image features may include optical flow, pixel value, brightness, etc. Optical flow is an important concept in computer vision and image processing, which refers to the motion information generated by the change of pixel brightness caused by the movement of an object or camera in two consecutive frames of images. Optical flow can be used to estimate the speed and direction of the movement of an object in a scene. By analyzing images at different time points, the movement of an object on the image plane can be understood. The video expansion method can usually be a processing method between two video sub-segments, for example, it can include: cyclic alignment, transition effect, slow motion extension, adding static segments, etc. Cyclic alignment is a technology in video expansion and analysis, which is mainly used to process video segments with a cyclic mode, that is, to play the video segments in a loop. Such videos usually contain repeated actions or scenes. Transition effects may include: fade in and fade out, blur gradient, etc. Slow motion extension can be a slow motion processing of a video segment. Adding a static segment can be inserting a static picture related to a video sub-segment.

[0078] Specifically, the optical flow between adjacent video sub-segments can be calculated. The drastic degree of picture change between adjacent video sub-segments is determined based on the optical flow. For example, if the optical flow between the two video sub-segments is small, it means that the two videos are both slow-paced videos, then the video frames can be supplemented between the two video sub-segments by "circular alignment", thereby increasing the video duration. If the optical flow between the two video sub-segments is large, a transition effect can be added between the two video sub-segments, thereby increasing the video duration. It should be noted that those skilled in the art can flexibly select a video expansion method based on actual conditions and based on the optical flow, and in some embodiments of the present disclosure, there is no restriction on which specific video expansion method is selected.

[0079] Furthermore, a video expansion method between the multiple video sub-segments is determined according to the optical flows of the multiple video sub-segments, and the multiple video sub-segments are processed according to the determined video expansion method. The processed multiple video sub-segments are merged to generate a target segment.

[0080] For example, the optical flows of adjacent frames of video images in adjacent video sub-segments may be calculated. Taking two video sub-segments as an example, the optical flow of the last frame of video images in the previous video sub-segment may be calculated, and the optical flow of the first frame of video images in the next video sub-segment may be calculated, and then the optical flow difference between the optical flow of the last frame of video images and the optical flow of the first frame of video images may be calculated, and the video expansion method between the two video sub-segments may be determined according to the optical flow difference. Then, the two video sub-segments are processed according to the determined video expansion method, and then the processed video sub-segments are merged, so as to increase the duration of the video segment to be expanded and obtain the target segment.

[0081] S208, in response to the video segment to be processed being a video segment to be cropped, determining a first video style of the video segment to be cropped, and determining a target segment in the video segment to be cropped based on a relationship between an adjustment duration matching the video segment to be cropped, the first video style, and a second video style of the editing template.

[0082] Specifically, when the user determines the editing template, each editing template will have multiple corresponding tags, such as landscape, sports, cheerful, etc. The second video style matching the editing template can be determined according to the tag corresponding to the editing template, and then the target segment related to the second video style can be screened from the video segment to be cut according to the relationship between the first video style and the second video style and using the duration threshold. In addition, the first video style of the video segment to be cut can be determined by visual style analysis, or by using a pre-trained neural network model.

[0083] S210, generating a target video at least according to the generated target segment and the editing template.

[0084] Specifically, the to-be-processed video segment among the multiple video segments may be replaced with the target segment, thereby generating the target video according to the editing template.

[0085] In the target video generation method, in response to the to-be-processed video segment being a to-be-expanded video segment, the number of clusters is determined according to the adjusted duration matched with the to-be-expanded video segment, and clustering is performed according to the feature information of the to-be-expanded video segment, the time information of each frame of the video image in the to-be-expanded video segment, and the number of clusters to obtain a plurality of video sub-segments. The number of clusters and the feature information can be used to accurately cluster the video images, and similar video images can be classified into one category. In addition, since the clusters formed by traditional clustering are regular in the feature space, they may be disconnected and irregular in time, which will result in the formation of many small video segments (segments clustered by video images). Therefore, in the clustering process, the time information can be used for clustering to ensure that the plurality of video sub-segments obtained after clustering are continuous in time. The video expansion method between the plurality of video sub-segments is determined according to the image features of the plurality of video sub-segments, and the expanded video segment is generated, so that the video segment can be accurately expanded, and the image features can be used to judge the drastic degree of video changes, so as to select a suitable video expansion method, ensure the effect of video expansion, and obtain a smoother target video. In response to the video segment to be processed being a video segment to be cropped, the first video style of the video segment to be cropped is determined, and the target segment in the video segment to be cropped is determined according to the relationship between the adjustment duration matching the video segment to be cropped, the first video style, and the second video style of the editing template. The determined first video style can provide information support for subsequent processing, thereby ensuring the accuracy of subsequent target segment determination. Since the style of the editing template will also have a certain degree of influence on the target segment identification, the second video style matching the editing template can be determined first, and the target segment in the video segment to be cropped is determined according to the duration threshold, the relationship between the first video style and the second video style. Thereby, the target segment is adapted to the second video style of the editing template. Finally, a target video is generated at least according to the generated target segment and the editing template, so that the degree of adaptation of the target segment and the editing template can be ensured.

[0086] In one embodiment, Figure 3 As shown, clustering is performed according to the feature information of the video segment to be expanded, the time information of each frame of the video image in the video segment to be expanded, and the number of clusters to obtain multiple video sub-segments, including:

[0087] S302: Clustering the video images according to feature information in each frame of the video image, time information corresponding to each frame of the video image, and the number of clusters to obtain a clustering result that matches the number of clusters.

[0088] Specifically, the time information corresponding to each frame of the video image can be determined. Then, the feature information and the time information in each frame of the image are combined. Then, the video image is clustered according to the combined information and the number of clusters to obtain a clustering result that matches the number of clusters.

[0089] S304: Determine cluster boundaries between the clustering results, and segment the video segment to be expanded according to the cluster boundaries to obtain a plurality of video sub-segments.

[0090] The clustering boundary refers to the boundary or limit used to separate different clusters in cluster analysis. Figure 4 As shown, the clustering results are divided into two categories, namely clustering results A and clustering results B. The cluster boundary in clustering results A may be S1, and the cluster boundary in clustering results B may be S2.

[0091] Specifically, after determining the cluster boundary, the characteristic information contained in the cluster boundary can be determined, the video image corresponding to the characteristic information can be determined, and the determined video image can be divided into one segment, and so on to obtain multiple video sub-segments.

[0092] In this embodiment, by using cluster boundaries, the video clip to be expanded can be accurately segmented, and similar video images can be divided into one segment, thereby ensuring the smoothness of video expansion.

[0093] In one embodiment, determining the cluster boundaries between the cluster results includes:

[0094] In response to the existence of a time overlapping region in the clustering results, determining a cluster boundary between the clustering results based on a time point in the time overlapping region;

[0095] Specifically, the time overlapping area may be an area that overlaps in time but belongs to different clustering results. Figure 5 As shown, taking cluster results A and cluster results B as an example, cluster results A and cluster results B overlap in the time period from time X1 to time X2. Therefore, the cluster results in the time period from X1 to X2 can be the time overlapping area.

[0096] Specifically, when there are time overlapping regions in the clustering results, the clustering results need to be divided according to the time points in the time overlapping regions, so as to determine the clustering boundaries between the clustering results.

[0097] like Figure 6 As shown, the determining the clustering boundary between the clustering results based on the time points in the time overlapping area includes:

[0098] S402, determining a time median point in the time overlapping area, and determining a clustering boundary between clustering results according to the time median point.

[0099] Specifically, continue with Figure 5 As shown in the figure, the time period corresponding to the time overlapping area ( Figure 5 The time median point is determined by using the time median point to distinguish the cluster results A from the cluster results B. The feature information before the time median point can be classified into the cluster results A, and the feature information after the time median point can be classified into the cluster results B.

[0100] Alternatively, S404, determining the total amount of feature information contained in the time overlapping region, and determining the first number of clustering results having the same time overlapping region.

[0101] S406: Determine a time point for equally dividing the total number according to the first number from the time overlapping region, and determine a clustering boundary between clustering results based on the time point.

[0102] Specifically, the total number of feature information contained in the time overlapping region and the first number of clustering results with the same time overlapping region can be determined. The number of feature information corresponding to each time point corresponding to the time overlapping region can also be determined. The total number is divided equally according to the first number, and then the time point for dividing the total number equally is determined according to the number of feature information corresponding to each time point, and the time point can be a cluster boundary.

[0103] In some exemplary embodiments, for example, the total number of feature information contained in the time overlap region is 120, and the clustering results containing the time overlap region are two. There are five time points in the time overlap region, and the number of feature information corresponding to each time point is 10, 30, 20, 50, and 10, respectively. Therefore, it can be determined that the third time point is the cluster boundary, and there are 60 feature information before the third time point, and there are 60 feature information after the third time point. It can be understood that the above is only for illustration.

[0104] In this embodiment, the clustering results can be accurately divided by the time points in the time overlapping area, thereby ensuring the accuracy of subsequent processing. By using the time median point for division, the time boundary can be quickly determined in the time overlapping area, thereby improving the processing speed. In addition, the total amount of feature information contained in the time overlapping area can be determined; the time point for dividing the total amount in the time overlapping area is determined, and the cluster boundary between the clustering results is determined based on the time point, so that the determined time point can divide the feature information contained in the time overlapping area in half, thereby ensuring that the clustering features on both sides of the cluster boundary in the time overlapping area are basically the same, and ensuring the accuracy of the feature information in each clustering result. There will be no problem of too much or too little feature information due to the uneven distribution of feature information in multiple clustering results due to the appearance of time overlapping areas.

[0105] In one embodiment, Figure 7 As shown, after obtaining the clustering result matching the number of clusters, the method further includes:

[0106] S502: Calculate the silhouette coefficient of the clustering result.

[0107] S504: In response to the silhouette coefficient being less than a preset coefficient threshold, the feature information is decomposed using a principal component analysis method to obtain feature information of multiple dimensions.

[0108] Among them, the Silhouette Coefficient is an indicator used to evaluate the quality of clustering results, reflecting the similarity between a data point and its cluster, as well as the similarity with other clusters. The value of the Silhouette Coefficient is between -1 and 1, which can help judge the effect of clustering. Principal Component Analysis (PCA) is a commonly used dimensionality reduction technique used to map high-dimensional data to a low-dimensional space while retaining as much original information as possible. Usually, the coefficient threshold can be 0.5.

[0109] Specifically, the silhouette coefficient of each clustering result can be calculated. Then the silhouette coefficient is compared with the preset coefficient threshold. When the silhouette coefficient is less than the coefficient threshold, it can be determined that the clustering quality is poor, indicating that the feature points in the existing feature space are not suitable for clustering into multiple classes. For example, random noise data cannot be clustered into multiple classes. Therefore, for all feature information, assume that their feature vector latitude is n, and perform PCA decomposition on them to obtain new feature information with k latitude.

[0110] S506: Combining the feature information of the multiple dimensions with the time information corresponding to each frame of the image to obtain a modified cluster feature group.

[0111] S508: Perform clustering according to the modified clustering feature group and the number of clusters to obtain a modified clustering result.

[0112] Specifically, the new feature information and time can be combined to obtain a new cluster feature group, which can be a modified cluster feature group. Then clustering is performed according to the modified cluster feature group and the number of clusters to obtain a modified cluster result. For the specific implementation method here, please refer to the above embodiment, which will not be repeated here. Then, the steps mentioned in the above embodiment can be performed again according to the modified cluster result.

[0113] In this embodiment, due to the maximum variance theory of the PCA algorithm and the characteristics of reducing noise interference, the feature vector distance between new feature points is larger than before, and it is easier to find the inherent difference of the data and cluster it into multiple categories, thereby ensuring the accuracy of clustering.

[0114] In one embodiment, clustering the video images according to feature information in each frame of the video image, time information corresponding to each frame of the video image, and the number of clusters to obtain a clustering result matching the number of clusters includes:

[0115] The feature information in each frame of the video image and the time information corresponding to each frame of the video image are combined to obtain a cluster feature group.

[0116] Clustering is performed according to the cluster feature group and the cluster quantity to obtain a clustering result.

[0117] Specifically, the feature information and the corresponding time in each frame of the video image can be combined to form a new feature group, and the new feature group and the number of clusters are used to perform clustering calculation to obtain the final clustering result.

[0118] In some exemplary embodiments, a roulette wheel algorithm may be used for clustering. For example, a feature point in a feature group may be randomly selected as the first cluster center c1; the shortest distance between each of the remaining feature points in the feature group and the current existing cluster center (i.e., the distance to the nearest cluster center) is calculated, represented by D(x); the larger the value, the greater the probability of being selected as the cluster center; the roulette wheel method is used to select the next cluster center. The above steps are repeated until the same number of cluster centers as the number of clusters are selected.

[0119] In some other exemplary embodiments, the following formula may be used to iteratively adjust the cluster center:

[0120]

[0121] Among them, V represents the total variance or error measure of the clustering, which is usually used to evaluate the tightness of the clustering. The smaller the value, the better the clustering effect. K represents the number of clusters, that is, the number of different clusters into which the data is divided. i Represents the set of all data points in the i-th clustering result. j represents the jth data point belonging to cluster Si. i represents the center (mean) of the i-th cluster, that is, the average value of all data points in cluster Si. tj is the collection time of the j-th data; tistart is the earliest collection time of all data in the i-th class, that is, the start collection time of the class. j ∈[t istart , t end ] means that for any feature point xj in the i-th class Si, its collection time must be in the collection time interval of this class, which means that its collection time is not in other classes, which means that the collection time between the data in each class is continuous.

[0122] In this embodiment, by combining feature information and time information, the image acquisition time corresponding to all feature points in the same class can be made continuous, so that clustering can be performed better.

[0123] In one embodiment, Figure 8 As shown, the determining the first video style of the video segment to be cropped includes:

[0124] S602: Perform color extraction on each frame of the video image in the video segment to be cropped to obtain color information.

[0125] S604, constructing a color spectrum of each frame of video image according to the color information, and determining the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum.

[0126] Color extraction is a technology in the field of computer vision and image processing, which aims to identify and extract the color information contained in an image or video. Color information includes: saturation, brightness, color distribution and other information. A color atlas is a collection of color samples for each frame of a video image.

[0127] Specifically, color extraction is performed on each frame of video image to obtain color information. Color space conversion: Convert each frame of video image from BGR / RGB to a color space that is more suitable for analysis (such as HSV). Calculate the color histogram of each frame of video image to obtain color information. Organize the extracted color information into a color spectrum. You can use a dictionary or data frame (such as Pandas DataFrame) to store the color information of each frame of video image. For each frame of the color spectrum, calculate the frequency of use of different color values. You can consider using a clustering algorithm (such as K-means) to cluster colors and identify the main colors. Extract color combination features (such as primary color, auxiliary color) and analyze them. You can also use pre-trained models or machine learning methods to authenticate scene styles. Specific color combinations often correspond to a certain style, such as low saturation and high contrast may point to retro or literary styles.

[0128] S606: Determine the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped.

[0129] Specifically, according to the changes between each frame of video images, the movement mode of the lens (such as panning, pushing, pulling, shaking, etc.) can be recorded and classified. Fast and unstable movement may indicate a dynamic or tense style, while slow and steady shots may belong to a narrative or lyrical style.

[0130] S608: Determine a first video style of the video segment to be cropped according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0131] Specifically, the camera movement style and scene style of each frame can be encoded for subsequent analysis. Different camera movement methods can be represented by numerical values ​​or category labels. For example: translation: 0, rotation: 1, scaling: 2, stillness: 3. According to the color combination, the corresponding label is used. Primary and secondary colors or cluster labels can be used. The camera movement style and scene style of each frame are integrated into a data structure, such as a list or a data frame. The style combination of each frame in the entire video clip is analyzed, and the most common combination is found and summarized. The frequency of occurrence of each camera movement style and scene style combination is counted. Based on the frequency, the camera movement style and scene style combination with the highest frequency is selected. The first video style of the video clip to be cropped is determined based on the camera movement style and scene style combination with the highest frequency.

[0132] In this embodiment, the scene style of each frame of video image is determined by color analysis, and the camera movement style of each frame of video image is determined by the changes between each frame of video image. The first video style can be accurately determined based on the scene style and the camera movement style, thereby ensuring the accuracy of the target segment determination.

[0133] In one embodiment, determining the first video style of the video segment to be cropped according to the camera movement style between each frame of video image and the scene style of each frame of video image includes:

[0134] The image style of each frame of video image is determined according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0135] According to the image style of each frame of video image, the style ratio in the to-be-cropped video segment is determined.

[0136] The first video style of the video segment to be cropped is determined according to the image style with the largest style ratio.

[0137] Image style refers to the characteristics and artistic effects of an image in visual expression, including the uniqueness of color, texture, shape, line, composition, etc. Image styles can include: landscape style, sports style, portrait style, etc.

[0138] Specifically, in order to accurately determine the first video style of the video clip to be cropped, each frame of the video image in the video clip to be cropped can be processed separately. For each frame of the video image, the image style of the current frame of the video image can be determined according to its corresponding camera style and scene style. Then, the image style of each frame of the video image in the video clip to be cropped is counted, and the video images with the same image style are classified into one category. Then, the number of video images in each category is determined, and the style ratio of each category of video images is determined according to the number of video images in each category and the total number of video images. According to the style ratio of each category of video images, the image style with the largest style ratio is selected as the first video style.

[0139] In this embodiment, the style ratio is determined by using the image style of each frame of video image, and the first video style is determined by using the style ratio, so that the first video style that best matches the video segment to be cropped can be accurately found, thereby improving the accuracy of determining the target segment.

[0140] In one embodiment, Fig. 9 As shown, determining the target segment in the video segment to be cropped according to the relationship between the adjusted duration matching the video segment to be cropped, the first video style, and the second video style of the editing template includes:

[0141] S702: In response to the second video style being the same as the first video style, determine a first material region in each frame of the video segment to be cropped that has a matching relationship with the first video style.

[0142] The first material area may include: people, various landscapes (eg, mountains, water, trees, buildings, etc.), and objects (eg, cars, bicycles, etc.).

[0143] Specifically, when the second video style is the same as the first video style, material recognition can be performed on each frame of the video clip to be cropped, and the material region in each frame of the video image can be identified. The material region is then screened according to the first video style to determine the first material region.

[0144] In some exemplary embodiments, a frame of video image is used as an example for explanation, for example, it is recognized that the material areas in the current frame of video image include: A1 person material area, A2 person material area, B1 landscape material area, B2 landscape material area and C1 object material area. If the current first video style is a sports style (shooting people in motion), the first material area can be A1 person material area and A2 person material area. It can be understood that the above is only for illustration.

[0145] S704: Determine a style score of each frame of the video image according to the first material area, and determine a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold.

[0146] Specifically, the style score of each frame of video image can be determined according to information such as the proportion of the first material area and the area of ​​the first material area. After the style score of each frame of video image is determined, each frame of video image can be sorted according to the style score, and the frames with larger style scores can be sorted in front. Then, multiple frames of video images are selected according to the duration threshold to determine the target segment.

[0147] In some exemplary embodiments, usually, during the video shooting process, the content of the video will not suddenly change every moment, so the video images with similar style scores are usually continuous. If the frame rate of the video is 60 frames / s and the duration threshold is 3s, 180 frames of video images can be selected as the target segment according to the style score. The 180 frames of video images can be one or more continuous video images.

[0148] S706: In response to the second video style being different from the first video style, determine a second material region in each frame of the video segment to be cropped that has a matching relationship with the second video style.

[0149] Specifically, when the second video style is different from the first video style, since the video is generated using a clip template, it is necessary to determine the second material area that matches the second video style in each frame of the video image. It should be noted that, usually, the video segment to be cropped involved in some embodiments of the present disclosure may be a panoramic video, so each frame of the video segment to be cropped contains more material information, and the second material area is usually recognized. If the second material area is not recognized, the second material area of ​​the frame of the video image can be determined to be 0, and the corresponding style score of the frame of the video image will also be 0.

[0150] S708: Determine a style score of each frame of the video image according to the second material area, and determine a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold.

[0151] Specifically, the style score of each frame of video image can be determined according to the second material area in each frame of video image. After the style score of each frame of video image is determined, each frame of video image can be sorted according to the style score, and the frame with the largest style score can be sorted in the front. Then, multiple frames of video images are selected according to the duration threshold to determine the target segment. (For the specific implementation here, please refer to the above step S704, which will not be repeated here.)

[0152] In this embodiment, by comparing the second video style with the first video style, when they are the same, the style score of each frame of the video image is determined according to the first material area, thereby determining the target segment. When they are different, the style score of each frame of the video image is determined according to the second material area, thereby determining the target segment. This can ensure that the final target segment is more suitable for the editing template.

[0153] In one embodiment, determining the style score of each frame of the video image according to the first material area includes at least one of the following:

[0154] Determining a style score of each frame of the video image according to the number of the first material regions;

[0155] The style score of each frame of the video image is determined according to the proportion of the first material area in the video image.

[0156] Specifically, the more the number of first material regions in a certain frame of video image is, the greater the style score of the frame of video image is. The greater the proportion of the first material regions in a certain frame of video image in the entire video image is, the greater the style score of the frame of video image is.

[0157] In one embodiment, Fig.10As shown, when the first video style is a sports style. When the first video style includes a sports style. Wherein, the sports style refers to a visual style and expression method, which is usually used to convey a sense of dynamics, vitality and passion. This style is common in sports, extreme sports, dance, fitness videos or any video content that emphasizes action and activity. The determination of the target segment in the video segment to be cropped according to the relationship between the duration threshold, the first video style and the second video style includes:

[0158] S802: In response to the second video style being the same as the first video style, determine gesture information corresponding to each frame of the video image in the video segment to be cropped.

[0159] Among them, the attitude information can usually be IMU information. IMU information can usually be measured using an IMU. IMU (Inertial Measurement Unit) is a device used to measure and report the acceleration and angular velocity of an object. IMU is usually composed of multiple sensors, including accelerometers, gyroscopes, and sometimes magnetometers. Its main functions and components are as follows: Accelerometer: Measures the linear acceleration of an object in three axes (X, Y, Z). Through acceleration data, the motion state and direction changes of an object can be sensed. Gyroscope: Measures the angular velocity of an object (that is, the speed at which an object rotates around its axis), thereby enabling it to detect changes in the direction of the object. Magnetometer (optional): Measures the direction of the earth's magnetic field, which can be used to assist in determining the spatial orientation of an object, especially positioning in heading.

[0160] Specifically, when the second video style is the same as the first video style, it can be determined that the style adapted by the editing template is also a sports style. At this time, the IMU information generated by the device during the process of shooting each frame of the video segment to be cut can be obtained.

[0161] S804, determining a variation range of the posture information between each frame of video images according to the posture information corresponding to each frame of video images.

[0162] Specifically, since the IMU information reflects the various postures of the device, the change amplitude of the posture information between each frame of video images can be determined based on the IMU information corresponding to each frame of video images, and the change amplitude between each frame of video images can be determined based on the change amplitude of the posture information.

[0163] S806, sorting each frame of video image according to the change amplitude, and filtering the sorted video images according to the duration threshold to obtain at least one frame of video image that meets the duration threshold.

[0164] Specifically, because it is a sports style, the more dramatic the picture changes, the more dramatic the movement, and the greater the probability of a wonderful scene. Therefore, each frame of video images can be sorted according to the change range, and the video images with large changes are placed in the front position. Then, according to the duration threshold, the sorted video images can be screened to obtain at least one frame of video images that meets the duration threshold.

[0165] S808: Determine a target segment in the video segment to be cropped according to at least one frame of video image that meets the duration threshold.

[0166] In some exemplary embodiments, ten frames of video images are used as an example for explanation. Among the ten frames of video images, the change amplitude of the posture information between the fifth frame and the sixth frame of video images is the largest, and the change amplitudes between the video images of the remaining frames are the same. Therefore, the fifth frame and the sixth frame of video images can be placed at the front. If it is determined according to the duration threshold that 4 frames of video images need to be selected. However, the change amplitudes between the video images of the remaining frames are the same. Since the fifth frame of video images and the sixth frame of video images have been determined, in order to ensure the smoothness of the determined target segment, two frames of video images can be selected near the fifth frame of video images and the sixth frame of video images, and the fourth frame of video images and the seventh frame of video images can be selected. The fourth frame, the fifth frame, the sixth frame and the seventh frame of video images can be used as target segments.

[0167] In this embodiment, when the first video style is a motion style, since the device will move quickly or adjust the direction during shooting in the motion style, the posture information corresponding to each frame of the video image can be determined. The posture information can be used to determine the change range of the video image during the shooting process, thereby accurately determining the image with intense motion and ensuring the accuracy of the target segment determination.

[0168] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0169] Based on the same inventive concept, the embodiment of the present disclosure also provides a target video generation device for implementing the target video generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target video generation device embodiments provided below can refer to the limitations of the target video generation method above, and will not be repeated here.

[0170] In one embodiment, Fig.11 As shown, a target video generation device 900 is provided, comprising: a video segment determination module 802, a clustering processing module 804, an expansion module 806, a target segment determination module 808 and a target video generation module 810, wherein:

[0171] The video segment determination module 802 is used to determine the video segment to be processed among the multiple video segments according to the positions of the multiple video segments input into the editing template, and determine the matching adjustment time length of each of the video segments to be processed;

[0172] The cluster processing module 804 is used for determining the number of clusters according to the adjusted duration matched with the video segment to be expanded in response to the to-be-processed video segment being the to-be-expanded video segment, and performing clustering according to the feature information of the video segment to be expanded, the time information of each frame of the video image in the video segment to be expanded, and the number of clusters to obtain a plurality of video sub-segments;

[0173] An expansion module 806, configured to determine a video expansion method between the plurality of video sub-segments according to the image features of the plurality of video sub-segments, and generate a target segment;

[0174] a target segment determination module 808, configured to determine a first video style of the video segment to be cropped in response to the video segment to be processed being the video segment to be cropped, and determine a target segment in the video segment to be cropped according to a relationship between an adjustment duration matched with the video segment to be cropped, the first video style, and a second video style of the editing template;

[0175] The target video generation module 810 is used to generate a target video at least according to the generated target segment and the editing template.

[0176] In one embodiment of the device, the cluster processing module 804 includes:

[0177] A clustering module, used to cluster the video images according to feature information in each frame of the video image, time information corresponding to each frame of the video image and the number of clusters, to obtain a clustering result matching the number of clusters;

[0178] The segmentation module is used to determine the clustering boundaries between the clustering results, and segment the video segment to be expanded according to the clustering boundaries to obtain multiple video sub-segments.

[0179] In one embodiment of the device, the segmentation module comprises:

[0180] The cluster boundary determination module is used to determine the cluster boundaries between the cluster results based on the time points in the time overlapping area in response to the existence of the time overlapping area in the cluster results.

[0181] The cluster boundary determination module is also used to determine the time median point in the time overlapping area, and determine the cluster boundary between the clustering results based on the time median point; or, determine the total amount of feature information contained in the time overlapping area, and determine the first number of clustering results with the same time overlapping area; determine the time point from the time overlapping area that divides the total amount into two equal parts according to the first number, and determine the cluster boundary between the clustering results based on the time point.

[0182] In one embodiment of the device, the device also includes: a dimensionality expansion module, used to calculate the silhouette coefficient of the clustering result; in response to the silhouette coefficient being less than a preset coefficient threshold, using principal component analysis to decompose the feature information to obtain feature information of multiple dimensions.

[0183] The clustering module is further used to combine the feature information of the multiple dimensions with the time information corresponding to each frame of the video image to obtain a modified clustering feature group; clustering is performed according to the modified clustering feature group and the number of clusters to obtain a modified clustering result.

[0184] In one embodiment of the device, the target segment determination module 808 includes:

[0185] A color extraction module is used to extract color from each frame of the video image in the video segment to be cropped to obtain color information;

[0186] A scene style determination module, used to construct a color spectrum of each frame of video image according to the color information, and determine the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum;

[0187] A camera movement style determination module, used to determine the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped;

[0188] The video style determination module is used to determine the first video style of the video segment to be cropped according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0189] In one embodiment of the device, the target segment identification module includes:

[0190] The first material determination module is configured to determine, in response to the second video style being the same as the first video style, a first material region in each frame of the video segment to be cropped that has a matching relationship with the first video style.

[0191] The target recognition module is used to determine the style score of each frame of the video image according to the first material area, and determine the target segment in the video segment to be cropped according to the style score of each frame of the video image and the duration threshold.

[0192] The second material determination module is configured to determine, in response to the second video style being different from the first video style, a second material region in each frame of the video segment to be cropped that matches the second video style.

[0193] The target recognition module is further used to determine the style score of each frame of the video image according to the second material area, and determine the target segment in the video segment to be cropped according to the style score of each frame of the video image and the duration threshold.

[0194] In one embodiment of the device, when the first video style is a motion style, the target segment identification module is further used to determine, in response to the second video style being the same as the first video style, posture information corresponding to each frame of video image in the video segment to be cropped; determine, based on the posture information corresponding to each frame of video image, a variation range of the posture information between each frame of video image; sort each frame of video image according to the variation range, and filter the sorted video images based on the duration threshold to obtain at least one frame of video image that meets the duration threshold; and determine the target segment in the video segment to be cropped based on at least one frame of video image that meets the duration threshold.

[0195] Each module in the above-mentioned target video generation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0196] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.12As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video clip data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target video generation method is implemented.

[0197] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a target video generation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0198] Those skilled in the art will understand that Fig.11 The structure shown in or 12 is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0199] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0200] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0201] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0202] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0203] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. A target video generation method, characterized in that: The method comprises: Determine, according to the positions of the multiple video clips input into the editing template, a video clip to be processed among the multiple video clips, and determine the matching adjustment time length of each of the video clips to be processed; In response to the to-be-processed video segment being a to-be-expanded video segment, determining the number of clusters according to the adjusted duration matched by the to-be-expanded video segment, and performing clustering according to feature information of the to-be-expanded video segment, time information of each frame of video image in the to-be-expanded video segment, and the number of clusters to obtain a plurality of video sub-segments; Determine a video expansion method between the plurality of video sub-segments according to the image features of the plurality of video sub-segments to generate a target segment; In response to the to-be-processed video segment being a to-be-cropped video segment, determining a first video style of the to-be-cropped video segment, and determining a target segment in the to-be-cropped video segment according to a relationship between an adjusted duration matching the to-be-cropped video segment, the first video style, and a second video style of the editing template; A target video is generated at least according to the generated target segment and the editing template.

2. The method according to claim 1, characterized in that The clustering is performed according to the feature information of the video segment to be expanded, the time information of each frame of the video image in the video segment to be expanded, and the number of clusters to obtain a plurality of video sub-segments, including: Clustering the video images according to feature information in each frame of the video image, time information corresponding to each frame of the video image, and the number of clusters to obtain a clustering result that matches the number of clusters; A cluster boundary between the clustering results is determined, and the video segment to be expanded is segmented according to the cluster boundary to obtain a plurality of video sub-segments.

3. The method according to claim 2, characterized in that The determining the clustering boundaries between the clustering results includes: In response to the existence of a time overlapping region in the clustering results, determining a cluster boundary between the clustering results based on a time point in the time overlapping region; The determining the clustering boundaries between the clustering results based on the time points in the time overlapping area includes: Determine a time median point in the time overlapping area, and determine a cluster boundary between clustering results according to the time median point; Alternatively, determining the total amount of feature information contained in the time overlapping region, and determining the first number of clustering results having the same time overlapping region; A time point for equally dividing the total number according to the first number is determined from the time overlapping region, and a clustering boundary between clustering results is determined based on the time point.

4. The method according to claim 2, characterized in that: After obtaining the clustering results matching the number of clusters, the method further includes: Calculating the silhouette coefficient of the clustering result; In response to the silhouette coefficient being less than a preset coefficient threshold, decomposing the feature information using a principal component analysis method to obtain feature information of multiple dimensions; Combining the feature information of the multiple dimensions with the time information corresponding to each frame of the image to obtain a modified cluster feature group; Clustering is performed according to the modified clustering feature group and the number of clusters to obtain a modified clustering result.

5. The method according to claim 1, characterized in that The determining the first video style of the video segment to be cropped includes: Performing color extraction on each frame of the video image in the video segment to be cropped to obtain color information; Constructing a color spectrum of each frame of video image according to the color information, and determining the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum; Determining the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped; The first video style of the video segment to be cropped is determined according to the camera movement style between each frame of video image and the scene style of each frame of video image.

6. The method according to claim 1, characterized in that The determining of the target segment in the video segment to be cropped according to the relationship between the adjusted duration matching the video segment to be cropped, the first video style, and the second video style of the editing template comprises: In response to the second video style being the same as the first video style, determining a first material region in each frame of the video image of the to-be-cropped video segment that has a matching relationship with the first video style; Determining a style score of each frame of the video image according to the first material area, and determining a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold; In response to the second video style being different from the first video style, determining a second material region in each frame of the video segment to be cropped that has a matching relationship with the second video style; According to the second material area, a style score of each frame of the video image is determined, and according to the style score of each frame of the video image and the duration threshold, a target segment in the video segment to be cropped is determined.

7. The method according to claim 1, characterized in that When the first video style is a motion style, determining a target segment in the to-be-cropped video segment according to the relationship between the duration threshold, the first video style, and the second video style includes: In response to the second video style being the same as the first video style, determining gesture information corresponding to each frame of the video image in the video segment to be cropped; Determine the change range of the posture information between each frame of video images according to the posture information corresponding to each frame of video images; Sorting each frame of video image according to the change amplitude, and screening the sorted video images according to the duration threshold to obtain at least one frame of video image that meets the duration threshold; According to at least one frame of video image that meets the duration threshold, a target segment in the video segment to be cropped is determined.

8. A target video generating device, characterized in that: The device comprises: A video segment determination module, used to determine a video segment to be processed among the multiple video segments according to the positions of the multiple video segments input into the editing template, and determine the matching adjustment time length of each of the video segments to be processed; a cluster processing module, configured to, in response to the to-be-processed video segment being a to-be-extended video segment, determine the number of clusters according to the adjusted duration matched with the to-be-extended video segment, and perform clustering according to feature information of the to-be-extended video segment, time information of each frame of video image in the to-be-extended video segment, and the number of clusters, to obtain a plurality of video sub-segments; An expansion module, configured to determine a video expansion method between the plurality of video sub-segments according to the image features of the plurality of video sub-segments, and generate a target segment; a target segment determination module, configured to determine a first video style of the video segment to be cropped in response to the video segment to be processed being the video segment to be cropped, and determine a target segment in the video segment to be cropped according to a relationship between an adjustment duration matched with the video segment to be cropped, the first video style, and a second video style of the editing template; The target video generation module is used to generate a target video based on at least the generated target segment and the editing template.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images