Video cutting method and device, computer equipment and storage medium

Abstract

The invention relates to a video clipping method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: determining an editing template, and obtaining a to-be-clipped video clip; the video clip to be clipped is a video clip of which the video duration is greater than a duration threshold; the duration threshold value is determined according to an editing template; performing color analysis and shot language analysis on the to-be-clipped video clip, and determining a first video style of the to-be-clipped video clip according to analysis results of the color analysis and the shot language analysis; and determining a second video style matched with the editing template, and determining a target clip in the to-be-clipped video clip according to the duration threshold and the relationship between the first video style and the second video style. By adopting the method, the wonderful moments in the video can be automatically identified so as to adapt to the editing template.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video cropping method, device, computer equipment, and storage medium. Background Art

[0002] With the development of various portable shooting devices, users will use portable shooting devices to record in various application scenarios. After recording, in order to quickly edit into a film, users usually use editing templates to edit. Usually, in order to adapt to the editing template, the video with too long duration will be cropped.

[0003] However, in the process of trimming videos that are too long, if a manual selection method is used, the efficiency of the editing will be reduced when there are too many videos. Therefore, there is an urgent need for a method that can automatically identify the wonderful moments in the video and adapt the editing template. Summary of the invention

[0004] Based on this, it is necessary to provide a video cropping method, device, computer equipment, and storage medium to address the above technical problems.

[0005] In a first aspect, the present disclosure provides a video cropping method. The method comprises:

[0006] Determine a clipping template and obtain a video segment to be clipped; the video segment to be clipped is a video segment whose video duration is greater than a duration threshold; the duration threshold is determined according to the clipping template;

[0007] Performing color analysis and lens language analysis on the video segment to be cropped, and determining a first video style of the video segment to be cropped according to the analysis results of the color analysis and the lens language analysis;

[0008] A second video style matching the editing template is determined, and a target segment in the video segment to be cropped is determined according to the relationship between the duration threshold, the first video style, and the second video style.

[0009] In one of the embodiments, determining the target segment in the to-be-cropped video segment according to the relationship between the duration threshold, the first video style, and the second video style includes:

[0010] In response to the second video style being the same as the first video style, determining a first material region in each frame of the video image of the to-be-cropped video segment that has a matching relationship with the first video style;

[0011] Determining a style score of each frame of the video image according to the first material area, and determining a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold;

[0012] In response to the second video style being different from the first video style, determining a second material region in each frame of the video segment to be cropped that has a matching relationship with the second video style;

[0013] According to the second material area, a style score of each frame of the video image is determined, and according to the style score of each frame of the video image and the duration threshold, a target segment in the video segment to be cropped is determined.

[0014] In one embodiment, determining the style score of each frame of the video image according to the first material area includes at least one of the following:

[0015] Determining a style score of each frame of the video image according to the number of the first material regions;

[0016] The style score of each frame of the video image is determined according to the proportion of the first material area in the video image.

[0017] In one embodiment, when the first video style includes a motion style; determining the target segment in the to-be-cropped video segment according to the relationship between the duration threshold, the first video style, and the second video style includes:

[0018] In response to the second video style being the same as the first video style, determining gesture information corresponding to each frame of the video image in the video segment to be cropped;

[0019] Determine the change range of the posture information between each frame of video images according to the posture information corresponding to each frame of video images;

[0020] Sorting each frame of video image according to the change amplitude, and screening the sorted video images according to the duration threshold to obtain at least one frame of video image that meets the duration threshold;

[0021] According to at least one frame of video image that meets the duration threshold, a target segment in the video segment to be cropped is determined.

[0022] In one embodiment, performing color analysis and lens language analysis on the video segment to be cropped, and determining the first video style of the video segment to be cropped according to the analysis results of the color analysis and lens language analysis, comprises:

[0023] Performing color extraction on each frame of the video image in the video segment to be cropped to obtain color information;

[0024] Constructing a color spectrum of each frame of video image according to the color information, and determining the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum;

[0025] Determining the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped;

[0026] The first video style of the video segment to be cropped is determined according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0027] In one of the embodiments, determining the first video style of the video segment to be cropped according to the camera movement style between each frame of video image and the scene style of each frame of video image includes:

[0028] Determine the image style of each frame of video image according to the camera movement style between each frame of video image and the scene style of each frame of video image;

[0029] Determining the style ratio in the to-be-cropped video segment according to the image style of each frame of the video image;

[0030] The first video style of the video segment to be cropped is determined according to the image style with the largest style ratio.

[0031] In a second aspect, the present disclosure further provides a video cropping device. The device comprises:

[0032] The video acquisition module is used to determine the editing template and acquire the video segment to be cropped; the video segment to be cropped is a video segment whose video duration is greater than a duration threshold; the duration threshold is determined according to the editing template;

[0033] A video analysis module, configured to perform color analysis and lens language analysis on the video segment to be cropped, and determine a first video style of the video segment to be cropped according to the analysis results of the color analysis and the lens language analysis;

[0034] The segment determination module is used to determine a second video style that matches the editing template, and determine a target segment in the video segment to be cropped according to the relationship between the duration threshold, the first video style, and the second video style.

[0035] In a third aspect, the present disclosure further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0036] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0037] In a fifth aspect, the present disclosure further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0038] In the above embodiments, a clipping template is determined, a video clip to be cut is obtained, a color analysis and a lens language analysis are performed on the video clip to be cut, and the first video style of the video clip to be cut is determined based on the analysis results of the color analysis and the lens language analysis. Through the color analysis and the lens language analysis, the visual expression of the video clip can be effectively understood, and the analysis results can determine the first video style of the video clip to be cut, and the first video style can provide information support for subsequent processing, thereby ensuring the accuracy of the subsequent target clip determination. Since the style of the clipping template will also have a certain degree of influence on the target clip identification, the second video style that matches the clipping template can be determined first, and the target clip in the video clip to be cut can be determined based on the duration threshold, the relationship between the first video style and the second video style. Thereby, the target clip is adapted to the second video style of the clipping template. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A diagram showing an application environment of a video cropping method in an embodiment;

[0041] Figure 2 is a flow chart of a video cropping method in one embodiment;

[0042] Figure 3 This is a flow chart of an implementation method of step S206 in an embodiment;

[0043] Figure 4 This is a flow chart of another implementation method of step S206 in one embodiment;

[0044] Figure 5 Schematic diagram of the process of step S204 in one embodiment;

[0045] Figure 6 is a schematic diagram of the process of step S508 in one embodiment;

[0046] Figure 7 is a schematic block diagram of the structure of a video cropping device in one embodiment;

[0047] Figure 8 is a schematic diagram of the internal structure of a computer device in one embodiment;

[0048] Fig. 9 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0052] The present disclosure provides a video cropping method, which can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The user can determine the editing template in the terminal 102. The terminal 102 can obtain the video segment to be cut in the server 104. The video segment to be cut is a video segment whose video duration is greater than the duration threshold. The duration threshold can be determined according to the editing template. The terminal 102 can perform color analysis and lens language analysis on the video segment to be cut, and determine the first video style of the video segment to be cut according to the analysis results of the color analysis and lens language analysis. The terminal 102 can also determine the second video style that matches the editing template, and determine the target segment in the video segment to be cut according to the relationship between the duration threshold, the first video style and the second video style. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0053] In one embodiment, Figure 2 As shown, a video cropping method is provided, which is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0054] S202, determining a clipping template and obtaining a video segment to be clipped; the video segment to be clipped is a video segment whose video duration is greater than a duration threshold; the duration threshold is determined according to the clipping template.

[0055] Among them, the editing template refers to the preset format or design style used in the video editing process to help editors or users quickly produce and arrange videos. Editing templates usually include elements such as video clips, transition effects, text styles, sound effects, etc., which can improve the creation efficiency and consistency of the video. Many video editing software and platforms (such as Adobe PremierePro, Final Cut Pro, Filmora, etc.) provide various editing templates, and users can choose and adjust them according to their needs and styles. Using editing templates can effectively save time. Multiple video clips are usually used in editing templates, and then transition effects are added to multiple video clips to generate videos. The video length of each video clip in the editing template can be a duration threshold. The video length of each video clip in the editing template can be the same or different. According to the position of the video clip to be cut input into the editing template, the current duration threshold can be determined. For example, there are three video clips in the editing template, the first video clip has a duration of 5 seconds, the second video clip has a duration of 6 seconds, and the third video clip has a duration of 3 seconds. If the video clip to be cut will be input into the second video clip, the duration threshold may be 6 seconds. It should be noted that the duration of the video clip to be cut needs to be greater than 6 seconds.

[0056] Specifically, when it is necessary to edit the captured video, the user can first determine an editing template in the terminal, and then input the captured multiple video clips into the editing template. The editing template in the terminal can compare the duration of the video clips therein with the duration of the video clips input into the editing template, thereby determining and obtaining the video clips to be edited.

[0057] S204, performing color analysis and lens language analysis on the video segment to be cropped, and determining a first video style of the video segment to be cropped according to analysis results of the color analysis and the lens language analysis.

[0058] Among them, color analysis usually refers to the study of the use of color in the video, including: color selection, color composition analysis, among which color selection mainly analyzes which main colors are used in the video clips, and determines how these colors convey emotions and themes. Color composition mainly studies the distribution and contrast of colors in the picture, observes how colors guide the audience's attention, and their balance and harmony in the picture. Lens language analysis focuses on the visual narrative and shooting techniques in the video clips, including: Lens type: analyze which lenses are used (such as close-ups, long shots, close-ups, etc.), and how these lenses affect the narrative or character performance. Lens movement direction, etc. The first video style can include: sports style, landscape style, etc.

[0059] Specifically, the video clip to be cropped can be subjected to color analysis to record the main color tone, color cast, contrast color and other information of the video clip to be cropped. The color composition is analyzed according to the main color tone, color cast and contrast color and other information to determine the scene style of the video clip to be cropped. Then, the video clip to be cropped is subjected to lens language analysis to determine the movement mode and lens movement speed of the video clip to be cropped during the shooting process to determine the lens style. The first video style of the video clip to be cropped is jointly determined according to the lens style and the scene style. For example, the color is very bright and warm, the lens movement direction is a pull-in lens, and the lens movement speed is fast, so the first video style can be determined to be a cheerful and sporty style. The color is very dark and cold, the lens movement direction is a pull-in lens, and the lens movement speed is fast, so the first video can be determined to be a serious and sporty style. It should be noted that according to the analysis results of different color analysis and lens language analysis, those skilled in the art can flexibly determine the first video style, and how to determine the first video style is not absolutely limited in some embodiments of the present disclosure.

[0060] S206, determining a second video style that matches the editing template, and determining a target segment in the video segment to be cropped according to the relationship between the duration threshold, the first video style, and the second video style.

[0061] Specifically, when the user determines the editing template, each editing template will have multiple corresponding tags, such as landscape, sports, cheerful, etc. The second video style matching the editing template can be determined according to the tags corresponding to the editing template, and then the target segment related to the second video style can be screened from the video segments to be cut according to the relationship between the first video style and the second video style and using the duration threshold.

[0062] In the above-mentioned video cropping method, a clipping template is determined, a video clip to be cropped is obtained, a color analysis and a lens language analysis are performed on the video clip to be cropped, and the first video style of the video clip to be cropped is determined based on the analysis results of the color analysis and the lens language analysis. Through the color analysis and the lens language analysis, the visual expression of the video clip can be effectively understood, and the analysis results can determine the first video style of the video clip to be cropped, and the first video style can provide information support for subsequent processing, thereby ensuring the accuracy of the subsequent determination of the target clip. Since the style of the clipping template will also have a certain degree of influence on the recognition of the target clip, the second video style that matches the clipping template can be determined first, and the target clip in the video clip to be cropped can be determined based on the duration threshold, the relationship between the first video style and the second video style. Thereby, the target clip is adapted to the second video style of the clipping template.

[0063] In one embodiment, Figure 3As shown, determining the target segment in the to-be-cropped video segment according to the relationship between the duration threshold, the first video style, and the second video style includes:

[0064] S302: In response to the second video style being the same as the first video style, determining a first material region in each frame of the video segment to be cropped that has a matching relationship with the first video style.

[0065] The first material area may include: people, various landscapes (eg, mountains, water, trees, buildings, etc.), and objects (eg, cars, bicycles, etc.).

[0066] Specifically, when the second video style is the same as the first video style, material recognition can be performed on each frame of the video clip to be cropped, and the material region in each frame of the video image can be identified. The material region is then screened according to the first video style to determine the first material region.

[0067] In some exemplary embodiments, a frame of video image is used as an example for explanation, for example, it is recognized that the material areas in the current frame of video image include: A1 person material area, A2 person material area, B1 landscape material area, B2 landscape material area and C1 object material area. If the current first video style is a sports style (shooting people in motion), the first material area can be A1 person material area and A2 person material area. It can be understood that the above is only for illustration.

[0068] S304: determining a style score of each frame of the video image according to the first material area, and determining a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold.

[0069] Specifically, the style score of each frame of video image can be determined according to information such as the proportion of the first material area and the area of ​​the first material area. After the style score of each frame of video image is determined, each frame of video image can be sorted according to the style score, and the frames with larger style scores can be sorted in front. Then, multiple frames of video images are selected according to the duration threshold to determine the target segment.

[0070] In some exemplary embodiments, usually, during the video shooting process, the content of the video will not suddenly change every moment, so the video images with similar style scores are usually continuous. If the frame rate of the video is 60 frames / s and the duration threshold is 3s, 180 frames of video images can be selected as the target segment according to the style score. The 180 frames of video images can be one or more continuous video images.

[0071] S306: In response to the second video style being different from the first video style, determine a second material region in each frame of the video segment to be cropped that matches the second video style.

[0072] Specifically, when the second video style is different from the first video style, since the video is generated using a clip template, it is necessary to determine the second material area that matches the second video style in each frame of the video image. It should be noted that, usually, the video segment to be cropped involved in some embodiments of the present disclosure may be a panoramic video, so each frame of the video segment to be cropped contains more material information, and the second material area is usually recognized. If the second material area is not recognized, the second material area of ​​the frame of the video image can be determined to be 0, and the corresponding style score of the frame of the video image will also be 0.

[0073] S308: determining a style score of each frame of the video image according to the second material area, and determining a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold.

[0074] Specifically, the style score of each frame of video image can be determined according to the second material area in each frame of video image. After the style score of each frame of video image is determined, each frame of video image can be sorted according to the style score, and the frame with the largest style score can be sorted in the front. Then, multiple frames of video images are selected according to the duration threshold to determine the target segment. (For the specific implementation here, please refer to the above step S304, which will not be repeated here.)

[0075] In this embodiment, by comparing the second video style with the first video style, when they are the same, the style score of each frame of the video image is determined according to the first material area, thereby determining the target segment. When they are different, the style score of each frame of the video image is determined according to the second material area, thereby determining the target segment. This can ensure that the final target segment is more suitable for the editing template.

[0076] In one embodiment, determining the style score of each frame of the video image according to the first material area includes at least one of the following:

[0077] Determining a style score of each frame of the video image according to the number of the first material regions;

[0078] The style score of each frame of the video image is determined according to the proportion of the first material area in the video image.

[0079] Specifically, the more the number of first material regions in a certain frame of video image is, the greater the style score of the frame of video image is. The greater the proportion of the first material regions in a certain frame of video image in the entire video image is, the greater the style score of the frame of video image is.

[0080] In one embodiment, when the first video style includes a sports style, the sports style refers to a visual style and expression technique that is usually used to convey a sense of dynamics, vitality, and passion. This style is common in sports, extreme sports, dance, fitness videos, or any video content that emphasizes action and activity. Figure 4 As shown, determining the target segment in the to-be-cropped video segment according to the relationship between the duration threshold, the first video style, and the second video style includes:

[0081] S402: In response to the second video style being the same as the first video style, determining gesture information corresponding to each frame of the video image in the video segment to be cropped.

[0082] Among them, the attitude information can usually be IMU information. IMU information can usually be measured using an IMU. IMU (Inertial Measurement Unit) is a device used to measure and report the acceleration and angular velocity of an object. IMU is usually composed of multiple sensors, including accelerometers, gyroscopes, and sometimes magnetometers. Its main functions and components are as follows: Accelerometer: Measures the linear acceleration of an object in three axes (X, Y, Z). Through acceleration data, the motion state and direction changes of an object can be sensed. Gyroscope: Measures the angular velocity of an object (that is, the speed at which an object rotates around its axis), thereby enabling it to detect changes in the direction of the object. Magnetometer (optional): Measures the direction of the earth's magnetic field, which can be used to assist in determining the spatial orientation of an object, especially positioning in heading.

[0083] Specifically, when the second video style is the same as the first video style, it can be determined that the style adapted by the editing template is also a sports style. At this time, the IMU information generated by the device during the process of shooting each frame of the video segment to be cut can be obtained.

[0084] S404, determining a variation range of the posture information between each frame of video images according to the posture information corresponding to each frame of video images.

[0085] Specifically, since the IMU information reflects the various postures of the device, the change amplitude of the posture information between each frame of video images can be determined based on the IMU information corresponding to each frame of video images, and the change amplitude between each frame of video images can be determined based on the change amplitude of the posture information.

[0086] S406, sorting each frame of video image according to the change amplitude, and filtering the sorted video images according to the duration threshold to obtain at least one frame of video image that meets the duration threshold.

[0087] Specifically, because it is a sports style, the more dramatic the picture changes, the more dramatic the movement, and the greater the probability of a wonderful scene. Therefore, each frame of video images can be sorted according to the change range, and the video images with large changes are placed in the front position. Then, according to the duration threshold, the sorted video images can be screened to obtain at least one frame of video images that meets the duration threshold.

[0088] S408: Determine a target segment in the video segment to be cropped according to at least one frame of video image that meets the duration threshold.

[0089] In some exemplary embodiments, ten frames of video images are used as an example for explanation. Among the ten frames of video images, the change amplitude of the posture information between the fifth frame and the sixth frame of video images is the largest, and the change amplitudes between the video images of the remaining frames are the same. Therefore, the fifth frame and the sixth frame of video images can be placed at the front. If it is determined according to the duration threshold that 4 frames of video images need to be selected. However, the change amplitudes between the video images of the remaining frames are the same. Since the fifth frame of video images and the sixth frame of video images have been determined, in order to ensure the smoothness of the determined target segment, two frames of video images can be selected near the fifth frame of video images and the sixth frame of video images, and the fourth frame of video images and the seventh frame of video images can be selected. The fourth frame, the fifth frame, the sixth frame and the seventh frame of video images can be used as target segments.

[0090] In this embodiment, when the first video style is a motion style, since the device will move quickly or adjust the direction during shooting in the motion style, the posture information corresponding to each frame of the video image can be determined. The posture information can be used to determine the change range of the video image during the shooting process, thereby accurately determining the image with intense motion and ensuring the accuracy of the target segment determination.

[0091] In one embodiment, Figure 5 As shown, the color analysis and the lens language analysis are performed on the video segment to be cropped, and the first video style of the video segment to be cropped is determined according to the analysis results of the color analysis and the lens language analysis, including:

[0092] S502: Perform color extraction on each frame of the video image in the video segment to be cropped to obtain color information.

[0093] S504, constructing a color spectrum of each frame of video image according to the color information, and determining the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum.

[0094] Color extraction is a technology in the field of computer vision and image processing, which aims to identify and extract the color information contained in an image or video. Color information includes: saturation, brightness, color distribution and other information. A color atlas is a collection of color samples for each frame of a video image.

[0095] Specifically, color extraction is performed on each frame of video image to obtain color information. Color space conversion: Convert each frame of video image from BGR / RGB to a color space that is more suitable for analysis (such as HSV). Calculate the color histogram of each frame of video image to obtain color information. Organize the extracted color information into a color spectrum. You can use a dictionary or data frame (such as Pandas DataFrame) to store the color information of each frame of video image. For each frame of the color spectrum, calculate the frequency of use of different color values. You can consider using a clustering algorithm (such as K-means) to cluster colors and identify the main colors. Extract color combination features (such as primary color, auxiliary color) and analyze them. You can also use pre-trained models or machine learning methods to authenticate scene styles. Specific color combinations often correspond to a certain style, such as low saturation and high contrast may point to retro or literary styles.

[0096] S506: Determine the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped.

[0097] Specifically, according to the changes between each frame of video images, the movement mode of the lens (such as panning, pushing, pulling, shaking, etc.) can be recorded and classified. Fast and unstable movement may indicate a dynamic or tense style, while slow and steady shots may belong to a narrative or lyrical style.

[0098] S508: Determine a first video style of the video segment to be cropped according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0099] Specifically, the camera movement style and scene style of each frame can be encoded for subsequent analysis. Different camera movement methods can be represented by numerical values ​​or category labels. For example: translation: 0, rotation: 1, scaling: 2, stillness: 3. According to the color combination, the corresponding label is used. Primary and secondary colors or cluster labels can be used. The camera movement style and scene style of each frame are integrated into a data structure, such as a list or a data frame. The style combination of each frame in the entire video clip is analyzed, and the most common combination is found and summarized. The frequency of occurrence of each camera movement style and scene style combination is counted. Based on the frequency, the camera movement style and scene style combination with the highest frequency is selected. The first video style of the video clip to be cropped is determined based on the camera movement style and scene style combination with the highest frequency.

[0100] In this embodiment, the scene style of each frame of video image is determined by color analysis, and the camera movement style of each frame of video image is determined by the changes between each frame of video image. The first video style can be accurately determined based on the scene style and the camera movement style, thereby ensuring the accuracy of the target segment determination.

[0101] In one embodiment, Figure 6 As shown, determining the first video style of the video segment to be cropped according to the camera movement style between each frame of video image and the scene style of each frame of video image includes:

[0102] S602: Determine the image style of each frame of video image according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0103] S604: Determine the style ratio in the video segment to be cropped according to the image style of each frame of the video image.

[0104] S606: Determine a first video style of the video segment to be cropped according to the image style with the largest style ratio.

[0105] Image style refers to the characteristics and artistic effects of an image in visual expression, including the uniqueness of color, texture, shape, line, composition, etc. Image styles can include: landscape style, sports style, portrait style, etc.

[0106] Specifically, in order to accurately determine the first video style of the video clip to be cropped, each frame of the video image in the video clip to be cropped can be processed separately. For each frame of the video image, the image style of the current frame of the video image can be determined according to its corresponding camera style and scene style. Then, the image style of each frame of the video image in the video clip to be cropped is counted, and the video images with the same image style are classified into one category. Then, the number of video images in each category is determined, and the style ratio of each category of video images is determined according to the number of video images in each category and the total number of video images. According to the style ratio of each category of video images, the image style with the largest style ratio is selected as the first video style.

[0107] In this embodiment, the style ratio is determined by using the image style of each frame of video image, and the first video style is determined by using the style ratio, so that the first video style that best matches the video segment to be cropped can be accurately found, thereby improving the accuracy of determining the target segment.

[0108] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0109] Based on the same inventive concept, the embodiment of the present disclosure also provides a video cropping device for implementing the above-mentioned video cropping method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more video cropping device embodiments provided below can refer to the limitations of the video cropping method above, and will not be repeated here.

[0110] In one embodiment, Figure 7 As shown, a video cropping device 700 is provided, comprising: a video acquisition module 702, a video analysis module 704 and a segment determination module 706, wherein:

[0111] The video acquisition module 702 is used to determine a clipping template and acquire a video segment to be clipped; the video segment to be clipped is a video segment whose video duration is greater than a duration threshold; the duration threshold is determined according to the clipping template;

[0112] The video analysis module 704 is used to perform color analysis and lens language analysis on the video segment to be cropped, and determine a first video style of the video segment to be cropped according to the analysis results of the color analysis and the lens language analysis;

[0113] The segment determination module 706 is used to determine a second video style that matches the editing template, and determine a target segment in the video segment to be cropped according to the relationship between the duration threshold, the first video style, and the second video style.

[0114] In one embodiment of the device, the segment determination module 706 includes:

[0115] The first material determination module is configured to determine, in response to the second video style being the same as the first video style, a first material region in each frame of the video segment to be cropped that has a matching relationship with the first video style.

[0116] The target recognition module is used to determine the style score of each frame of the video image according to the first material area, and determine the target segment in the video segment to be cropped according to the style score of each frame of the video image and the duration threshold.

[0117] The second material determination module is configured to determine, in response to the second video style being different from the first video style, a second material region in each frame of the video segment to be cropped that matches the second video style.

[0118] The target recognition module is further used to determine the style score of each frame of the video image according to the second material area, and determine the target segment in the video segment to be cropped according to the style score of each frame of the video image and the duration threshold.

[0119] In one embodiment of the device, the first material determination module is further used to determine the style score of each frame of video image according to the number of the first material areas; and determine the style score of each frame of video image according to the proportion of the first material areas in the video image.

[0120] In one embodiment of the device, when the first video style includes a motion style; the segment determination module 706 is also used to determine the posture information corresponding to each frame of the video image in the video segment to be cropped in response to the second video style being the same as the first video style; determine the change amplitude of the posture information between each frame of the video image based on the posture information corresponding to each frame of the video image; sort each frame of the video image according to the change amplitude, and filter the sorted video images based on the duration threshold to obtain at least one frame of the video image that meets the duration threshold; determine the target segment in the video segment to be cropped based on at least one frame of the video image that meets the duration threshold.

[0121] In one embodiment of the device, the video analysis module 704 includes:

[0122] A color extraction module is used to extract color from each frame of the video image in the video segment to be cropped to obtain color information;

[0123] A scene style determination module, used to construct a color spectrum of each frame of video image according to the color information, and determine the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum;

[0124] A camera movement style determination module, used to determine the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped;

[0125] The video style determination module is used to determine the first video style of the video segment to be cropped according to the camera movement style between each frame of video image and the scene style of each frame of video image.

[0126] In one embodiment of the device, the video style determination module is further used to determine the image style of each frame of video image based on the camera movement style between each frame of video image and the scene style of each frame of video image; determine the style proportion in the video segment to be cropped based on the image style of each frame of video image; and determine the first video style of the video segment to be cropped based on the image style with the largest style proportion.

[0127] Each module in the above video cropping device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0128] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as clipping templates and video clips to be cut. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video cutting method is implemented.

[0129] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a video cropping method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the computer device housing, or an external keyboard, touch pad or mouse, etc.

[0130] Those skilled in the art will understand that Figure 8 The structure shown in or 9 is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0131] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0132] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0133] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0135] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. A video cropping method, characterized in that: The method comprises: Determine a clipping template and obtain a video segment to be clipped; the video segment to be clipped is a video segment whose video duration is greater than a duration threshold; the duration threshold is determined according to the clipping template; Performing color analysis and lens language analysis on the video segment to be cropped, and determining a first video style of the video segment to be cropped according to the analysis results of the color analysis and the lens language analysis; A second video style matching the editing template is determined, and a target segment in the video segment to be cropped is determined according to the relationship between the duration threshold, the first video style, and the second video style.

2. The method according to claim 1, characterized in that The determining, according to the relationship between the duration threshold, the first video style, and the second video style, a target segment in the video segment to be cropped includes: In response to the second video style being the same as the first video style, determining a first material region in each frame of the video image of the to-be-cropped video segment that has a matching relationship with the first video style; Determining a style score of each frame of the video image according to the first material area, and determining a target segment in the to-be-cropped video segment according to the style score of each frame of the video image and the duration threshold; In response to the second video style being different from the first video style, determining a second material region in each frame of the video segment to be cropped that has a matching relationship with the second video style; According to the second material area, a style score of each frame of the video image is determined, and according to the style score of each frame of the video image and the duration threshold, a target segment in the video segment to be cropped is determined.

3. The method according to claim 2, characterized in that The determining, according to the first material area, a style score of each frame of the video image includes at least one of the following: Determining a style score of each frame of the video image according to the number of the first material regions; The style score of each frame of the video image is determined according to the proportion of the first material area in the video image.

4. The method according to claim 1, characterized in that: When the first video style includes a motion style; determining a target segment in the to-be-cropped video segment according to the relationship between the duration threshold, the first video style, and the second video style includes: In response to the second video style being the same as the first video style, determining gesture information corresponding to each frame of the video image in the video segment to be cropped; Determine the change range of the posture information between each frame of video images according to the posture information corresponding to each frame of video images; Sorting each frame of video image according to the change amplitude, and screening the sorted video images according to the duration threshold to obtain at least one frame of video image that meets the duration threshold; According to at least one frame of video image that meets the duration threshold, a target segment in the video segment to be cropped is determined.

5. The method according to claim 1, characterized in that The performing color analysis and lens language analysis on the video segment to be cropped, and determining the first video style of the video segment to be cropped according to the analysis results of the color analysis and the lens language analysis, comprises: Performing color extraction on each frame of the video image in the video segment to be cropped to obtain color information; Constructing a color spectrum of each frame of video image according to the color information, and determining the scene style of each frame of video image according to the use frequency and color combination of the colors indicated by the color spectrum; Determining the camera movement style between each frame of the video segment to be cropped according to the changes between each frame of the video image in the video segment to be cropped; The first video style of the video segment to be cropped is determined according to the camera movement style between each frame of video image and the scene style of each frame of video image.

6. The method according to claim 4, characterized in that The determining, according to the camera movement style between each frame of video image and the scene style of each frame of video image, the first video style of the video segment to be cropped includes: Determine the image style of each frame of video image according to the camera movement style between each frame of video image and the scene style of each frame of video image; Determining the style ratio in the to-be-cropped video segment according to the image style of each frame of the video image; The first video style of the video segment to be cropped is determined according to the image style with the largest style ratio.

7. A video cutting device, characterized in that: The device comprises: The video acquisition module is used to determine the editing template and acquire the video segment to be cropped; the video segment to be cropped is a video segment whose video duration is greater than a duration threshold; the duration threshold is determined according to the editing template; A video analysis module, configured to perform color analysis and lens language analysis on the video segment to be cropped, and determine a first video style of the video segment to be cropped according to the analysis results of the color analysis and the lens language analysis; The segment determination module is used to determine a second video style that matches the editing template, and determine a target segment in the video segment to be cropped according to the relationship between the duration threshold, the first video style, and the second video style.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images