Video processing method and device

Abstract

The invention discloses a video processing method and device, and belongs to the technical field of image processing. The method comprises the following steps: outputting score information of each video frame in a video to be processed; wherein the score information is determined based on at least one of the following information: shooting field information, shooting mirror information, shooting object information, shooting scene information and audio content information; if the score information of the adjacent N video frames is within a score range, dividing the N video frames into a video clip; wherein N is a positive integer; marking the video clips on a video frame progress control of the to-be-processed video according to different score ranges; wherein one score range corresponds to one piece of mark information, and different pieces of mark information represent different recommendation levels; receiving a first input of a user to the mark information; and outputting a set of video segments of the video to be processed in response to the first input.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a video processing method and device. Background Art

[0002] When users attend events or travel, they usually record a lot of videos and edit or compile them later. However, the recorded videos often have a lot of redundancy, and the highlights may be relatively short, so users need to edit the videos to retain the highlights.

[0003] In the prior art, the user needs to view each frame in the video to find the segment with the wonderful moment and cut it. However, a video may contain multiple wonderful segments, and the user needs to repeat the cutting operation multiple times.

[0004] It can be seen that currently in the post-editing of videos, it is cumbersome for users to filter video clips in the video. Summary of the invention

[0005] The purpose of the embodiment of the present application is to provide a video processing method that can solve the problem of cumbersome operations for users when filtering video clips in a video during post-editing of the video in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a video processing method, the method comprising: outputting score information of each video frame in a video to be processed; wherein the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, then the N video frames are divided into a video segment; wherein N is a positive integer; according to different score ranges, the video segment is marked on a video frame progress control of the video to be processed; wherein one score range corresponds to one marking information, and different marking information represents different recommendation levels; receiving a first input of the marking information by a user; and in response to the first input, outputting a collection of the video segments of the video to be processed.

[0007] In a second aspect, an embodiment of the present application provides a video processing device, comprising: a first output module, used to output score information of each video frame in a video to be processed; wherein the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; a division module, used to divide the N adjacent video frames into a video segment if the score information of the N adjacent video frames is within a score range; wherein N is a positive integer; a marking module, used to mark the video segment on the video frame progress control of the video to be processed according to different score ranges; wherein one score range corresponds to one marking information, and different marking information represents different recommendation levels; a first receiving module, used to receive a first input of the marking information from a user; and a second output module, used to output a collection of the video segments of the video to be processed in response to the first input.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, firstly, the score information of each video frame in the video to be processed is output; wherein the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment; then, according to different score ranges, the video segments are marked on the video frame progress control of the video to be processed; wherein, one score range corresponds to one marking information, and different marking information indicates different recommendation levels; finally, when the user's first input of the marking information is received, a collection of video segments of the video to be processed is output. It can be seen that in an embodiment of the present application, the video can be automatically divided into video segments, and the recommendation levels of different video segments can be marked. The user only needs to make a simple input on this basis to quickly filter out the segments that meet his needs, so that the user does not need to watch frame by frame, crop and save, and then splice the cropped and saved segments together, thereby simplifying the user's operation and improving the efficiency of video processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flowchart of a video processing method in some embodiments of the present application;

[0014] Figure 2 is one of the display schematic diagrams of the electronic device in some embodiments of the present application;

[0015] Figure 3 This is a second display schematic diagram of an electronic device in some embodiments of the present application;

[0016] Figure 4 This is a third display schematic diagram of an electronic device in some embodiments of the present application;

[0017] Figure 5 This is a fourth display schematic diagram of an electronic device in some embodiments of the present application;

[0018] Figure 6 This is a fifth display schematic diagram of an electronic device in some embodiments of the present application;

[0019] Figure 7 This is the sixth display schematic diagram of the electronic device in some embodiments of the present application;

[0020] Figure 8 This is the seventh display schematic diagram of the electronic device of some embodiments of the present application;

[0021] Fig. 9 is a block diagram of a video processing device according to some embodiments of the present application;

[0022] Fig.10Schematic diagram of the hardware structure of the electronic device of some embodiments of the present application;

[0023] Fig.11 It is a schematic diagram of the hardware structure of an electronic device of some embodiments of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings of the embodiments of the present application to clearly describe the technical solutions of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present application belong to the scope of protection of this application.

[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally a class, and the number of objects is not limited. For example, the target object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0026] The video processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0027] like Figure 1 FIG. 1 is a flowchart of a video processing method according to an embodiment of the present application. The method is applied to an electronic device as an example. The method includes:

[0028] Step 110: output score information of each video frame in the video to be processed; wherein the score information is determined based on at least one of the following information: shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information.

[0029] In this embodiment, the video to be processed can be any video to be subjected to video segment extraction. The score information can be used to characterize the wonderfulness of the video frame. Figure 2The play page shown includes a play area 201 and a video frame area 202. The video frame area 202 displays a cursor control 203, and the user can locate the currently played video frame through the cursor control 203. On this basis, a highlight mark control 204 can be displayed on the play page. After the user clicks the highlight mark control 204, the score information of each frame in the video to be processed is output, so that the video segment is marked in the video frame area 202 based on the score information of each frame.

[0030] In this embodiment, the shooting field information may refer to the shooting field of the video frame. For example, the shooting field may include but is not limited to the field of talent, beauty field, life field, humanities field, food field, editing field, parenting field, etc. Different shooting fields may correspond to different scores. The shooting movement information may refer to the shooting movement used when shooting the video frame. For example, the shooting movement may include but is not limited to push, pull, shake, move, etc. Different shooting movements may correspond to different scores. The shooting object information may be used to indicate the shooting object of the video frame. For example, the shooting object may include but is not limited to people, animals, scenery, etc. Different shooting objects may correspond to different scores. The shooting scene information may be used to indicate the shooting scene of the video frame. For example, the shooting scene may include but is not limited to indoor scenes, outdoor scenes, static scenes, sports scenes, etc. Different shooting scenes may correspond to different scores. The audio content information may be used to indicate the audio content corresponding to the shooting of the video frame. For example, the audio content may include but is not limited to voice, background noise, etc. Different audio content may correspond to different scores. The score information may be determined based on one or more of the above information. In practice, the score information may be obtained by weighted summing the scores of each information.

[0031] Optionally, the model can output the score information of each frame in the video to be processed. In practice, the model can be trained in advance for videos of different scenes, so that the model can score each video frame of the video to be processed and output the score information based on the definition of the excitement of different scenes and the relevant artificial intelligence (AI) algorithm.

[0032] In the training process, a large number of videos are prepared for a certain scene, and the reference scores of the excitement of each frame in the large number of videos are annotated in advance. The videos with annotated reference scores are learned using a deep network, the video features in the scene are extracted, and the actual excitement scores are output. The network parameters are obtained by combining the annotated reference scores of the excitement. In the application stage, the model calculates the video features of the video to be processed based on the network parameters and gives a score.

[0033] Among them, the video features at least include shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information.

[0034] For example, based on video features, the moments of user's main attention or interest can be summarized. These moments often present specific events that can attract viewers, so the video frames at these moments have higher scores for excitement. More specifically, these moments include action moments such as characters jumping, characters making a heart shape, characters clinking glasses, etc., and these moments are also good scenery moments, and these moments are also football and basketball goal moments.

[0035] Step 120: If the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment.

[0036] In this step, in the video to be processed, if the score information of N consecutive frames is within a score range, the N consecutive frames are regarded as a video segment, based on which, the video to be processed can be divided into multiple video segments, where N is a positive integer.

[0037] For example, the maximum range of score information that can be obtained is 0 to 100. Based on this maximum range, four score ranges can be divided, namely 0 to 25, 26 to 50, 51 to 75, and 75 to 100. If the score information of N consecutive frames is within the range of 0 to 25, these N frames are regarded as a video segment.

[0038] Step 130: Mark the video clips on the video frame progress control of the video to be processed according to different score ranges; wherein one score range corresponds to one marking information, and different marking information indicates different recommendation levels.

[0039] In this step, all video clips in the same score range are marked with the same marking information. Based on this marking method, users can quickly filter out all video clips in the same score range.

[0040] like Figure 3 As shown, a video frame progress control is displayed in the video frame area, on which all video clips within one score range are marked with black marks 301 , and all video clips within another score range are marked with white marks 302 .

[0041] Optionally, a correspondence between a score range and the marking information may be preset, and the correspondence may be customized by a user, so that the user can quickly identify a video segment with a higher score according to the marking information.

[0042] For example, the video clip marked with green has a higher score and is the video clip that the user most wants to keep; the video clip marked with yellow has the second highest score and is the video clip that the user second wants to keep; the video clip marked with red has a lower score and is the video clip that the user second least wants to keep; the video clip marked with gray has the lowest score and is the video clip that the user least wants to keep.

[0043] Step 140: Receive a first input of marking information from a user.

[0044] Here, the first input is used to select any marking information. The first input can be a touch input, or a voice command, or a specific gesture input by the user, or other feasible input, which can be determined according to actual use requirements, and is not limited in the embodiments of the present application. The specific gesture in the embodiments of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture. The click input in the embodiments of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input.

[0045] Step 150: In response to the first input, output a collection of video segments of the video to be processed.

[0046] Here, in response to the first input, video segments with the marking information selected by the user may be extracted, and the extracted video segments may be spliced ​​to obtain a collection of video segments.

[0047] For example, when marking clips with different score ranges, if the marking information selected by the user is the marking information corresponding to the video clip with the highest score range, then by first inputting the marking information, all clips with the highest scores can be retained with one click to generate a collection of video clips.

[0048] In addition, when marking segments under different score ranges, the user can adjust the marking information of a segment through input, so that the final marking information is not only automatically marked, but also marked based on user preferences.

[0049] In an embodiment of the present application, firstly, the score information of each video frame in the video to be processed is output; wherein the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment; then, according to different score ranges, the video segments are marked on the video frame progress control of the video to be processed; wherein, one score range corresponds to one marking information, and different marking information indicates different recommendation levels; finally, when the user's first input of the marking information is received, a collection of video segments of the video to be processed is output. It can be seen that in an embodiment of the present application, the video can be automatically divided into video segments, and the recommendation levels of different video segments can be marked. The user only needs to make a simple input on this basis to quickly filter out the segments that meet his needs, so that the user does not need to watch frame by frame, crop and save, and then splice the cropped and saved segments together, thereby simplifying the user's operation and improving the efficiency of video processing.

[0050] In the process of the video processing method of another embodiment of the present application, the step of marking different information on each segment includes:

[0051] Sub-step A1: Mark the video clips in the same score range with the same background color.

[0052] Optionally, in one implementation, on the video frame progress control, different colors are used to fill corresponding video clips.

[0053] For example, video clips with higher scores are filled with green, video clips with medium scores are filled with yellow, video clips with lower scores are filled with red, and video clips with the lowest score are filled with gray.

[0054] Optionally, on the video frame progress control, the transparency of the filled color is high, thereby ensuring that the background color does not cause serious interference to the display of the video frame.

[0055] In this embodiment, different background colors, such as red, yellow and green, will be presented on the video frame progress control. Users can quickly lock automatically recommended video clips or non-recommended video clips by color, so as to help users quickly lock multiple video clips in the video by automatically filtering related clips.

[0056] In the process of the video processing method of another embodiment of the present application, after step 130, the method further includes:

[0057] Step B1: receiving a second input of first marking information of a first video segment in the video segments.

[0058] In some embodiments of the present application, the second input is used to change the first tag information to replace the first tag information with the second tag information, so as to achieve the effect of adjusting the recommendation priority of the first video clip, so that the user can uniformly manage all clips under the same tag information. Exemplarily, the second input includes but is not limited to: the user's touch input of the on-screen controls or the screen area through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input, which can be determined according to actual use requirements and is not limited in the embodiments of the present invention. The specific gesture in the embodiments of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiments of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input.

[0059] like Figure 4 As shown, the second input may be that the user long presses a certain segment on the video frame progress control as the first video segment in this embodiment, thereby displaying a list of options 401 for various marking information, such as green-level options, yellow-level options, red-level options, and gray-level options. If the current first segment is marked as red, the user may click on the yellow-level option or the green-level option. Among them, options of different color levels represent different color markings. The description of level is used here to express that the color represents the recommendation priority.

[0060] In the case where the options of each tagged information are displayed, the options of each tagged information may be displayed in sequence according to the order of the recommendation priority indicated by each tagged information from high to low.

[0061] Step B2: In response to the second input, the first tag information of the first video segment is replaced with the second tag information associated with the second input.

[0062] The first marking information is different from the second marking information.

[0063] In this step, the first video segment is no longer marked as the first marking information, but is marked as the second marking information.

[0064] Application scenarios include: based on the evaluation of the first video clip's wonderfulness, the model considers the first video clip to be more wonderful and recommends it to the user as one of the first batch of clips. However, after watching the first video clip, the user believes that the first video clip does not express his or her shooting concept well, and thus manually deletes the first video clip from the first batch of clips and marks it as an alternative clip. Here, the first choice has a higher priority than the alternative. Furthermore, if the user believes that the above-mentioned alternative clip is still good in subsequent operations, it can be adjusted to the preferred clip at any time without the user having to re-crop or enter a complicated process.

[0065] In this embodiment, after the model marks the video clips in the video to be processed, the user can change the marking information of any video clip, so that automatic marking and manual marking assist each other, so that the final synthesized collection of video clips can not only display the wonderful content shot by the user, but also make the display close to or the same as the user's expected effect.

[0066] In the process of the video processing method of another embodiment of the present application, after step 130, the method further includes:

[0067] Step C1: When a second video segment among the video segments is displayed, receiving a third input to a first control and a second control on the second video segment.

[0068] In some embodiments of the present application, the third input is used for the user to operate the second video clip, so as to crop the second video clip into the third video clip in the initial state of the second video clip automatically divided by the model. Exemplarily, the third input includes but is not limited to: the user's touch input to the controls on the screen or the screen area through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input, which can be determined according to actual use requirements and is not limited in the embodiments of the present invention. The specific gesture in the embodiments of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiments of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input.

[0069] like Figure 5As shown, the user clicks on the second video segment on the video frame progress control, and the initial start position of the second video segment is displayed on the video frame progress control, and the first control 501 is displayed at the initial start position. If the duration of the second video segment just meets the width of the screen to display the initial end position of the second video segment at the same time, the initial end position of the second video segment is displayed, and the second control 502 is displayed at the initial end position. The third input may be that the user presses and holds the first control 501 and drags on the second segment, and / or the user presses and holds the second control 502 and drags on the second segment.

[0070] Step C2: In response to the third input, a third video clip is obtained according to the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video clip.

[0071] The marking information of the second video segment is used to mark the third video segment.

[0072] In this step, the second video segment is cropped into a third video segment via the third input, while the marking information of the second video segment is retained on the third video segment.

[0073] Optionally, based on user settings, the video segments other than the third video segment in the second video segment may be marked as information of a lower level of recommendation or other recommendation levels, or the video segments other than the third video segment in the second video segment may not be marked. When there are unmarked video segments in the video to be processed, it may be considered that the unmarked video segments have the lowest recommendation priority.

[0074] Optionally, if there is a clear audio boundary in the second video clip, for example, in a concert video, some parts have people singing and some parts have no singing sound, the audio feature information of the second video clip can be displayed to prompt the user to keep the clip with clearer audio.

[0075] like Figure 5 As shown, above the second video clip, audio feature information of the second video clip is displayed, and the audio feature information indicates the beginning and the end of the sound. Furthermore, the first control and the second control of the second video clip can be displayed at the beginning and the end of the sound, respectively, so that the user can directly save the clips cropped by the current first control and the second control.

[0076] In this embodiment, on the basis of automatically dividing the video clips, the user can also manually adjust the duration of each video clip, so that the video clips retained at the end can be used in the production of the video clip collection, satisfying the user's idea of ​​producing the video clip collection. In addition, this embodiment uses the audio as a reference to give cutting suggestions, thereby achieving accurate matching of the audio, and no longer requiring the video clips of this application to be selected and positioned on the image, so that the retained video clips can reflect more shooting details.

[0077] In the process of the video processing method of another embodiment of the present application, step 140 includes:

[0078] Sub-step D1: receiving a first input of the user for the third marking information in the marking information.

[0079] In some embodiments of the present application, the above-mentioned first input is used for the user to operate at least one tag information to trigger the unified output of all video clips under the at least one tag information. Exemplarily, the above-mentioned first input includes but is not limited to: the user's touch input to the on-screen controls or the screen area through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input, which can be determined according to actual use requirements and is not limited by the embodiments of the present invention. The specific gesture in the embodiments of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiments of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input.

[0080] For example, when a playback page is displayed, the user clicks on the edit option provided by the page. After the user clicks on the edit control, the editing page of the video to be processed is entered, such as Figure 6 As shown, a control 601 with a highlight mark is displayed on the editing page, and the user clicks the control 601 with a highlight mark, as shown in FIG. Figure 7 As shown, controls for triggering the display of video clips under various marking information appear, such as controls for triggering the display of video clips under various color level marks, specifically control 701 for displaying green-level clips. The user clicks control 701 for the green-level clip and also clicks other controls for displaying clips of a certain color level at the same time.

[0081] Optionally, in some implementations, in the video editing page, all video clips in the lowest score range may be automatically removed. For example, if the video clips in the lowest score range are marked in gray, all video clips marked in gray are removed; or if the video clips in the lowest score range are not marked, all video clips not marked are removed.

[0082] Further, step 150 includes:

[0083] Sub-step D2: In response to the first input, output all video segments marked by the third marking information in the video to be processed to obtain a collection of video segments.

[0084] For example, if at least one piece of information associated with the first input is green level, then only all video clips marked as green level are displayed on the video frame progress control. Figure 7 As shown, an indicator line 702 is displayed at the start position of each segment to facilitate the user to quickly locate it.

[0085] For example, if at least one information associated with the first input is green level and yellow level, then only all video clips marked as green level and yellow level are displayed on the video frame progress control. At the same time, an indicator line is displayed at the starting position of each video clip to facilitate user quick positioning.

[0086] In one implementation, Figure 7 As shown, in the video editing page, when the cursor 703 on the video frame progress control is located at a certain position, the highlight mark control 704 acts as a display control, and its background is filled with the mark color of the video clip at that position to indicate the user the mark information of the current video clip.

[0087] In addition, on the video editing page, users can still trim video clips and change the tag information of video clips at any time. Figure 8 As shown, the user long presses a video clip to trigger a list of options showing yellow, green, red and gray levels, and the user can change the marking color of the video clip.

[0088] Finally, the first input also includes: Figure 7 As shown, the user clicks the save control 705, so that only all video clips displayed on the current video frame progress control are output as a collection of synthesized video clips.

[0089] For example, on the video frame progress control, only all video clips marked as green are displayed, and the output is a collection of all video clips marked as green.

[0090] For example, on the video frame progress control, only all video clips marked as green and yellow are displayed, and the output includes a collection of all video clips marked as green and all video clips marked as yellow.

[0091] In more embodiments, during the process of playing a collection of video clips, the marking information of each video clip therein may also be displayed in the video frame area of ​​the playing page.

[0092] In this embodiment, the user can output a collection of video clips consisting of all video clips under one or more marking information based on one or more marking information, thereby realizing one-click output of all video clips without the user having to go through the tedious steps of first saving the video clips and then synthesizing the video clips.

[0093] In the process of the video processing method of another embodiment of the present application, before step 110, the method further includes:

[0094] Step E1: Receive a fourth input of at least two videos.

[0095] Step E2: In response to the fourth input, at least two videos are spliced ​​to obtain a video to be processed.

[0096] In some embodiments of the present application, the fourth input is used for the user to select at least two videos, so as to use the video spliced ​​from at least two videos as the video to be processed. The fourth input may be a fourth operation. Exemplarily, the fourth input includes but is not limited to: the user's touch input to the controls on the screen or the screen area through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which may be determined according to actual use requirements, and are not limited in the embodiments of the present invention. The specific gesture in the embodiments of the present application may be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiments of the present application may be a single-click input, a double-click input, or any number of click inputs, etc., and may also be a long press input or a short press input.

[0097] For example, the fourth input may be that in the album, a list of video objects is displayed, the user clicks on the selected option, then clicks on multiple video objects in sequence, and then triggers the display of the play option by long pressing, and the user clicks on the play option to enter the video play page, in which the multiple video objects selected by the user are sequentially spliced ​​into a long video in the order selected by the user, i.e., the video to be processed. Figure 8 As described above, the video frame progress control is composed of multiple segments, such as the first segment 801 and the second segment 802, and a segment can represent a video object.

[0098] In more embodiments, the user may also locate the highlights in each video one by one, and then combine the highlights of multiple videos into a collection.

[0099] In this embodiment, in terms of interactive mode, multiple videos can be imported into the editing project at one time, and then the highlight clips can be located in the spliced ​​long video. Among them, by importing multiple videos at one time and then locating the highlight clips, the user can directly evaluate whether the output of the film based on the located highlight clips is reasonable, and adjust the highlight clips at any time according to whether it is reasonable or not. This has a high fault tolerance rate. The user can adjust the length of the clip and the tag information of the clip at any time, which is convenient for the user to screen the highlight clips by comparing multiple video objects until the user is satisfied and saves the film.

[0100] In summary, firstly, the score information of each video frame in the video to be processed is output; wherein the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment; then, according to different score ranges, the video segments are marked on the video frame progress control of the video to be processed; wherein, one score range corresponds to one marking information, and different marking information indicates different recommendation levels; finally, upon receiving the first input of the marking information by the user, a collection of video segments of the video to be processed is output. It can be seen that in the embodiment of the present application, the video can be automatically divided into video segments, and the recommendation levels of different video segments can be marked. The user only needs to make a simple input on this basis to quickly filter out the segments that meet his or her needs, so that the user does not need to watch frame by frame, crop and save, and then splice the cropped and saved segments together, thereby simplifying the user's operation and improving the efficiency of video processing.

[0101] The video processing method provided in the embodiment of the present application can be executed by a video processing device. In the embodiment of the present application, the video processing device provided in the embodiment of the present application is described by taking the video processing method executed by the video processing device as an example.

[0102] Fig. 9 A block diagram of a video processing device according to an embodiment of the present application is shown, the device comprising:

[0103] A first output module 10 is used to output score information of each video frame in the video to be processed; wherein the score information is determined based on at least one of the following information: shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information;

[0104] A division module 20, configured to divide N adjacent video frames into a video segment if the score information of the N adjacent video frames is within a score range; wherein N is a positive integer;

[0105] A marking module 30, configured to mark the video clip on the video frame progress control of the video to be processed according to different score ranges; wherein one score range corresponds to one marking information, and different marking information indicates different recommendation levels;

[0106] A first receiving module 40 is used to receive a first input of the marking information by a user;

[0107] The second output module 50 is used to output the collection of video segments of the video to be processed in response to the first input.

[0108] In an embodiment of the present application, firstly, the score information of each video frame in the video to be processed is output; wherein, the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment; then, according to different score ranges, the video segments are marked on the video frame progress control of the video to be processed; wherein, one score range corresponds to one marking information, and different marking information indicates different recommendation levels; finally, when the user's first input of the marking information is received, a collection of video segments of the video to be processed is output. It can be seen that in an embodiment of the present application, the video can be automatically divided into video segments, and the recommendation levels of different video segments can be marked. The user only needs to make a simple input on this basis to quickly filter out the segments that meet his needs, so that the user does not need to watch frame by frame, crop and save, and then splice the cropped and saved segments together, thereby simplifying the user's operation and improving the efficiency of video processing.

[0109] Optionally, the marking module 30 includes: a marking unit, which is used to mark the video clips in the same score range with the same background color. Thus, the user can quickly lock the video clip by color, so as to help the user quickly lock multiple clips in the video by automatically screening related clips.

[0110] Optionally, the device further includes: a second receiving module for receiving a second input of first tag information of a first video segment in the video segments; a replacement module for replacing the first tag information of the first video segment with second tag information associated with the second input in response to the second input; wherein the first tag information is different from the second tag information. In this embodiment, after the model tags the video segments in the video to be processed, the user can change the tag of any video segment, so that automatic tagging and manual tagging assist each other, so that the final synthesized collection of video segments can not only display the wonderful content shot by the user, but also make the display close to or the same as the user's expected effect.

[0111] Optionally, the device further includes: a third receiving module, for receiving a third input of the first control and the second control on the second video clip when displaying the second video clip in the video clip; a processing module, for obtaining a third video clip in response to the third input according to the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video clip; wherein the marking information of the second video clip is used to mark the third video clip. By taking the audio as a reference and giving the cropping advice, the audio is accurately matched, and the video clip of the present application is no longer selected and positioned on the image, so that the retained video clip can reflect more shooting details.

[0112] Optionally, the first receiving module 10 includes: a receiving unit for receiving a first input of a user for third tag information in the tag information; and the second output module includes: an output unit for outputting all video clips marked by the third tag information in the video to be processed in response to the first input, so as to obtain a collection of the video clips. Thus, the user can output a collection of video clips consisting of all video clips under one or more tag information based on one or more information of the marked video clips, thereby realizing a one-key output of all video clips without the tedious operation steps of the user first saving the video clips and then synthesizing the video clips.

[0113] The device in the application embodiment can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the present application embodiment is not specifically limited.

[0114] The device of the embodiment of the present application may be a device having an action system. The action system may be an Android action system, an iOS action system, or other possible action systems, which are not specifically limited in the embodiment of the present application.

[0115] The device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0116] Alternatively, if Fig.10 As shown, an embodiment of the present application also provides an electronic device 100, including a processor 101, a memory 102, and a program or instruction stored in the memory 102 and executable on the processor 101. When the program or instruction is executed by the processor 101, each step of any of the above-mentioned video processing method embodiments is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0117] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0118] Fig.11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0119] The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, a processor 1010, a camera 1011 and other components.

[0120] Those skilled in the art will appreciate that the electronic device 1000 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1010 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0121] The processor 1010 is used to output the score information of each video frame in the video to be processed; wherein the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment; wherein N is a positive integer; according to different score ranges, the video segments are marked on the video frame progress control of the video to be processed; wherein one score range corresponds to one marking information, and different marking information indicates different recommendation levels; receiving a first input of the marking information from a user; and in response to the first input, outputting a collection of the video segments of the video to be processed. .

[0122] In an embodiment of the present application, firstly, the score information of each video frame in the video to be processed is output; wherein the score information is determined based on at least one of the following information: shooting field information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment; then, according to different score ranges, the video segments are marked on the video frame progress control of the video to be processed; wherein, one score range corresponds to one marking information, and different marking information indicates different recommendation levels; finally, when the user's first input of the marking information is received, a collection of video segments of the video to be processed is output. It can be seen that in an embodiment of the present application, the video can be automatically divided into video segments, and the recommendation levels of different video segments can be marked. The user only needs to make a simple input on this basis to quickly filter out the segments that meet his needs, so that the user does not need to watch frame by frame, crop and save, and then splice the cropped and saved segments together, thereby simplifying the user's operation and improving the efficiency of video processing.

[0123] Optionally, the processor 1010 is further configured to mark the video clips in the same score range with the same background color. Thus, the user can quickly lock the video clip by color, so as to help the user quickly lock multiple clips in the video by automatically screening related clips.

[0124] Optionally, the user input unit 1007 is used to receive a second input of first tag information of a first video segment in the video segment; the processor 1010 is also used to replace the first tag information of the first video segment with second tag information associated with the second input in response to the second input; wherein the first tag information is different from the second tag information. In this embodiment, after the model tags the video segments in the video to be processed, the user can change the tag of any video segment, so that the automatic tagging and manual tagging assist each other, so that the final synthesized collection of video segments can not only show the wonderful content shot by the user, but also make the display close to or the same as the user's expected effect.

[0125] Optionally, the user input unit 1007 is further used to receive a third input of the first control and the second control on the second video clip when displaying the second video clip in the video clip; wherein the various clips include the second clip; the processor 1010 is further used to obtain the third video clip in response to the third input according to the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video clip; wherein the marking information of the second video clip is used to mark the third video clip. By taking the audio as a reference and giving the cropping advice, the accurate matching of the audio is achieved, and the video clip of the present application is no longer selected and positioned on the image, so that the retained video clip can reflect more shooting details.

[0126] Optionally, the user input unit 1007 is further used to receive a first input from a user regarding third tag information in the tag information; the processor 1010 is further used to output all video segments marked by the third tag information in the video to be processed in response to the first input, and obtain a collection of the video segments. Thus, the user can output a collection of video segments consisting of all video segments under one or more tag information based on one or more information of marking the video segments, thereby realizing a one-key output method for all video segments, without the user having to first save the video segments and then synthesize the video segments in a cumbersome operation step.

[0127] It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes the image data of the static picture or video image obtained by the image capture device (such as a camera) in the video image capture mode or the image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1007 includes a touch panel 10071 and at least one of other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and an action rod, which will not be repeated here. The memory 1009 can be used to store software programs and various data, including but not limited to applications and action systems. The processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the action system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1010.

[0128] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0129] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1010.

[0130] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned video processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0131] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0132] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned video processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0133] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0134] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned video processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0135] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0137] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video processing method, It is characterized in that The method comprises: Outputting score information of each video frame in the video to be processed; wherein the score information is determined based on at least one of the following information: shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; If the score information of N adjacent video frames is within a score range, the N video frames are divided into a video segment; wherein N is a positive integer; According to different score ranges, marking the video clip on the video frame progress control of the video to be processed; wherein one score range corresponds to one marking information, and different marking information indicates different recommendation levels; receiving a first input of the marking information by a user; In response to the first input, a collection of the video segments of the video to be processed is output.

2. The method according to claim 1, It is characterized in that The step of marking the video clips includes: The video clips in the same score range are marked with the same background color.

3. The method according to claim 1, It is characterized in that After marking the video clip on the video frame progress control of the video to be processed according to different score ranges, the method further includes: receiving a second input of first tag information of a first video segment in the video segments; In response to the second input, replacing the first tag information of the first video segment with second tag information associated with the second input; The first marking information is different from the second marking information.

4. The method according to claim 1, It is characterized in that After marking the video clip on the video frame progress control of the video to be processed according to different score ranges, the method further includes: In a case where a second video segment in the video segments is displayed, receiving a third input to the first control and the second control on the second video segment; In response to the third input, a third video clip is obtained according to the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video clip; The marking information of the second video segment is used to mark the third video segment.

5. The method according to claim 1, It is characterized in that The receiving a first input of the marking information by the user includes: receiving a first input of a user on third marking information in the marking information; The step of outputting, in response to the first input, a collection of video segments of the video to be processed, comprises: In response to the first input, all video segments marked by the third marking information in the video to be processed are output to obtain a collection of the video segments.

6. A video processing device, It is characterized in that The device comprises: A first output module is used to output score information of each video frame in the video to be processed; wherein the score information is determined based on at least one of the following information: shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; A division module, configured to divide N adjacent video frames into a video segment if the score information of the N adjacent video frames is within a score range; wherein N is a positive integer; a marking module, used for marking the video clip on the video frame progress control of the video to be processed according to different score ranges; wherein one score range corresponds to one marking information, and different marking information indicates different recommendation levels; A first receiving module, used for receiving a first input of the marking information by a user; The second output module is used to output the collection of video segments of the video to be processed in response to the first input.

7. The device according to claim 6, It is characterized in that The marking module comprises: The marking unit is used to mark the video clips in the same score range with the same background color.

8. The device according to claim 6, It is characterized in that The device also includes: A second receiving module, configured to receive a second input of first marking information of a first video segment in the video segments; a replacement module, configured to replace the first tag information of the first video segment with second tag information associated with the second input in response to the second input; The first marking information is different from the second marking information.

9. The device according to claim 6, It is characterized in that The device also includes: A third receiving module, configured to receive a third input to the first control and the second control on the second video segment when displaying the second video segment in the video segment; a processing module, configured to obtain, in response to the third input, a third video clip according to the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with audio feature information of the second video clip; The marking information of the second video segment is used to mark the third video segment.

10. The device according to claim 6, It is characterized in that The first receiving module includes: A receiving unit, configured to receive a first input of a user on third marking information in the marking information; The second output module comprises: An output unit is used to output all video segments marked by the third marking information in the video to be processed in response to the first input, so as to obtain a collection of the video segments.

Images