Video processing method and apparatus

Abstract

The application discloses a video processing method and device, and belongs to the technical field of image processing. The method comprises the following steps: outputting score information of each video frame in a to-be-processed video; wherein the score information is determined based on at least one of the following information: shooting field information, shooting lens information, shooting object information, shooting scene information and audio content information; if score information of adjacent N video frames is in a score range, the N video frames are divided into a video segment; wherein N is a positive integer; according to different score ranges, a video frame progress control of the to-be-processed video is marked with the video segment; wherein one score range corresponds to one marking information, and different marking information represents different recommendation levels; receiving a first input of a user on the marking information; and in response to the first input, outputting a collection of the video segment of the to-be-processed video.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a video processing method and apparatus. Background Technology

[0002] When users participate in activities or travel, they often record a large number of videos, which are then edited or compiled into compilations later. However, the recorded videos often contain a lot of redundancy, and the highlights may be relatively short. Therefore, users need to edit the videos to retain the best parts.

[0003] In existing technology, users need to examine every frame of the video to find the most exciting moments for trimming. Since a video may contain multiple exciting segments, users need to repeat the trimming process multiple times.

[0004] It is evident that currently, in the post-editing of videos, the process of selecting video clips is cumbersome for users. Summary of the Invention

[0005] The purpose of this application is to provide a video processing method that can solve the problem of cumbersome operation for users when selecting video segments in the post-editing of videos in the prior art.

[0006] In a first aspect, embodiments of this application provide a video processing method, the method comprising: outputting score information of each video frame in a video to be processed; wherein the score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames is within a score range, then dividing the N video frames into a video segment; wherein N is a positive integer; marking the video segment on a video frame progress control of the video to be processed according to different score ranges; wherein one score range corresponds to one marking information, and different marking information represents different recommendation levels; receiving a first input from a user on the marking information; and responding to the first input, outputting a collection of the video segments of the video to be processed.

[0007] Secondly, embodiments of this application provide a video processing apparatus, the apparatus comprising: a first output module, configured to output score information of each video frame in a video to be processed; wherein the score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information; a division module, configured to divide the N video frames into a video segment if the score information of N adjacent video frames falls within a score range; wherein N is a positive integer; a marking module, configured to mark the video segment on a video frame progress control of the video to be processed according to different score ranges; wherein one score range corresponds to one marking information, and different marking information represents different recommendation levels; a first receiving module, configured to receive a first input from a user regarding the marking information; and a second output module, configured to output a collection of the video segments of the video to be processed in response to the first input.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In the embodiments of this application, the score information of each video frame in the video to be processed is first output. The score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information. If the score information of N adjacent video frames falls within a score range, the N video frames are divided into a video segment. Then, according to the different score ranges, the video segment is marked on the video frame progress control of the video to be processed. Each score range corresponds to one mark, and different marks represent different recommendation levels. Finally, upon receiving the user's first input of the mark information, a collection of video segments from the video to be processed is output. Therefore, in the embodiments of this application, the video can be automatically divided into video segments, and different video segments can be marked with recommendation levels. Users only need to perform simple input to quickly filter out segments that meet their needs, eliminating the need for users to view, crop, save, and then stitch the cropped segments together frame by frame, thus simplifying user operations and improving video processing efficiency. Attached Figure Description

[0013] Figure 1 This is a flowchart of a video processing method according to some embodiments of this application;

[0014] Figure 2 This is one of the schematic diagrams of an electronic device according to some embodiments of this application;

[0015] Figure 3 This is a second schematic diagram of an electronic device according to some embodiments of this application;

[0016] Figure 4 This is the third of several schematic diagrams showing electronic devices according to embodiments of this application;

[0017] Figure 5 This is a fourth schematic diagram of an electronic device according to some embodiments of this application;

[0018] Figure 6 This is the fifth of several schematic diagrams showing electronic devices according to embodiments of this application;

[0019] Figure 7 This is the sixth of several schematic diagrams showing electronic devices according to embodiments of this application;

[0020] Figure 8 This is the seventh of several schematic diagrams showing electronic devices according to embodiments of this application;

[0021] Figure 9 This is a block diagram of a video processing apparatus according to some embodiments of this application;

[0022] Figure 10Hardware structure diagrams of electronic devices according to some embodiments of this application;

[0023] Figure 11 These are schematic diagrams of the hardware structure of electronic devices according to some embodiments of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, there may be one or more target objects. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] The video processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0027] like Figure 1 The diagram illustrates a flowchart of a video processing method according to an embodiment of this application. Taking the application of this method to an electronic device as an example, the method includes:

[0028] Step 110: Output the score information of each video frame in the video to be processed; wherein, the score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information.

[0029] In this embodiment, the video to be processed can be any video from which video segments are to be extracted. Scoring information can be used to characterize the quality of video frames. Application scenarios include, for example, in... Figure 2The playback page shown includes a playback area 201 and a video frame area 202. The video frame area 202 displays a cursor control 203, which allows the user to locate the currently playing video frame. Furthermore, a highlight marker control 204 can be displayed on the playback page. When the user clicks the highlight marker control 204, the score information for each frame of the video to be processed is output, thus marking the video segment in the video frame area 202 based on the score information of each frame.

[0030] In this embodiment, the shooting domain information can refer to the shooting domain of the video frame. For example, the shooting domain can include, but is not limited to, talent domain, beauty domain, lifestyle domain, humanities domain, food domain, editing domain, parenting domain, etc. Different shooting domains can correspond to different scores. The shooting camera movement information can refer to the shooting camera movement used when shooting the video frame. For example, shooting camera movement can include, but is not limited to, push, pull, pan, tilt, etc. Different shooting camera movements can correspond to different scores. The shooting object information can be used to indicate the shooting object of the video frame. For example, the shooting object can include, but is not limited to, people, animals, scenery, etc. Different shooting objects can correspond to different scores. The shooting scene information can be used to indicate the shooting scene of the video frame. For example, the shooting scene can include, but is not limited to, indoor scenes, outdoor scenes, static scenes, moving scenes, etc. Different shooting scenes can correspond to different scores. The audio content information can be used to indicate the audio content corresponding to the shooting of the video frame. For example, the audio content can include, but is not limited to, speech, background noise, etc. Different audio content can correspond to different scores. The score information can be determined based on one or more of the above information. In practice, the score information can be obtained by weighted summation of the scores of each piece of information.

[0031] Optionally, the model can output the score information for each frame of the video to be processed. In practice, the model can be pre-trained for videos of different scenes, so that the model can score each video frame of the video to be processed and output the score information according to the definition of the excitement level of different scenes using relevant artificial intelligence (AI) algorithms.

[0032] During training, a large number of videos are prepared for a specific scene. Reference scores are pre-labeled for the level of excitement of each frame in these videos. A deep network learns from these videos with labeled reference scores, extracting video features from the scene and outputting the actual excitement score. These scores are then combined with the labeled reference scores to obtain the network parameters. In the application phase, the model calculates the video features of the video to be processed based on the network parameters and provides a score.

[0033] Among them, video features include at least the shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information.

[0034] For example, based on video features, we can summarize the moments when users' main attention or interest is focused. These moments often present specific events that attract viewers, and therefore, the video frames at these moments receive higher scores for their engagingness. More specifically, these moments include action moments such as people jumping, people making heart shapes with their hands, and people clinking glasses; moments of scenic beauty with good camera movement; and moments of scoring goals in football or basketball.

[0035] Step 120: If the scores of N adjacent video frames are within a certain range, then divide the N video frames into a video segment.

[0036] In this step, if the scores of N consecutive frames in the video to be processed fall within a certain score range, these N consecutive frames are considered as a single video segment. Based on this, multiple video segments can be divided from the video to be processed. Here, N is a positive integer.

[0037] For example, the maximum range of obtainable score information is 0 to 100. Based on this maximum range, four score ranges can be divided: 0 to 25, 26 to 50, 51 to 75, and 75 to 100. If the score information of N consecutive frames is within the range of 0 to 25, then these N frames are considered as a video segment.

[0038] Step 130: Based on the different score ranges, mark the video segments on the video frame progress control of the video to be processed; where one score range corresponds to one mark, and different mark information represents different recommendation levels.

[0039] In this step, all video clips within the same score range are marked with the same tagging information. This tagging method helps users quickly filter out all video clips within the same score range.

[0040] like Figure 3 As shown, a video frame progress control is displayed in the video frame area. On this video frame progress control, all video segments under one score range are marked with black markers 301, and all video segments under another score range are marked with white markers 302.

[0041] Optionally, the correspondence between score ranges and tag information can be preset. This correspondence can be customized by the user to facilitate the user in quickly identifying video segments with higher scores based on the tag information.

[0042] For example, video clips marked in green have a higher score and are the ones users most want to keep; video clips marked in yellow have the second highest score and are the ones users second most want to keep; video clips marked in red have a lower score and are the ones users second least want to keep; and video clips marked in gray have the lowest score and are the ones users least want to keep.

[0043] Step 140: Receive the user's first input on the tagging information.

[0044] Here, the first input is used to select any marked information. The first input can be touch input, voice command, a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage needs, and this application embodiment does not limit it. The specific gesture in this application embodiment can be any one of the following: single-click gesture, swipe gesture, drag gesture, pressure recognition gesture, long-press gesture, area change gesture, double-press gesture, and double-tap gesture. The single-click input in this application embodiment can be single-click input, double-click input, or any number of clicks, and can also be long-press input or short-press input.

[0045] Step 150: In response to the first input, output a collection of video segments from the video to be processed.

[0046] Here, in response to the first input, video segments with user-selected tag information can be extracted, and the extracted video segments can be spliced ​​together to obtain a collection of video segments.

[0047] For example, when marking segments under different score ranges, if the user selects the marking information corresponding to the video segment with the highest score range, then by inputting the marking information as the first input, all segments with the highest scores can be retained with one click to generate a collection of video segments.

[0048] Furthermore, when marking segments within different score ranges, users can adjust the marking information of a segment by inputting their own information, so that the final marking information is not only automatically marked, but also based on user preferences.

[0049] In the embodiments of this application, the score information of each video frame in the video to be processed is first output. The score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information. If the score information of N adjacent video frames falls within a score range, the N video frames are divided into a video segment. Then, according to the different score ranges, the video segment is marked on the video frame progress control of the video to be processed. Each score range corresponds to one mark, and different marks represent different recommendation levels. Finally, upon receiving the user's first input of the mark information, a collection of video segments from the video to be processed is output. Therefore, in the embodiments of this application, the video can be automatically divided into video segments, and different video segments can be marked with recommendation levels. Users only need to perform simple input to quickly filter out segments that meet their needs, eliminating the need for users to view, crop, save, and then stitch the cropped segments together frame by frame, thus simplifying user operations and improving video processing efficiency.

[0050] In another embodiment of the video processing method of this application, the step of marking different information for each segment includes:

[0051] Sub-step A1: Mark video clips with the same score range with the same background color.

[0052] Alternatively, in one implementation, different colors are used to fill the corresponding video segments on the video frame progress control.

[0053] For example, video clips with high scores are filled with green, video clips with medium scores are filled with yellow, video clips with low scores are filled with red, and video clips with the lowest scores are filled with gray.

[0054] Optionally, the fill color in the video frame progress control has high transparency to ensure that the background color does not cause significant interference to the display of the video frame.

[0055] In this embodiment, different background colors, such as red, yellow, and green, will be displayed on the video frame progress control. Users can quickly lock onto automatically recommended or unrecommended video segments by using the colors, so as to help users quickly lock onto multiple video segments by automatically filtering relevant segments.

[0056] In another embodiment of the video processing method of this application, after step 130, the method further includes:

[0057] Step B1: Receive the second input of the first tag information of the first video segment in the video clip.

[0058] In some embodiments of this application, the second input is used to modify the first tag information, replacing it with the second tag information, thereby adjusting the recommendation priority of the first video segment and facilitating unified management of all segments under the same tag information by the user. Exemplarily, the second input includes, but is not limited to: touch input by the user using a finger or stylus to controls on the screen, touch input to the screen area, voice commands input by the user, specific gestures input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this embodiment of the invention does not limit it. Specific gestures in this application embodiment can be any one of a single-click gesture, a swipe gesture, a drag gesture, a pressure-recognition gesture, a long-press gesture, an area-changing gesture, a double-press gesture, or a double-tap gesture; the click input in this application embodiment can be a single-click input, a double-tap input, or any number of clicks, and can also be a long-press input or a short-press input.

[0059] like Figure 4 As shown, the second input can be that the user long-presses a segment on the video frame progress control, which becomes the first video segment in this embodiment, thereby displaying an option list 401 with various marking information, such as green, yellow, red, and gray level options. If the current first segment is marked as red, the user can click on a yellow or green option. Here, different color levels represent different color markings; the use of levels here indicates that the color represents the recommendation priority.

[0060] When displaying options for each tag information, the options for each tag information can be displayed sequentially in descending order of the recommended priority indicated by each tag information.

[0061] Step B2: In response to the second input, replace the first tag information of the first video segment with the second tag information associated with the second input.

[0062] The first tag information is different from the second tag information.

[0063] In this step, the first video segment is no longer marked as the first tag information, but is marked as the second tag information.

[0064] In one application scenario, the model assesses the quality of a first video clip and deems it excellent, recommending it to the user as part of a preferred selection. However, if the user, after watching the first clip, feels it doesn't adequately express their shooting concept, they manually remove it from the preferred selection and mark it as an alternative. Here, the preferred selection has higher priority than the alternatives. Furthermore, if the user subsequently finds one of the alternative clips satisfactory, they can easily revert it to the preferred selection without needing to re-edit or go through a cumbersome process.

[0065] In this embodiment, after the model marks the video segments in the video to be processed, the user can change the marking information of any video segment. Thus, automatic marking and manual marking complement each other, so that the final synthesized collection of video segments can not only display the wonderful content captured by the user, but also make the display close to or the same as the user's expected effect.

[0066] In another embodiment of the video processing method of this application, after step 130, the method further includes:

[0067] Step C1: When displaying the second video segment in the video clip, receive a third input to the first control and the second control on the second video segment.

[0068] In some embodiments of this application, the aforementioned third input is used by the user to operate on the second video segment, so as to trim the second video segment into a third video segment in the initial state of the second video segment automatically divided by the model. Exemplarily, the aforementioned third input includes, but is not limited to: user input via touch devices such as fingers or styluses to controls on the screen, touch input to the screen area, or voice commands input by the user, or specific gestures input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this embodiment of the invention does not impose limitations. Specific gestures in the embodiments of this application can be any one of a single-click gesture, a swipe gesture, a drag gesture, a pressure-recognition gesture, a long-press gesture, an area-change gesture, a double-press gesture, or a double-tap gesture; the click input in the embodiments of this application can be a single-click input, a double-tap input, or any number of clicks, and can also be a long-press input or a short-press input.

[0069] like Figure 5As shown, when a user clicks on the second video segment in the video frame progress control, the initial start position of the second video segment is displayed in the video frame progress control, and the first control 501 is displayed at the initial start position. If the duration of the second video segment is just enough to display the initial end position of the second video segment within the width of the screen, then the initial end position of the second video segment is displayed, and the second control 502 is displayed at the initial end position. The aforementioned third input can be that the user presses and holds the first control 501 and drags it on the second segment, and / or the user presses and holds the second control 502 and drags it on the second segment.

[0070] Step C2: In response to the third input, obtain the third video segment according to the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video segment.

[0071] The tagging information of the second video segment is used to tag the third video segment.

[0072] In this step, the second video segment is cropped into a third video segment after the third input, while the marking information of the second video segment is retained on the third video segment.

[0073] Optionally, based on user settings, video segments in the second video segment other than the third video segment can be marked with information at a lower recommendation level or other recommendation levels, or the video segments in the second video segment other than the third video segment can be left unmarked. When there are unmarked video segments in the video to be processed, the unmarked video segments are considered to have the lowest recommendation priority.

[0074] Optionally, if there are obvious audio boundaries in the second video clip, such as in a concert video where some parts have someone singing and others do not, the audio feature information of the second video clip can be displayed to prompt the user to keep the clip with clearer audio.

[0075] like Figure 5 As shown, above the second video clip, the audio feature information of the second video clip is displayed, indicating the start and end of the sound. Furthermore, the first and second controls of the second video clip can be displayed at the start and end of the sound respectively, so that the user can directly save the clip currently cropped by the first and second controls.

[0076] In this embodiment, in addition to automatically dividing the video segments, users can also manually adjust the duration of each video segment so that the final retained video segments can be used to create a video clip collection, satisfying the user's ideas for creating such a collection. Furthermore, this embodiment uses audio as a reference to provide cropping suggestions, thereby achieving precise audio matching and preventing the selected video segments from being positioned directly on the image. This allows the retained video segments to showcase more of the shooting details.

[0077] In another embodiment of the video processing method of this application, step 140 includes:

[0078] Sub-step D1: Receive the user's first input on the third tag information in the tag information.

[0079] In some embodiments of this application, the aforementioned first input is used by a user to operate on at least one marker information to trigger a unified output of all video segments under that at least one marker information. Exemplarily, the aforementioned first input includes, but is not limited to: touch input by the user using a finger or stylus to control on the screen, touch input to a screen area, or voice commands input by the user, or specific gestures input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this embodiment of the invention does not impose limitations. Specific gestures in the embodiments of this application can be any one of a single-click gesture, a swipe gesture, a drag gesture, a pressure-recognition gesture, a long-press gesture, an area-change gesture, a double-press gesture, or a double-tap gesture; the click input in the embodiments of this application can be a single-click input, a double-tap input, or any number of clicks, and can also be a long-press input or a short-press input.

[0080] For example, when a playback page is displayed, if a user clicks on the edit option provided on the page, and then clicks the edit control, the user is taken to the editing page for the video to be processed. Figure 6 As shown, a control 601 with a highlight mark is displayed on the editing page. When the user clicks on the control 601 with the highlight mark, as shown... Figure 7 As shown, controls appear to trigger the display of video clips under various marker information. For example, controls are displayed to trigger the display of video clips under various color level markers. Specifically, control 701 can display green level clips. When the user clicks control 701 for green level clips, they can also click other controls that display clips of a certain color level at the same time.

[0081] Optionally, in some implementations, all video segments within the lowest score range can be automatically removed from the video editing page. For example, if video segments within the lowest score range are marked in gray, then all video segments marked in gray are removed; or, if video segments within the lowest score range are not marked, then all unmarked video segments are removed.

[0082] Further, step 150 includes:

[0083] Sub-step D2: In response to the first input, output all video segments marked by the third tag information in the video to be processed, and obtain a collection of video segments.

[0084] For example, if at least one piece of information associated with the first input is a green level, then only all video segments marked as green will be displayed on the video frame progress control. Meanwhile, if... Figure 7 As shown, an indicator line 702 is displayed at the beginning of each segment to help the user quickly locate it.

[0085] For example, if at least one piece of information associated with the first input is green and yellow, then only all video segments marked as green and yellow will be displayed on the video frame progress control. At the same time, an indicator line will be displayed at the beginning of each video segment to help the user quickly locate it.

[0086] In one implementation, such as Figure 7 As shown, in the video editing page, when the cursor 703 on the video frame progress control is at a certain position, the highlight mark control 704 acts as a display control, and its background is filled with the mark color of the video segment at that position to indicate the mark information of the current video segment to the user.

[0087] Additionally, on the video editing page, users can still trim video clips and change the tagging information of video clips at any time. For example... Figure 8 As shown, when a user long-presses on a video clip, a list of options for yellow, green, red, and gray levels is displayed, allowing the user to change the marker color of the video clip.

[0088] Finally, the first input also includes: such as Figure 7 As shown, when the user clicks the save control 705, only all the video segments displayed on the current video frame progress control are output as a collection of synthesized video segments.

[0089] For example, if you display only all video clips marked as green on the video frame progress control, the output will be a collection of all video clips marked as green.

[0090] For example, if only all video clips marked as green and yellow are displayed on the video frame progress control, the output will be a set including all video clips marked as green and all video clips marked as yellow.

[0091] In more embodiments, during the playback of a collection of video clips, the tagging information of each video clip can also be displayed in the video frame area of ​​the playback page.

[0092] In this embodiment, the user can output a collection of video clips consisting of all video clips under one or more tagged information based on one or more tagged information, thereby realizing a one-click output of all video clips without the cumbersome operation steps of saving video clips first and then synthesizing them.

[0093] In another embodiment of the video processing method of this application, before step 110, the method further includes:

[0094] Step E1: Receive a fourth input for at least two videos.

[0095] Step E2: In response to the fourth input, stitch together at least two videos to obtain the video to be processed.

[0096] In some embodiments of this application, the aforementioned fourth input is used for a user to select at least two videos to create a video by splicing the at least two videos as the video to be processed. The fourth input can be a fourth operation. Exemplarily, the aforementioned fourth input includes, but is not limited to: user input via touch devices such as fingers or styluses to controls on the screen, touch input to the screen area, or voice commands input by the user, or specific gestures input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this embodiment of the invention does not impose limitations. Specific gestures in the embodiments of this application can be any one of a single-click gesture, a swipe gesture, a drag gesture, a pressure-recognition gesture, a long-press gesture, an area-change gesture, a double-press gesture, or a double-tap gesture; the click input in the embodiments of this application can be a single-click input, a double-tap input, or any number of clicks, and can also be a long-press input or a short-press input.

[0097] For example, the fourth input mentioned above could be: displaying a list of video objects in the photo album; the user clicks on the selected option, then clicks on multiple video objects in sequence; and then, by long-pressing, triggers the display of the playback option. Clicking the playback option takes the user to the video playback page, where the multiple video objects selected by the user are sequentially pieced together to form a long video, i.e., the video to be processed. Figure 8 The video frame progress control consists of multiple segments, such as the first segment 801 and the second segment 802, with each segment representing a video object.

[0098] In other embodiments, users can also locate highlights in each video individually and then combine the highlights from multiple videos into a single collection.

[0099] In this embodiment, regarding the interaction method, multiple videos can be imported into the editing project at once, and then the best segments can be located in the spliced ​​long video. Importing multiple videos at once and then locating the best segments allows users to directly evaluate whether the output video based on the located best segments is reasonable, and adjust the best segments at any time based on their suitability. This method has a high margin of error, allowing users to adjust the length and tagging information of segments at any time. It facilitates users to filter best segments by comparing multiple video objects until they are satisfied and then save the final video.

[0100] In summary, the process first outputs the score information of each video frame in the video to be processed. This score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information. If the scores of N adjacent video frames fall within a certain score range, these N video frames are divided into a video segment. Then, based on the different score ranges, the video segments are marked on the video frame progress control of the video to be processed. Each score range corresponds to one mark, and different marks represent different recommendation levels. Finally, upon receiving the user's first input of the mark information, a collection of video segments from the video to be processed is output. Therefore, in the embodiments of this application, the video can be automatically divided into video segments, and different video segments can be marked with recommendation levels. Users only need to perform simple input to quickly filter out segments that meet their needs, eliminating the need for users to view, crop, save, and then stitch the cropped segments together frame by frame, thus simplifying user operations and improving video processing efficiency.

[0101] The video processing method provided in this application can be executed by a video processing device. This application uses a video processing device executing the video processing method as an example to illustrate the video processing device provided in this application.

[0102] Figure 9 A block diagram of a video processing apparatus according to an embodiment of this application is shown. The apparatus includes:

[0103] The first output module 10 is used to output the score information of each video frame in the video to be processed; wherein the score information is determined based on at least one of the following: shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information;

[0104] The segmentation module 20 is used to segment the N video frames into a video segment if the score information of N adjacent video frames is within a score range; where N is a positive integer.

[0105] The marking module 30 is used to mark the video segments on the video frame progress control of the video to be processed according to different score ranges; wherein, one score range corresponds to one marking information, and different marking information represents different recommendation levels;

[0106] The first receiving module 40 is used to receive the user's first input on the tagging information;

[0107] The second output module 50 is configured to output a collection of video segments of the video to be processed in response to the first input.

[0108] In the embodiments of this application, the score information of each video frame in the video to be processed is first output. The score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information. If the score information of N adjacent video frames falls within a score range, the N video frames are divided into a video segment. Then, according to the different score ranges, the video segment is marked on the video frame progress control of the video to be processed. Each score range corresponds to one mark, and different marks represent different recommendation levels. Finally, upon receiving the user's first input of the mark information, a collection of video segments from the video to be processed is output. Therefore, in the embodiments of this application, the video can be automatically divided into video segments, and different video segments can be marked with recommendation levels. Users only need to perform simple input to quickly filter out segments that meet their needs, eliminating the need for users to view, crop, save, and then stitch the cropped segments together frame by frame, thus simplifying user operations and improving video processing efficiency.

[0109] Optionally, the marking module 30 includes a marking unit for marking video segments with the same background color within the same score range. This allows users to quickly locate video segments by color, enabling automatic filtering of relevant segments and helping users quickly identify multiple segments within a video.

[0110] Optionally, the device further includes: a second receiving module, configured to receive a second input of first tagging information for a first video segment in the video clip; and a replacement module, configured to, in response to the second input, replace the first tagging information of the first video segment with second tagging information associated with the second input; wherein the first tagging information is different from the second tagging information. In this embodiment, after the model tags the video segments in the video to be processed, the user can change the tagging of any video segment, thereby automatic tagging and manual tagging complement each other, so that the final synthesized collection of video segments can not only display the wonderful content shot by the user, but also make the display close to or the same as the user's expected effect.

[0111] Optionally, the device further includes: a third receiving module, configured to receive a third input to a first control and a second control on the second video segment when displaying a second video segment in the video clip; and a processing module, configured to, in response to the third input, obtain a third video segment based on a start time indicated by the first control and an end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with audio feature information of the second video segment; and wherein the tagging information of the second video segment is used to tag the third video segment. By using audio as a reference to provide cropping suggestions, precise audio matching is achieved, preventing the selected video segment of this application from being positioned on the image, thereby allowing the retained video segment to reflect more shooting details.

[0112] Optionally, the first receiving module 10 includes: a receiving unit, configured to receive a first input from a user regarding the third tagging information in the tagging information; the second output module includes: an output unit, configured to, in response to the first input, output all video segments tagged by the third tagging information in the video to be processed, thereby obtaining a collection of the video segments. Thus, the user can output a collection of video segments composed of all video segments under one or more tagging information based on one or more tagging information, achieving a one-click output of all video segments without the cumbersome steps of saving video segments first and then synthesizing them.

[0113] The device in the embodiments of this application can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0114] The device in this application embodiment can be a device with an action system. The action system can be an Android action system, an iOS action system, or other possible action systems, and this application embodiment does not specifically limit it.

[0115] The apparatus provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0116] Optionally, such as Figure 10 As shown, this application embodiment also provides an electronic device 100, including a processor 101, a memory 102, and a program or instructions stored in the memory 102 and executable on the processor 101. When the program or instructions are executed by the processor 101, they implement the various steps of any of the above video processing method embodiments and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0117] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0118] Figure 11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0119] The electronic device 1000 includes, but is not limited to, the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, processor 1010, camera 1011, etc.

[0120] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0121] The processor 1010 is configured to output score information for each video frame in the video to be processed; wherein the score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information; if the score information of N adjacent video frames falls within a score range, then the N video frames are divided into a video segment; wherein N is a positive integer; according to different score ranges, the video segment is marked on the video frame progress control of the video to be processed; wherein one score range corresponds to one mark, and different mark information represents different recommendation levels; receive a first input from the user regarding the mark information; and in response to the first input, output a collection of the video segments of the video to be processed.

[0122] In the embodiments of this application, the score information of each video frame in the video to be processed is first output. The score information is determined based on at least one of the following: shooting area information, shooting movement information, shooting object information, shooting scene information, and audio content information. If the score information of N adjacent video frames falls within a score range, the N video frames are divided into a video segment. Then, according to the different score ranges, the video segment is marked on the video frame progress control of the video to be processed. Each score range corresponds to one mark, and different marks represent different recommendation levels. Finally, upon receiving the user's first input of the mark information, a collection of video segments from the video to be processed is output. Therefore, in the embodiments of this application, the video can be automatically divided into video segments, and different video segments can be marked with recommendation levels. Users only need to perform simple input to quickly filter out segments that meet their needs, eliminating the need for users to view, crop, save, and then stitch the cropped segments together frame by frame, thus simplifying user operations and improving video processing efficiency.

[0123] Optionally, the processor 1010 is also configured to mark video segments with the same score range with the same background color. This allows users to quickly locate video segments by color, enabling automatic filtering of relevant segments and helping users quickly identify multiple segments within a video.

[0124] Optionally, the user input unit 1007 is configured to receive a second input of first tagging information for a first video segment in the video clip; the processor 1010 is further configured to, in response to the second input, replace the first tagging information of the first video segment with second tagging information associated with the second input; wherein the first tagging information is different from the second tagging information. In this embodiment, after the model tags the video segments in the video to be processed, the user can change the tagging of any video segment, thereby automatic tagging and manual tagging complement each other, so that the final synthesized collection of video segments can not only display the wonderful content shot by the user, but also make the display close to or the same as the user's expected effect.

[0125] Optionally, the user input unit 1007 is further configured to receive a third input to a first control and a second control on the second video segment when displaying a second video segment in the video clips; wherein each segment includes the second segment; the processor 1010 is further configured to, in response to the third input, obtain a third video segment based on a start time indicated by the first control and an end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video segment; wherein the tagging information of the second video segment is used to tag the third video segment. By using audio as a reference to provide cropping suggestions, precise audio matching is achieved, preventing the selected video segment of this application from being positioned on the image, thereby allowing the retained video segment to reflect more shooting details.

[0126] Optionally, the user input unit 1007 is further configured to receive a first input from the user regarding the third tagging information in the tagging information; the processor 1010 is further configured to, in response to the first input, output all video segments in the video to be processed tagged by the third tagging information, thereby obtaining a collection of the video segments. Thus, the user can output a collection of video segments composed of all video segments under one or more tagging information based on one or more tagging information, thereby achieving a one-click output of all video segments without the cumbersome steps of saving video segments first and then synthesizing them.

[0127] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes image data of still images or video images obtained by an image capture device (such as a camera) in video image capture mode or image capture mode. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here. The memory 1009 can be used to store software programs and various data, including but not limited to applications and motion systems. Processor 1010 may integrate an application processor and a modem processor. The application processor mainly handles the action system, user page, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 1010.

[0128] The memory 1009 can be used to store software programs and various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0129] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.

[0130] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described video processing method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0131] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0132] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described video processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0133] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0134] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the video processing method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0135] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0137] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A video processing method, characterized in that, The method includes: Output the score information of each video frame in the video to be processed; wherein, the score information is determined based on at least one of the following: shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; If the scores of N adjacent video frames fall within a certain range, then the N video frames are divided into a video segment; where N is a positive integer. Based on the different score ranges, the video segments are marked on the video frame progress control of the video to be processed; wherein, one score range corresponds to one mark, and different mark information represents different recommendation levels; Receive the user's first input regarding the tagging information; In response to the first input, a collection of the video segments of the video to be processed is output.

2. The method according to claim 1, characterized in that, The step of marking the video segment includes: Video clips with the same score range are labeled with the same background color.

3. The method according to claim 1, characterized in that, After marking the video segment on the video frame progress control of the video to be processed according to different score ranges, the method further includes: Receive a second input of first tagging information for a first video segment in the video segment; In response to the second input, the first tag information of the first video segment is replaced with the second tag information associated with the second input; The first tagging information is different from the second tagging information.

4. The method according to claim 1, characterized in that, After marking the video segment on the video frame progress control of the video to be processed according to different score ranges, the method further includes: When displaying a second video segment from the video clips, a third input is received to the first and second controls on the second video segment; In response to the third input, a third video segment is obtained based on the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video segment; The tagging information of the second video segment is used to tag the third video segment.

5. The method according to claim 1, characterized in that, The first input from the user regarding the tagging information includes: Receive the user's first input regarding the third tag information in the tag information; The step of outputting a set of video segments of the video to be processed in response to the first input includes: In response to the first input, all video segments marked by the third tagging information in the video to be processed are output to obtain a collection of the video segments.

6. A video processing apparatus, characterized in that, The device includes: The first output module is used to output the score information of each video frame in the video to be processed; wherein the score information is determined based on at least one of the following: shooting area information, shooting camera movement information, shooting object information, shooting scene information, and audio content information; The segmentation module is used to divide the N video frames into a video segment if the score information of N adjacent video frames is within a score range; where N is a positive integer. The tagging module is used to tag the video segments on the video frame progress control of the video to be processed according to different score ranges; wherein, one score range corresponds to one tagging information, and different tagging information represents different recommendation levels; The first receiving module is used to receive the user's first input on the tagging information; The second output module is configured to output a collection of video segments of the video to be processed in response to the first input.

7. The apparatus according to claim 6, characterized in that, The marking module includes: A marking unit is used to mark video segments with the same background color for those with the same score range.

8. The apparatus according to claim 6, characterized in that, The device further includes: The second receiving module is used to receive a second input of first tagging information for the first video segment in the video segment; A replacement module is configured to, in response to the second input, replace the first tag information of the first video segment with the second tag information associated with the second input; The first tagging information is different from the second tagging information.

9. The apparatus according to claim 6, characterized in that, The device further includes: The third receiving module is used to receive a third input to the first and second controls on the second video segment when the second video segment is displayed; The processing module is configured to, in response to the third input, obtain a third video segment based on the start time indicated by the first control and the end time indicated by the second control; wherein the recommended positions of the first control and the second control are associated with the audio feature information of the second video segment; The tagging information of the second video segment is used to tag the third video segment.

10. The apparatus according to claim 6, characterized in that, The first receiving module includes: The receiving unit is configured to receive a first input from the user regarding the third tag information in the tag information; The second output module includes: The output unit is configured to respond to the first input by outputting all video segments marked by the third tag information in the video to be processed, thereby obtaining a set of the video segments.

Images