Video processing method, electronic device, chip system and storage medium

Abstract

The present application relates to the field of video processing technology, aims to solve the problem of low video editing efficiency, and provides a video processing method, electronic device, chip system and storage medium. The video processing method is applied to an electronic device, including displaying a video album interface, the video album interface includes video options of one or more videos stored in the electronic device; in response to an input of selecting a target video option from the video options in the video album interface, obtaining the original video indicated by the target video option; obtaining the shot label and scene label in the original video; in response to the first input, editing the original video according to the target editing rule, the shot label and the scene label to obtain a new video. It can automatically and efficiently provide new videos with artistic appeal, reduce the requirements for the user's video editing ability, and improve the efficiency of video editing.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing technology, and in particular to a video processing method, an electronic device, a chip system, and a storage medium. Background Art

[0002] With the development of video technology in electronic devices, users have an increasing demand for video editing functions. However, existing video editing mainly uses manual methods to extract the required video clips from the original video and then splice the extracted video clips, which requires certain video editing capabilities of users, and manual operations will consume a lot of time, thereby reducing the efficiency of video editing. Summary of the invention

[0003] The embodiments of the present application provide a video processing method, an electronic device, a chip system, and a storage medium, which can automatically and efficiently provide new videos with artistic appeal, reduce the requirements for the user's video editing ability, and improve video editing efficiency.

[0004] In a first aspect, the present application provides a video processing method, which is applied to an electronic device, the method comprising: displaying a video album interface, the video album interface including video options of one or more videos stored in the electronic device, in response to an input of selecting a target video option from the video options in the video album interface, obtaining an original video indicated by the target video option, obtaining shot tags and scene tags in the original video, the shot tags including the shot type, start time and end time of the shot clip, and the scene tags including the scene type, start time and end time of the scene clip; and, in response to a first input, editing the original video according to the target editing rules, the shot tags and the scene tags to obtain a new video.

[0005] For the original video that has been recorded, the image stream of the original video can be analyzed from the perspective of the lens, which can provide clips of different lens types for subsequent video editing, enrich the video material, optimize the video editing effect, and improve the user experience. According to the editing rules, multiple lens clips in the original video are automatically edited to obtain a new video. Automatic video editing can reduce the user's video editing ability requirements and improve the efficiency of video editing. Furthermore, the editing rules reflect the "editing thinking" of video editing, which is used to achieve a smooth and meaningful way to combine the selected clips. Through the editing rules combined with the clips selected according to the lens type, users can be provided with new videos that are more cinematic, more artistically appealing, and more attractive to video viewers. Furthermore, based on the original video including lens tags and scene tags, the steps of detecting the image frames of the original video can be reduced, and editing can be performed according to the lens tags and scene tags of the original video to improve the efficiency of video editing.

[0006] In one possible implementation, the method further includes: in response to a second input, displaying an editing style interface, the editing style interface including editing options corresponding to each editing style; in response to an input of selecting a target editing option from the editing options of the editing style interface, using the editing style corresponding to the target editing option as the target editing style, and using the editing rules corresponding to the target editing style as the target editing rules.

[0007] By providing an editing style interface, users can select the editing style they need, and then edit the original video into the editing style they selected, thereby optimizing the video editing effect and improving the user experience.

[0008] In one possible implementation, the electronic device stores an editing rule library, which stores editing rules corresponding to each editing style. The method also includes: when no input is obtained to instruct the user to select a target editing style (such as no input is obtained to select a target editing option from the editing options in the editing style interface), determining the editing rule with the highest matching degree in the editing rule library according to the shot type in the original video; determining whether the editing rule with the highest matching degree meets preset requirements; if so, using the editing rule with the highest matching degree as the target editing rule; if not, using the default editing rule as the target editing rule.

[0009] In a possible implementation, when responding to the first input (ie, when preparing to edit), it is detected that no input indicating that the user has selected the target editing style has been obtained, and the editing rules are automatically matched.

[0010] By providing an editing rule library, the user's needs for various editing styles can be met. Automatically matching editing rules in the editing rule library according to the lens type in the original video can reduce user operations, and using the editing rule with the highest matching degree as the target editing rule is conducive to providing new videos with better video effects, optimizing video editing effects, and improving user experience. When the editing rule with the highest matching degree does not meet the preset requirements and the user does not select an editing style, the default editing rule is used as the target editing rule. The default editing rule is used to provide more different lens types for the new video, which is conducive to optimizing the video effect of the new video.

[0011] In a possible implementation, the method further includes: when the original video does not include a shot tag, performing shot detection on the image frames in the original video to obtain the shot tag of the original video, and saving the shot tag; when the original video does not include a scene tag, performing scene detection on the image frames in the original video to obtain the scene tag of the original video, and saving the scene tag; when the original video does not include a shot tag and a scene tag, performing shot detection and scene detection on the image frames in the original video to obtain the shot tag and scene tag of the original video, and saving the shot tag and the scene tag.

[0012] By determining that the original video does not include the corresponding label, the corresponding detection is performed. That is, when the original video includes the corresponding label, the detection can be reduced, avoiding the method of blindly performing detection during video editing, thereby improving the efficiency of video editing.

[0013] In a second aspect, the present application provides a video processing method, which is applied to an electronic device, the method comprising: in response to a third input, starting to record an original video and displaying a recording interface, the recording interface being used to display an image stream of the original video acquired in real time by a camera of the electronic device, performing lens detection on image frames in the image stream to obtain multiple lens segments and lens labels, each lens label corresponding to a lens segment, the lens label including the lens type, start time and end time of the corresponding lens segment, in response to a fourth input, ending the recording of the original video; editing multiple lens segments in the original video according to target editing rules to obtain a new video, and saving the original video, the new video and the lens labels.

[0014] During the video recording process, the image stream of the original video is analyzed from the perspective of the lens, which can provide clips of different lens types for subsequent video editing, enrich the video material, optimize the video editing effect, and improve the user experience. According to the editing rules, multiple lens clips and lens labels in the original video are automatically edited to obtain a new video. Automatic video editing can reduce the user's video editing ability requirements and improve the efficiency of video editing. Furthermore, the editing rules reflect the "editing thinking" of video editing, which is used to achieve a smooth and meaningful way to combine the selected clips. Through the editing rules combined with the clips selected according to the lens type, users can be provided with new videos that are more cinematic, more artistic, and more attractive to video viewers. Furthermore, the lens labels obtained by detecting the image frames during the video recording process are saved, so that when the original video is edited later, the steps of detecting the image frames of the original video can be reduced, and the original video can be edited according to the lens labels to improve the efficiency of video editing.

[0015] In one possible implementation, the method also includes: in response to a fifth input, displaying an editing style interface, the editing style interface including editing options corresponding to each editing style; in response to an input of selecting a target editing option from the editing options of the editing style interface, using the editing style corresponding to the target editing option as the target editing style, and using the editing rules corresponding to the target editing style as the target editing rules.

[0016] By providing an editing style interface, users can select the editing style they need, and then edit the original video into the editing style they selected, thereby optimizing the video editing effect and improving the user experience.

[0017] In one possible implementation, the target editing rules indicate the shot type and / or scene type of the required clip. After the editing rules corresponding to the target editing style are used as the target editing rules, the method also includes: obtaining the shot type and / or scene type of the required clip, and displaying a prompt interface, wherein the prompt interface includes the shot type and / or scene type.

[0018] By displaying a prompt interface to prompt the user of the shot type, or scene type, or shot type and scene type required for the editing style selected by the user, the user can record the corresponding shot type and / or scene type when recording the video according to the prompt, so that the recorded original video is more in line with the requirements of the selected editing style, which is conducive to improving video editing efficiency and providing new videos with better video effects.

[0019] In a possible implementation, the electronic device stores an editing rule library, which stores editing rules corresponding to various editing styles. The method further includes: in response to a sixth input, displaying an editing style option, the editing style option is used for a user to select an editing style; when the editing style option indicates automatic matching of the editing style, determining the editing rule with the highest degree of matching with the original video in the editing rule library according to the type of shots in the original video; determining whether the editing rule with the highest degree of matching meets the preset requirements; if so, using the editing rule with the highest degree of matching as the target editing rule; if not, using the default editing rule as the target editing rule. In a possible implementation, when no input is made to the editing style option, the editing style option indicates automatic matching of the editing style by default.

[0020] By providing an editing rule library, the user's needs for various editing styles can be met. Automatically matching editing rules in the editing rule library according to the lens type in the original video can reduce user operations, and using the editing rule with the highest matching degree as the target editing rule is conducive to providing new videos with better video effects, optimizing video editing effects, and improving user experience. When the editing rule with the highest matching degree does not meet the preset requirements and the user does not select an editing style, the default editing rule is used as the target editing rule. The default editing rule is used to provide more different lens types for the new video, which is conducive to optimizing the video effect of the new video.

[0021] In a possible implementation, performing shot detection on the image frames in the image stream to obtain the shot tags includes: in response to the electronic device entering a one-record-multiple-receive mode by default when recording a video or in response to a seventh input, performing shot detection and scene detection on the image frames in the image stream to obtain the shot tags and the scene tags, the seventh input indicating that the electronic device enters the one-record-multiple-receive mode, and the scene tags including the scene type, the start time, and the end time of the scene segment;

[0022] Then, multiple shot segments in the original video are edited according to the target editing rules to obtain a new video including: according to the target editing rules, shot tags and scene tags, wonderful shot segments are cut out from the original video, and the wonderful shot segments include the shot type and scene type indicated by the target editing rules; according to the target editing rules, the wonderful shot segments are combined to obtain a new video.

[0023] The electronic device may detect the image frames in the image stream by default when recording a video, or may detect the image frames in the image stream in response to the seventh input, thereby providing the user with a variety of ways to enter the detection. Further, the detection may include one or more of shot detection and scene detection, that is, the image stream of the original video may be analyzed from the perspective of the shot and / or scene, and more video materials required for subsequent video editing may be provided. Furthermore, wonderful shot segments are cropped from the original video according to the target editing rules, shot tags and scene tags, so that the wonderful shot segments simultaneously meet the shot type and scene type indicated by the segments required by the target editing rules, and not only the shot type-related content is provided, but also the scene type-related content is provided, so as to provide users with more and more accurate content expressions and optimize the video effects.

[0024] In one possible implementation, saving the original video, the new video, and the shot tags includes: saving multiple shot clips in the original video; saving the shot tags to the video file of the original video; displaying the original video, the new video, and the shot clips to receive user input on any one or more videos among the original video, the new video, and the shot clips.

[0025] By saving the shot tags to the original video file, it is easy to edit the original video later, which improves the editing efficiency. Saving and displaying the original video, new video and shot clips allows users to input the video obtained during the recording process, such as playing and editing, to meet user needs and improve user experience.

[0026] In one possible implementation, the method further includes: in response to an eighth input, displaying a one-record-multiple-gain interface for displaying the footage obtained by the electronic device; in response to a selection of a footage, determining that the selected footage is a target footage; and in response to a ninth input, editing the target footage according to a target editing rule to obtain a new video.

[0027] Among them, the footage clips displayed in the "one-record, multiple-gain" interface include one or more of the following: footage clips obtained during the video recording process, footage clips obtained by detecting original videos saved by the electronic device (such as videos that have been shot or downloaded from the Internet), and footage clips of new videos that have been edited.

[0028] By providing a one-record-multiple-receive interface to display the footage clips obtained by the electronic device, it is convenient for users to create footage clips, meet user needs, and improve user experience.

[0029] In a third aspect, the present application provides a video processing method for application to an electronic device, the method comprising: obtaining an image stream of an original video; performing shot detection on image frames in the image stream to determine the shot type of the image frames; determining shot segments based on the shot types of multiple image frames; and, editing multiple shot segments in the original video according to target editing rules to obtain a new video.

[0030] By analyzing the image stream of the original video from the perspective of the lens, it is possible to provide clips of different lens types for subsequent video editing, enrich video materials, optimize video editing effects, and improve user experience. Multiple lens clips in the original video are automatically edited according to the editing rules to obtain a new video. Automatic video editing can reduce the user's video editing ability requirements and improve video editing efficiency. Furthermore, the editing rules reflect the "editing thinking" of video editing, which is used to achieve a smooth and meaningful way to combine the selected clips. By combining the editing rules with the clips selected according to the lens type, users can be provided with new videos that are more cinematic, more artistically appealing, and more attractive to video viewers.

[0031] In a possible implementation, the electronic device stores a lens type library, which includes lens types in one or more of the following dimensions: a lens type related to lens size, a lens type related to lens composition, a lens type related to lens movement, a lens type related to lens angle and position, and a lens type related to lens duration; performing lens detection on image frames in an image stream to determine the lens type of the image frame includes: performing lens detection on the image frames in the image stream according to the lens type library to determine the lens type of the image frame.

[0032] By constructing a lens type library, it is possible to detect and analyze image frames in the image stream from different dimensions (such as lens size, lens composition, lens movement, lens angle and position, and lens size, etc.), thereby enriching video materials for subsequent video editing, so that the new video obtained by subsequent editing can be more cinematic, more artistically appealing, and more able to attract the attention of video viewers.

[0033] In a possible implementation, the electronic device stores a camera algorithm library, and performs lens detection on image frames in an image stream according to a lens type library, and determining the lens type of the image frame includes: calling a detection algorithm in the camera algorithm library to perform lens detection on the image frames in the image stream to obtain a detection result of the image frame; and determining the lens type of the image frame according to the lens type library and the detection result of the image frame.

[0034] The image frames in the image stream are subjected to lens detection through the detection algorithm, and the lens type of the image frame can be determined based on the detection result and the lens type library, thereby improving the efficiency of lens type determination.

[0035] In one possible implementation, the electronic device stores an editing rule library, which includes editing rules corresponding to various editing styles, and the editing styles include one or more of the following: editing styles related to scenes, editing styles related to directors, and editing styles related to film and television works. The method also includes: in response to an input selecting a target editing style from the editing styles in the editing rule library, using the editing rules corresponding to the target editing style as the target editing rules; or, the electronic device can automatically match the editing styles in the editing rule library, such as, when no input is obtained to indicate that the user has selected the target editing style (that is, the user has not input an input to the electronic device to select the target editing style from the editing rule library), determining the editing rule with the highest matching degree in the editing rule library according to the shot type in the original video; judging whether the editing rule with the highest matching degree meets the preset requirements; if so, using the editing rule with the highest matching degree as the target editing rule; if not, when the editing rule with the highest matching degree does not meet the preset requirements and the user has not selected an editing style, using the default editing rule as the target editing rule.

[0036] By providing editing styles of different dimensions, such as editing styles related to scenes, editing styles related to directors, and editing styles related to film and television works, it is possible to meet the various needs of users and improve the user experience. When the user does not automatically select an editing style, the editing rules are automatically matched in the editing rule library according to the lens type in the original video, which can reduce the user's operations. The editing rule with the highest matching degree is used as the target editing rule, which is conducive to providing new videos with better video effects, optimizing video editing effects, and improving user experience. When the editing rule with the highest matching degree does not meet the preset requirements and the user does not select an editing style, by using the default editing rule as the target editing rule, the default editing rule is used to provide more different lens types for the new video, which is conducive to optimizing the video effect of the new video.

[0037] In one possible implementation, the editing rule with the highest matching degree indicates the lens type of the required segment, and determining whether the editing rule with the highest matching degree meets the preset requirements includes: calculating the number of pending lens segments of the original video to obtain a first value, where the lens type of the pending lens segments is the lens type of the required segment; calculating the number of required segments indicated by the editing rule with the highest matching degree to obtain a second value; calculating the ratio of the first value to the second value; when the ratio is less than a preset threshold, determining that the editing rule with the highest matching degree does not meet the preset requirements; when the ratio is greater than or equal to the preset threshold, determining that the editing rule with the highest matching degree meets the preset requirements.

[0038] The preset threshold value can be set according to actual conditions, such as 50%.

[0039] By judging whether the editing rule with the highest matching degree meets the preset requirements, the matching degree of the automatically matched editing rules can be guaranteed, and then an editing that is more in line with the original video can be provided to ensure the effect of the new video.

[0040] In a possible implementation, the method also includes: performing scene detection on image frames in the image stream to determine the scene type of the image frames; determining scene fragments according to the scene types of multiple image frames; then editing multiple shot fragments in the original video according to the target editing rule to obtain a new video including: determining multiple wonderful shot fragments in the image stream according to the position of the shot fragment, the shot type of the shot fragment, the position of the scene fragment, the scene type of the scene fragment and the target editing rule, the wonderful shot fragments including the shot type and scene type indicated by the target editing rule; and editing multiple wonderful shot fragments according to the target editing rule to obtain a new video.

[0041] By cutting out wonderful shot clips from the original video according to the target editing rules, the position of the shot clips, the shot type of the shot clips, the scene type of the scene clips and the position of the scene clips, the wonderful shot clips can simultaneously meet the shot type and scene type indicated by the clips required by the target editing rules, and not only provide content related to the shot type but also provide content related to the scene type, thereby providing users with more and more accurate content expression and optimizing video effects.

[0042] In a possible implementation, the target editing rule indicates the shot type and scene type of the required clip, and according to the position of the shot clip, the shot type of the shot clip, the scene type of the scene clip, the position of the scene clip and the target editing rule, determining multiple wonderful shot clips in the image stream includes: determining candidate scene clips according to the scene type of the required clip, the position of the scene clip and the scene type of the scene clip, the candidate scene clips include the scene type of the required clip; determining wonderful shot clips from the candidate scene clips according to the shot type of the required clip, the position of the shot clip and the shot type of the shot clip; or, determining candidate shot clips according to the shot type of the required clip, the position of the shot clip and the shot type of the shot clip, the candidate shot clips include the shot type of the required clip; determining wonderful shot clips from the candidate shot clips according to the scene type of the required clip, the position of the scene clip and the scene type of the scene clip.

[0043] It can be understood that the contents of multiple wonderful shot clips in the image stream determined according to the target editing rules, shot tags and scene tags can refer to determining multiple wonderful shot clips in the image stream according to the positions of the shot clips, the shot types of the shot clips, the positions of the scene clips, the scene types of the scene clips and the target editing rules, and the determined wonderful shot clips include the scene type and shot type indicated by the required clips.

[0044] In one possible implementation, the combination logic of the target editing rules indicates the position of the desired segment in the new video, and then multiple wonderful shot segments are edited according to the target editing rules to obtain a new video, including: combining multiple wonderful shot segments according to the combination logic of the target editing rules to obtain a new video; or adjusting multiple wonderful shot segments according to the target editing rules; and combining the adjusted wonderful shot segments according to the combination logic of the target editing rules to obtain a new video.

[0045] By indicating the position of the required fragments in the new video through combination logic, it is easy to combine the selected fragments in a smooth and meaningful way, so that the narrative of the new video is more coherent, smooth and rhythmic, or the new video can better highlight and highlight the emotional elements, and the audience can feel the emotional and psychological changes of the characters in the new video more deeply, or the overall rhythm of the new video is more compact, powerful and fascinating.

[0046] In one possible implementation, determining a shot segment based on the shot types of multiple image frames includes: setting an observation queue and setting a preset storage length of the observation queue; storing the shot types of the image frames in the observation queue; when the number of shot types stored in the observation queue reaches the preset storage length and the proportion of the first shot type in the observation queue reaches a preset ratio, outputting the first shot type, the first shot type being the shot type with the largest proportion in the observation queue; when the number of shot types stored in the observation queue does not reach the preset storage length, or the number of shot types stored in the observation queue reaches the preset storage length and there is no first shot type with a proportion that reaches a preset ratio, outputting an invalid state; and determining a shot segment based on the invalid state and the first lens type.

[0047] It is understandable that the scene segments may also be determined by setting an observation queue.

[0048] By setting an observation queue to determine the lens segment, it is convenient to accurately determine the start and end times of the lens segment and the lens type of the lens segment.

[0049] In a possible implementation, setting the observation queue includes: setting a corresponding observation queue for each lens type of each dimension, and the storage length of the observation queue corresponding to each lens type of each dimension is consistent. Alternatively, setting a corresponding observation queue and the storage length of the observation queue for each lens type of each dimension, and the storage length of the observation queue corresponding to each lens type of each dimension may be inconsistent.

[0050] By setting corresponding observation queues for lens types of different dimensions, such as setting one observation queue for lens size and another observation queue for lens composition, it is helpful to improve the accurate positioning of lens fragments.

[0051] In one possible implementation, determining a shot segment based on an invalid state and a first shot type includes: determining a start time of the shot segment in response to a transition from outputting the invalid state to outputting the first shot type; determining an end time of the shot segment in response to a transition from outputting the first shot type to outputting the invalid state; and using the first shot type as the shot type of the shot segment.

[0052] In a possible implementation, determining a shot segment based on shot types of multiple image frames includes: comparing shot types of N adjacent image frames in an image stream to obtain a comparison result, where N is an integer greater than or equal to 2; and determining the shot segment based on the comparison result of the N adjacent image frames.

[0053] It can be understood that the scene segment can also be determined by comparing the scene types of N adjacent image frames in the image stream.

[0054] By comparing and determining the scene segment, it is convenient to accurately determine the start time and end time of the scene segment and the scene type of the scene segment.

[0055] In a possible implementation, when N is 2, determining the shot segment according to the comparison result of adjacent N frames of image frames includes: when the comparison result indicates that both the front frame and the rear frame of the adjacent N frames of image frames include the second shot type, outputting a first state, the front frame is an image frame of the adjacent N frames of image frames that is located before the rear frame, and the first state is used to indicate to continue to judge the second shot type in the image stream; when the comparison result indicates that the rear frame of the adjacent N frames of image frames includes the second lens type and the front frame does not include the second lens type, determining the rear frame as the starting frame of the shot segment; when the comparison result indicates that the front frame of the adjacent N frames of image frames includes the second lens type and the rear frame does not include the second lens type, determining the front frame as the ending frame of the shot segment; and using the second lens type as the lens type of the shot segment.

[0056] In a fourth aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the processor and the memory are coupled, and the memory is used to store a computer program. When the computer program is executed by the processor, the electronic device executes any one of the above methods.

[0057] In a fifth aspect, the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute any of the above methods.

[0058] In a sixth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes any one of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0060] Figure 2 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application.

[0061] Figures 3A to 3Q A schematic diagram of a video recording scenario provided in an embodiment of the present application.

[0062] Figure 4 A timing diagram for generating a new video provided in an embodiment of the present application.

[0063] FIG. 5A to FIG. 5C A schematic diagram of a video synthesis scenario provided in an embodiment of the present application.

[0064] FIG. 6A to FIG. 6C A schematic diagram of an automatic video editing scenario provided in an embodiment of the present application.

[0065] FIG. 7A to FIG. 7G A schematic diagram of a video editing scene provided in an embodiment of the present application.

[0066] Figure 8 A flowchart of a video processing method provided in an embodiment of the present application.

[0067] Fig. 9 A flowchart of another video processing method provided in an embodiment of the present application.

[0068] Fig.10 Another timing diagram for generating a new video provided in an embodiment of the present application.

[0069] Fig.11 A schematic flow chart of a method for determining shot segments provided in an embodiment of the present application.

[0070] Fig. 12A A schematic diagram of a method for determining scene segments provided in an embodiment of the present application.

[0071] Fig. 12B A schematic diagram of a method for determining shot segments provided in an embodiment of the present application.

[0072] Fig. 12C A schematic diagram of a method for determining highlight footage provided in an embodiment of the present application.

[0073] Fig.13 A flowchart of another video processing method provided in an embodiment of the present application.

[0074] Fig.14 A module diagram of a video processing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In this article, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprising a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0076] In this application, "at least one" means one or more, and "more than one" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0077] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0078] In the absence of conflict, the following embodiments and features of the embodiments may be combined with each other.

[0079] The present application provides a video processing method, which can automatically and efficiently provide a new video with artistic appeal by processing the original video. The video processing method of the present application analyzes the image stream of the original video (at least 2 frames of original image frames constitute an image sequence) from the perspective of the lens, and can provide clips of different lens types, enrich video materials, optimize video editing effects, and improve user experience. When selecting clips from the image stream, you can focus on the richness of lens types, or you can focus on the combination of scene types and lens types, or you can select the required clips according to the editing rules. When editing the selected clips, it is not simply superimposing the selected clips, but editing the selected clips according to the editing rules, such as combining the selected clips in a smooth and meaningful way to obtain a new video. The editing rules reflect the "editing thinking" of video editing, which can make the narrative of the new video more coherent, smooth, and rhythmic, or make the new video more prominent and prominent in emotional elements, and the audience can also feel the emotional changes and psychological changes of the characters in the new video more deeply, or make the overall rhythm of the new video more compact, powerful and fascinating. For users, videos can be edited automatically, which reduces the requirements on users' video editing ability and improves video editing efficiency. The new videos obtained by editing have a more cinematic feel, more artistic appeal, and can attract more attention from video viewers.

[0080] Specifically, the present application can set a "one-record, multiple-get" mode in an electronic device. After the electronic device enters the "one-record, multiple-get" mode, when recording the original video or when the electronic device edits the recorded original video in the album, the electronic device obtains the image stream of the original video, and determines the lens fragment (or lens tag) and / or scene fragment (or scene tag) by analyzing the image stream. The electronic device can cut out the corresponding lens fragment from the original video according to the determined lens fragment position (or lens tag), or cut out the corresponding scene fragment from the original video according to the determined scene fragment position (or scene tag), or cut out the wonderful lens fragment from the original video according to the editing rules, scene tags (or scene fragments) and lens tags (or lens fragments). The electronic device can also edit the cropped new video fragment according to the editing rules to generate a new video with artistic appeal. Among them, the new video fragment includes one or more of the lens fragment, scene fragment and wonderful lens fragment. In addition, the user can view the shot video, new video fragment and new video in applications such as gallery.

[0081] It can be understood that the term "one record, multiple gains" in this application or on the user interface may also have other names, such as one-click blockbuster, AI one-click blockbuster, one-click multiple gains, one-click multiple shots, one-click output, etc.

[0082] The term "original video" in the embodiments of the present application refers to the video to be processed, which can be a video being shot or a video that has been shot, or a video that has been edited. The shot video can be a video shot by a camera application that comes with the electronic device or a third-party application, or it can be a video downloaded from the Internet.

[0083] The term "shot tag" in the embodiment of the present application includes the shot type of the shot segment and the position of the shot segment. The position of the shot segment indicates the position of the shot segment in the original video, and may include the start time and end time of the shot segment. The start time of the shot segment may be the frame number of the start frame of the shot segment or the timestamp of the start frame (such as the timestamp of the start frame). Correspondingly, the end time of the shot segment may be the frame number of the end frame of the shot segment or the timestamp of the end frame. The shot type may be a type divided based on the film industry, and the specific shot type may refer to the following Table 1.

[0084] The term "scene tag" in the embodiment of the present application includes the scene type of the scene segment and the position of the scene segment. The position of the scene segment indicates the position of the scene segment in the original video, and may include the start time and the end time of the scene segment. Among them, the start time of the scene segment may be the frame number of the start frame of the scene segment or the timestamp of the start frame. Correspondingly, the end time of the scene segment may be the frame number of the end frame of the shot segment or the timestamp of the end frame.

[0085] The term "shot segment" in the embodiment of the present application includes the smallest granularity segment corresponding to a certain shot type. For example, the original video includes 60 frames, of which the 1st to 30th frames shoot close-up shots of person A, the 31st to 40th frames switch from shooting person A to shooting person B, and the 41st to 60th frames shoot medium and close-up shots of person B. Then the 1st to 30th frames are shot segments corresponding to close-up shots, the 41st to 60th frames are shot segments corresponding to medium and close-up shots, and the image frames from the 1st to 60th frames have only a single subject, so the 1st to 60th frames correspond to shot segments of single-subject shots.

[0086] It can be understood that when the electronic device determines a shot segment, the shot tag of the shot segment can be obtained.

[0087] The term "highlight clips" in the embodiments of the present application refers to video clips divided by combining scene types and shot types, such as video clips divided from the original video according to the scene types and shot types indicated by the editing rules.

[0088] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0089] It is understandable that the following Figures 3A to 3Q , FIG. 5A to FIG. 5C , FIG. 6A to FIG. 6C as well as FIG. 7A to FIG. 7G The user interface shown does not constitute a limitation on the corresponding functional interface provided by the embodiment of the present application, especially does not constitute a limitation on the buttons on the interface and the interface layout style. In other embodiments, the interface provided by the present application may also be in other forms, may have more or fewer functional controls, and the corresponding functional controls may be deployed in different interfaces, and the present application does not impose any limitation on this.

[0090] The input acting on the "user interface" in the following embodiments of this application includes but is not limited to: click operation, long press operation, and sliding operation, which are not specifically limited in this application. The input received by the electronic device in the embodiments of this application includes but is not limited to: voice input and the above-mentioned input acting on the "user interface".

[0091] A video processing method provided in an embodiment of the present application is applied to electronic devices. The electronic devices include but are not limited to mobile phones, tablet computers, desktops, laptops, notebook computers, ultra-mobile personal computers (Ultra-mobile Personal Computer, UMPC), handheld computers, netbooks, personal digital assistants (Personal Digital Assistant, PDA), wearable electronic devices, vehicle-mounted electronic devices, augmented reality (Augmented Reality, AR) devices, virtual reality (Virtual Reality, VR) devices, etc. The present application does not specifically limit the specific form of the electronic devices.

[0092] See also Figure 1, exemplarily introduces the hardware structure of an electronic device 100 provided in an embodiment of the present application.

[0093] like Figure 1 As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a display screen 130, a camera 140, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor module 170, etc. The sensor module 170 includes but is not limited to a gyroscope sensor 170A, an acceleration sensor 170B, and a distance sensor 170C.

[0094] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0095] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0096] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0097] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, and / or a universal serial bus (USB) interface, etc.

[0098] The electronic device 100 implements the display function through a GPU, a display screen 130, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 130 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0099] The display screen 130 is used to display images, videos, etc. The display screen 130 includes a display panel. The display panel can be a liquid crystal display (LCD). The display screen panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 130, where N is a positive integer greater than 1.

[0100] The display screen 130 can be used to display Figures 3A to 3Q , FIG. 5A to FIG. 5C , FIG. 6A to FIG. 6C as well as FIG. 7A to FIG. 7G The user interface shown.

[0101] The electronic device 130 can realize the shooting function through the ISP, the camera 140, the video codec, the GPU, the display screen 130 and the application processor.

[0102] The ISP is used to process the data fed back by the camera 140. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 140.

[0103] The camera 140 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 140, where N is a positive integer greater than 1.

[0104] As in Figures 3A to 3Q In the scene, the camera 140 collects images in real time to obtain the image stream of the original video.

[0105] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0106] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0107] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0108] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0109] The internal memory 121 may be used to store computer executable program codes, which may include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area.

[0110] The memory of the electronic device 100 (such as the internal memory 121 or the external memory card) can be used to store a lens type library, a scene type library and a editing rule library, which will be described in detail below.

[0111] The gyro sensor 170A may be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 170A.

[0112] The acceleration sensor 170B can detect the magnitude of acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device.

[0113] The distance sensor 170C is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 170C to measure the distance to the subject to achieve fast focusing.

[0114] In some embodiments, the electronic device 100 may further include an audio module, a speaker, a receiver, a microphone, an earphone interface, and the audio function is implemented by the audio module, the speaker, the receiver, the microphone, the earphone interface, and the application processor, such as recording during video recording.

[0115] It should be understood that Figure 1 The electronic device 100 shown is only an example, and the electronic device 100 may have more Figure 1 The more or less components shown in the figure can be combined with two or more components, or can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0116] See also Figure 2 , which exemplarily introduces the software architecture of the electronic device provided in the embodiments of the present application.

[0117] like Figure 2 As shown, the layered architecture divides the operating system of the electronic device into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. For example, the operating system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the hardware abstraction layer, the kernel layer, and the hardware layer.

[0118] The application layer can include a series of application packages. Application packages can include camera, gallery, etc.

[0119] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0120] Exemplarily, the application framework layer may include a camera access interface and a gallery access interface, wherein the camera access interface may include camera management and camera devices. The camera access interface is used to provide an application programming interface and a programming framework for camera applications. The gallery access interface is used to provide an application programming interface and a programming framework for gallery applications.

[0121] The Hardware Abstraction Layer (HAL) is an interface layer between the operating system kernel and the upper-level software. Its purpose is to abstract the hardware and provide a virtual hardware platform for the operating system. The hardware abstraction layer is an abstract interface driven by the device kernel, which is used to implement the application programming interface that provides access to the underlying device to the higher-level Java API framework. HAL contains multiple library modules, such as camera hardware abstraction, display, Bluetooth, audio, etc. Each library module implements an interface for a specific type of hardware component. When the system framework layer API requires access to the hardware of the portable device, the operating system will load the library module for the hardware component.

[0122] In the embodiment of the present application, the hardware abstraction layer may include a camera algorithm library and a camera hardware abstraction layer.

[0123] The camera algorithm library includes but is not limited to: human body detection algorithm, face detection algorithm, subject detection algorithm, face attribute detection algorithm, scene detection algorithm, transition detection algorithm, saliency detection algorithm, depth detection algorithm, motion area detection algorithm, camera movement recognition algorithm, zoom recognition algorithm and other detection algorithms. Face attribute detection algorithms include but are not limited to: age detection algorithm, expression detection algorithm, gender detection algorithm and appearance detection algorithm.

[0124] In an embodiment of the present application, the camera hardware abstraction layer can perform lens detection on the image stream to determine the lens type. Specifically, the camera hardware abstraction layer can obtain the image stream and transmit the obtained image stream to the camera algorithm library, and obtain the detection result after detecting the image stream from the camera algorithm library, and determine the lens type according to the detection result.

[0125] The human body detection algorithm can be used to perform human body detection. Specifically, the electronic device detects whether the image frame includes a human body through the human body detection algorithm. If a human body is detected, the human body frame is determined, and the key points of the human body are detected based on the human body frame, so as to obtain the human body frame information. The key points of the human body include but are not limited to: head, shoulders, arms, hands, legs and feet. The human body frame information includes but is not limited to: the position of the human body frame and the size of the human body frame.

[0126] For example, the electronic device can use the geometric model of the position of the human body parts to model the human body, and perform human body detection based on the established model. For example, the electronic device can build and train models for identifying multiple body parts such as the neck, shoulders, limbs, etc., thereby obtaining a universal human body recognition model.

[0127] The face detection algorithm can be used to perform face detection. Specifically, the electronic device detects whether the image frame includes a face through the face detection algorithm. If a face is detected, the face frame is determined, and the key points of the face, the face posture, etc. are detected based on the face frame, so as to obtain the face frame information. The key points of the face include but are not limited to: eyes, nose, and mouth. The face frame information includes but is not limited to: the position of the face frame, the size of the face frame, and the face Euler angle.

[0128] The face detection algorithm includes a Viola-Jones cascade classifier algorithm based on Haar features. Other methods may also be used to detect faces, which is not limited in this application.

[0129] The subject detection algorithm can be used to determine the subject category in the image frame, such as human body, cat, dog, vehicle, etc., and assign a unique identity document (ID) to the subject. The subject detection algorithm can also determine the total number of subjects in the image frame based on the determined subject.

[0130] The scene detection algorithm is used to identify the scene type of each image frame in the image stream. Exemplarily, the electronic device can use a trained scene classification model to identify the scene type of the image frame in the image stream. Among them, the training of the scene classification model may include: pre-establishing a data set through a large amount of image data with scene types labeled. Then, the data set is input into the scene classification model to train the scene classification model. Among them, the scene classification model can use a neural network, for example, it can be a convolutional neural network, a fully convolutional neural network, a deep neural network, a BP neural network, etc.

[0131] The scene transition detection algorithm is used to determine the scene transition position and scene transition category in the image stream. The scene transition category may include image zoom, fast camera movement, and video subject transition. For example, the video subject transition can be specifically divided into: the video subject changes from scenery to people, people to scenery, people to food, food to people, people to ancient buildings, ancient buildings to scenery, etc.

[0132] The electronic device can use the trained transition recognition model to identify the transition position and transition category of the scene transition in the image stream. The training of the transition recognition model may include: pre-establishing a data set through a large number of image streams with transition positions and transition categories marked. Then, the data set is input into the transition recognition model to train the transition recognition model. The transition recognition model may use a neural network, for example, a 3D convolutional neural network, etc.

[0133] The saliency detection algorithm is used to detect the object area that can most attract the visual attention of the human eye from the image. The saliency detection algorithm may include at least two of frequency tuning (Frequency Tuned, FT), histogram contrast (Histogram-based contrast, HC), local cues (Local Cues, LC) and spectral residual (Spectral Residual, SR).

[0134] In some embodiments, based on the existing convolutional neural network architecture, the convolutional neural network architecture is fine-tuned to achieve salient object detection for the image. For example, a trained salient detection algorithm can be used to identify salient regions of each image frame in the image stream. The salient detection algorithm can use a neural network, for example, a convolutional neural network, etc. The training of the model can refer to the above, and will not be repeated here.

[0135] The depth detection algorithm is used to predict the depth of the image to obtain the depth information of the image, and the depth information includes but is not limited to: a depth map. A depth map refers to an image with pixel depth. Pixel depth is used to represent the vertical distance between the plane of the image and the object represented by the pixel (i.e., the shooting distance). Exemplarily, a trained depth detection algorithm can be used to predict the depth of each image frame in the image stream to obtain the depth information of the image frame. The depth detection algorithm can use a neural network, for example, it can be a convolutional neural network, etc. The training of the model can refer to the above and will not be repeated here.

[0136] The motion region detection algorithm is used to detect motion regions in an image. Exemplarily, a trained motion region detection algorithm can be used to perform motion detection on each image frame in an image stream to determine whether the image frame has a motion region. If there is a motion region, the motion region is tracked to obtain information about the motion region, such as position, size, etc. The motion region detection algorithm can use a neural network, for example, a convolutional neural network, etc. The training of the model can refer to the above and will not be repeated here. In some embodiments, the motion region detection algorithm includes but is not limited to: a background subtraction algorithm, a multi-frame difference algorithm, and an optical flow algorithm.

[0137] The camera movement recognition algorithm can be used to identify the camera movement type, and the identified camera movement types include but are not limited to: push shot, pull shot, pan shot, tilt-shift shot, lift shot, and lower shot. Exemplarily, the posture change of the electronic device can be determined based on the data of the sensor in the electronic device (such as the gyroscope sensor 180B and the acceleration sensor 170B), and then the push-pull shot, pan shot, or tilt-shift shot can be determined based on the posture change. In other embodiments, a trained camera movement detection algorithm can be used to detect multiple image frames in the image stream to determine the camera movement type. The camera movement detection algorithm can use a neural network, for example, it can be a convolutional neural network, etc. The training of the model can refer to the above and will not be repeated here.

[0138] The zoom recognition algorithm is used to determine the zoom recognition result of the image stream. The zoom recognition result includes whether the zoom is changed and the change of focal length (such as from 1.0 to 5.0, or from 5.0 to 1.0). Exemplarily, the zoom recognition result can be determined according to the zoom-related parameter settings input by the user, such as the setting of the zoom ratio control 326 described below. For another example, a trained zoom recognition algorithm can be used to detect multiple image frames in the image stream to determine the zoom recognition result. The zoom recognition algorithm can use a neural network, for example, it can be a convolutional neural network, etc. The training of the model can refer to the above and will not be repeated here.

[0139] The age detection algorithm is used to detect the age of the subject in the image frame and determine the age attribute. For example, the electronic device can identify the age attribute corresponding to the face image based on a face age estimation algorithm that integrates local binarization pattern (LBP) and histogram of gradient hog (HOG) features.

[0140] The expression detection algorithm is used to detect the expression of the human subject in the image frame and determine the expression attribute. The expression detection algorithm can perform expression recognition based on the human frame information and / or the face frame information.

[0141] In some embodiments, the gender of the subject can be identified by a trained gender detection algorithm. The face value of the subject can be identified by a trained face value detection algorithm. The models corresponding to the gender detection algorithm and the face value detection algorithm can use a neural network, for example, a convolutional neural network, etc. The training of the model can refer to the above, and will not be repeated here.

[0142] It can be understood that the specific implementation method of each detection algorithm in the above-mentioned camera algorithm library can also refer to the existing technology, and this application does not make any specific limitation on this.

[0143] In some embodiments, in order to improve the detection speed of the image stream of the original video, before inputting the image stream into the camera algorithm library or detecting the image stream through the detection algorithm of the camera algorithm library, the electronic device may preprocess the image stream first, and then detect the preprocessed image stream through the detection algorithm in the camera algorithm library. Preprocessing includes but is not limited to: reducing the resolution of the image stream, sampling the image stream at intervals, reducing the resolution of the image stream and performing interval sampling, etc.

[0144] Wherein, the image stream may be processed by reducing the resolution: the electronic device first processes the image stream of the original video by reducing the resolution (for example, reducing the resolution from 4K to 640*480) to obtain a low-definition image stream. Then, the low-definition image stream is input into the corresponding detection algorithm. Exemplarily, in order to improve the recognition speed of the transition position and transition category of the scene conversion in the image stream. Before the electronic device inputs the image stream into the transition recognition model, the image stream may be processed by reducing the resolution (for example, reducing the resolution from 4K to 640*480) to obtain a low-definition image stream. Then, the low-definition image stream is input into the transition recognition model, and transition detection is performed to identify the transition position and transition category in the low-definition image stream. The electronic device can determine the corresponding transition position and transition category in the image stream of the original video according to the transition position and transition category in the low-definition image stream.

[0145] Wherein, performing interval sampling on the image stream may include: the electronic device may perform interval sampling on the image stream of the original video before inputting the image stream of the original video into the scene classification model (for example, taking 1 frame out of every 3 frames) to obtain a sampled image stream, and record the frame number of the sampled image frame in the sampled image stream in the image stream of the original video, and input the sampled image stream into the neural network classification model to identify the scene type of each sampled image frame in the sampled image stream. After identifying the scene type of each sampled image frame in the sampled image stream, the electronic device may, based on the scene type and frame number of the sampled image frame, mark multiple image frames in the image stream of the original video that have the same frame number as the sampled image frame and are adjacent to the sampled image frame as the scene type corresponding to the sampled image frame. For example, the electronic device may take 1 image frame out of every 3 image frames in the image stream of the original video as the sampled image frame. Wherein, the image frame No. 77 in the image stream of the original video is the sampled image frame, and the scene type of the sampled image frame with the frame number 77 is "character". Then, the electronic device may mark the scene types of the 77th image frame, the 76th image frame, and the 78th image frame in the image stream of the original video as “person”.

[0146] Among them, reducing the resolution of the image stream and performing interval sampling includes: the electronic device first reduces the resolution of the image stream of the original video, then performs interval sampling on the image stream with reduced resolution, and inputs the sampled image stream with reduced resolution into the corresponding detection algorithm.

[0147] The electronic device can analyze the lens type from four dimensions: lens size, lens movement (i.e., camera movement), lens composition, and lens angle and position. The lens size can indicate the field of view of the lens used during shooting. The lens movement can indicate the movement form of the lens during shooting. The lens composition can indicate the layout and structure of the picture during shooting. The angle and position of the lens can indicate the spatial relationship between the lens and the subject during shooting. For example, the angle of the lens refers to the geometric angle formed by the camera and the subject during shooting.

[0148] It is understandable that the electronic device can also analyze the shot type from other dimensions, which is not specifically limited in this application. For example, the shot type can also be divided into long shots or short shots according to the shot duration, where the duration division of long shots and short shots can be set according to actual conditions.

[0149] Taking the analysis of lens types from four dimensions of lens size, lens movement, lens composition, and lens angle and position as an example, please refer to Table 1 for an exemplary introduction to lens types that can be determined by the camera hardware abstraction layer.

[0150] Table 1

[0151]

[0152]

[0153] It can be understood that the lens types in the dimension of lens size include but are not limited to: panoramic lenses to close-up lenses in the first column of Table 1, the lens types in the dimension of lens movement include but are not limited to: fixed lenses to panning lenses in the second column of Table 1, the lens types in the dimension of lens angle and position include but are not limited to: ground plane height lenses to oblique angle lenses in the third column of Table 1, and the lens types in the dimension of lens composition include but are not limited to: establishing shots to insert shots in the fourth column of Table 1.

[0154] The electronic device may pre-build a lens type library, and may build the lens types included in the lens type library according to actual conditions. The lens types shown in Table 1 are only examples, and the lens types of the present application are not limited to those shown in Table 1.

[0155] In an embodiment of the present application, the camera hardware abstraction layer may include a lens recognition module. The lens recognition module is used to perform lens detection on the image stream of the original video and determine the lens type of each image frame in the image stream. For example, the lens recognition module can determine the lens type based on the detection result output by the above detection algorithm, and / or determine the lens type based on the sensor data of the electronic device (such as the gyroscope sensor 170A, the acceleration sensor 170B, and the distance sensor 170C), and / or determine the lens type based on the user's related operations on the lens adjustment (such as the setting of the zoom ratio control 326).

[0156] The lens recognition module can identify the lens type included in the image frame according to the lens type library. Exemplarily, taking the camera hardware abstraction layer analyzing the lens type from four dimensions of lens size, lens movement, lens composition, and lens angle and position as an example, the lens recognition module can include a lens size recognition unit, a lens movement recognition unit, a lens angle position recognition unit, and a lens composition recognition unit. The lens size recognition unit is used to identify the lens type related to the lens size, the lens movement recognition unit is used to identify the lens type related to the lens movement, the lens angle position recognition unit is used to identify the lens type related to the lens angle and position, and the lens composition recognition unit is used to identify the lens type related to the lens composition.

[0157] Taking Table 1 as an example, for a certain image frame A, the lens size recognition unit can determine that the lens type of image frame A is any lens type in the first column of Table 1, the lens movement recognition unit can determine that the lens type of image frame A is any lens type in the second column of Table 1, the lens angle position recognition unit can determine that the lens type of image frame A is any lens type in the third column of Table 1, and the lens composition recognition unit can determine that the lens type of image frame A is any lens type in the fourth column of Table 1.

[0158] Please refer to Table 2, which exemplarily introduces how the above four recognition units determine the lens type.

[0159] Table 2

[0160]

[0161]

[0162] As shown in Table 2, the camera hardware abstraction layer (lens size recognition unit) can perform lens size recognition according to one or more data of the scene detection result of the scene detection algorithm, the face detection result and face key points of the face detection algorithm, the body detection result and body key points of the body detection algorithm, the depth map (such as the shooting distance result) of the depth detection algorithm, and the salient target detection result of the saliency detection algorithm, thereby obtaining the lens type related to the lens size.

[0163] Exemplarily, the lens size recognition unit may determine the lens type according to the proportion of the human body frame and / or the face frame in the picture, such as if the proportion exceeds a preset proportion value, it is a close-up shot, etc. The lens type may also be determined according to the depth detection algorithm or the shooting distance obtained by the distance sensor 170C, such as if the shooting distance exceeds a preset distance threshold, it is a long-shot shot, etc. The lens type may also be determined according to the proportion of the salient frame of the salient area in the picture. The lens type may also be determined according to the identified specific scene type, such as if it is identified that the image frame includes a sky and white cloud scene, then the lens type of the image frame may be determined to be a long-shot shot.

[0164] The camera hardware abstraction layer (lens angle position recognition unit) can recognize the lens angle according to the face Euler angle of the face detection algorithm and the human body key points, camera internal parameters and external parameters of the human body detection algorithm, thereby obtaining information related to the lens angle. It can also obtain information related to the lens position according to the shooting distance. According to one or both of the information related to the lens angle and the information related to the lens position, the lens type related to the lens angle position can be obtained.

[0165] The camera hardware abstraction layer (camera movement recognition unit) can perform lens motion recognition based on the motion area detection result (such as motion area information) output by the motion area detection algorithm, the camera movement recognition result output by the camera movement recognition algorithm, and the zoom recognition result output by the zoom recognition algorithm, thereby obtaining the lens type related to the camera movement.

[0166] Exemplarily, the camera movement recognition unit can determine the zoom recognition result according to the zoom-related parameter settings input by the user to the electronic device, and then determine the lens type according to the zoom recognition result. If the zoom recognition result is that the focal length is continuously increasing, the lens type is determined to be a push lens. The zoom result can also be determined according to the image zooming and the lens push-pull. The posture change of the electronic device can also be determined according to the data of the sensor of the electronic device, and then the camera movement recognition result is obtained according to the posture change.

[0167] The camera hardware abstraction layer (lens composition recognition unit) can perform lens composition recognition based on the scene detection results of the scene detection algorithm, the face detection results of the face detection algorithm, and the body detection results of the body detection algorithm, thereby obtaining the lens type related to the lens composition.

[0168] The above-mentioned lens size recognition unit recognizes the lens size, the lens movement recognition unit recognizes the lens movement, the lens angle position recognition unit recognizes the lens size and angle, and the lens composition recognition unit recognizes the lens composition. The specific implementation method can be set according to actual conditions, and this application does not make any specific limitations on this.

[0169] It can be understood that the lens type identification basis shown in Table 2 is only an example. The corresponding information can be obtained according to the actual situation to further identify the corresponding lens type. The information includes but is not limited to the detection results of the detection algorithm, and this application does not make any specific limitations on this.

[0170] In an embodiment of the present application, the camera hardware abstraction layer can determine the lens segment according to the lens type of multiple image frames, and then obtain the corresponding lens tag.

[0171] For example, the shot size recognition unit can determine the shot segment related to the shot size in the image stream according to the shot type related to the shot size of a plurality of consecutive image frames, and then obtain the corresponding shot tag according to the starting time, ending time and the shot type related to the shot size in the shot segment. Correspondingly, the camera movement recognition unit can determine the shot segment related to the shot motion in the image stream according to the shot type related to the lens motion of a plurality of consecutive image frames, and then obtain the corresponding shot tag according to the starting time, ending time and the shot type related to the lens motion in the shot segment. The shot composition recognition unit can determine the shot segment related to the shot composition in the image stream according to the shot type related to the lens composition of a plurality of consecutive image frames, and then obtain the corresponding shot tag according to the starting time, ending time and the shot type related to the lens composition in the shot segment. The lens angle position recognition unit can determine the shot segment related to the lens position angle in the image stream according to the lens type related to the lens position angle of a plurality of consecutive image frames, and then obtain the corresponding shot tag according to the starting time, ending time and the shot type related to the lens position angle in the shot segment.

[0172] In some other embodiments, the camera hardware abstraction layer may include a label module (not shown). The label module may determine the lens label according to the lens type of the multiple image frames. The label module may set a corresponding label unit according to the lens recognition module, such as setting a first label unit for the lens size recognition unit, the first label unit is used to determine the lens label related to the lens type and the lens size. The first label unit determines the lens label according to a plurality of consecutive image frames with the same lens type identified by the lens size recognition unit. A second label unit is set for the camera movement recognition unit, the second label unit is used to determine the lens label related to the lens type and the lens movement. The second label unit determines the lens label according to a plurality of consecutive image frames with the same lens type identified by the camera movement recognition unit. A third label unit is set for the lens composition recognition unit, the third label unit is used to determine the lens label related to the lens type and the lens composition. The third label unit determines the lens label according to a plurality of consecutive image frames with the same lens type identified by the lens composition recognition unit. A fourth label unit is set for the lens angle position recognition unit, the fourth label unit is used to determine the lens label related to the lens type and the lens position angle. The fourth label unit determines the lens label according to a plurality of consecutive image frames with the same lens type identified by the lens angle position recognition unit.

[0173] In the embodiment of the present application, the image frames in the shot segment may include one or more shot types. For example, in the example corresponding to the above shot segment, the first frame to the 30th frame are close-up shot segments, and the image frames in the shot segment include two shot types: close-up and single-body.

[0174] In an embodiment of the present application, the shot recognition module can determine the shot score of each image frame and can also determine the shot score of each shot segment. The shot score can be set according to the shot type included in the image frame. For example, the more shot types an image frame includes, the greater its shot score. For example, for image frame A, the shot size recognition unit determines that the shot type is a close-up, and the shot composition recognition unit determines that its shot type is a single subject. For image frame B, the shot size recognition unit determines that the shot type is a long shot. The shot recognition module (or label module) can obtain the shot scores of image frames A and B based on the above four recognition units. Based on the fact that image A includes more shot types than image frame B, the shot score of image frame A is higher than that of image frame B. When determining the shot score of a shot segment, the shot scores of all image frames in the shot segment can be calculated and then averaged, or the shot score of the shot segment can be calculated according to other methods.

[0175] In some embodiments, the shot tag may also include a shot score, such as the score of a shot segment, or the shot scores of all image frames in the shot segment.

[0176] In an embodiment of the present application, the camera hardware abstraction layer can perform scene detection on the image stream to determine the scene type. Specifically, the camera hardware abstraction layer can obtain the image stream and transmit the obtained image stream to the camera algorithm library, and obtain the detection result after detecting the image stream from the camera algorithm library, and determine the scene type according to the detection result.

[0177] In an embodiment of the present application, the camera hardware abstraction layer includes a scene recognition module. The scene recognition module can perform scene detection and determine scene segments, and then output scene tags according to the scene segments, such as outputting scene tags of scene segments according to the scene type of each image frame and the transition position and transition category of the scene transition. Scene types include but are not limited to people, scenery, food, Spring Festival, Christmas, architecture, beach, fireworks, plants, snow scene or travel.

[0178] The electronic device may construct a scene type library in advance, and may construct the scene types included in the scene type library according to actual conditions. The scene recognition module recognizes the scene type included in the image frame according to the scene type library.

[0179] In some embodiments, the scene type constructed by the electronic device may include one or more categories, taking a birthday scene and a gourmet cooking scene as examples, as shown in Table 3 below.

[0180] Table 3

[0181]

[0182] The first category in Table 3 includes birthday scenes and food cooking. The scene types of birthday scenes can be further subdivided into singing birthday songs, wearing birthday hats, blowing candles, cheering, and eating cakes. The scene types of food cooking can be further subdivided into preparing food, cooking, plating, and film display.

[0183] In the embodiment of the present application, the scene recognition module can determine the scene score of each image frame and can also determine the scene score of each scene segment. For a certain image frame, the camera hardware abstraction layer can determine its scene score and shot score. The scene score is a score automatically output during scene detection, which can include the score of facial expression, the score of action detection, and the score of image quality evaluation.

[0184] In some embodiments, the scene tag may also include a scene score, such as a score of a scene segment, or a scene score of all image frames in the scene segment. When determining the shot score of a scene segment, the scene score of all image frames in the scene segment may be calculated and then averaged, or the scene score of the scene segment may be calculated in other ways.

[0185] In an embodiment of the present application, the electronic device may pre-store editing rules. Different editing rules may be set in advance according to different editing styles. The editing rules may indicate the required new video segment (i.e., the required segment). For example, the editing rules may indicate the shot type of the required segment, and the electronic device may cut out the segment of the shot type indicated by the editing rules from the image stream. The editing rules may indicate the scene type of the required segment, and the electronic device may cut out the segment of the scene type indicated by the editing rules from the image stream. The editing rules may indicate the shot type and scene type of the required segment, and the electronic device may cut out the wonderful shot segment from the image stream according to the indicated shot type and scene type.

[0186] In the embodiment of the present application, the editing rule may also indicate the combination logic of the required segments. The combination logic is used to indicate the positional relationship of each required segment in the new video. When generating a new video according to the editing rule, the sequence and splicing relationship between the required segments are determined according to the combination logic, and then the positional relationship of each required segment in the new video can be determined.

[0187] In the embodiment of the present application, the editing rule may also indicate the duration of the required segment, such as indicating the duration of segments of different shot types, and / or the duration of segments of different scene types, and / or the duration of segments of different wonderful shots. The electronic device may cut out a new video segment of the corresponding duration according to the editing rule.

[0188] In some embodiments, the editing rule may also indicate the total duration of the new video. The electronic device may edit a new video having a duration equal to the total duration indicated by the editing rule, that is, the sum of the durations of the new video segments constituting the new video is the total duration.

[0189] In some embodiments, the editing rules may also indicate the lens material corresponding to the editing style. The lens material may include lens clips of various lens types, such as establishing lens clips, aerial lens clips, etc. The lens material of the lens type corresponding to the editing style may be added to the editing style according to actual conditions.

[0190] In some embodiments, the editing rules may also indicate the sound material corresponding to the editing style. The sound material includes the sound used in the scene type, such as the soundtrack for the birthday party scene is a birthday song.

[0191] In some embodiments, the editing rules may also indicate special effect materials, including but not limited to special effects such as beautification, filters, background replacement, or fade-in and fade-out.

[0192] In some embodiments, the editing rules may also indicate how to add special effect materials, such as placing a fade-in and fade-out special effect at the beginning of a new video.

[0193] In an embodiment of the present application, in addition to setting corresponding editing rules according to different editing styles, the electronic device can also pre-store default editing rules. The default editing rule is based on the type of shots in the original video, and can be used to indicate the acquisition and editing of shot segments in the original video. The combination logic of the default editing rule includes but is not limited to: combining shot segments in chronological order, or combining shot segments with large differences in shot types. The default editing rule is used to make the generated new video reflect more different shot types.

[0194] It can be understood that the default editing rules may also include corresponding duration, total duration, music, special effects materials, and methods of adding special effects materials.

[0195] It is understandable that the electronic device can pre-build an editing rule library, so that when building a lens type library, it can ensure that the lens type library includes the lens type indicated by each editing rule in the editing rule library, and when building a scene type library, it can ensure that the scene type library includes the scene type indicated by each editing rule in the editing rule library. The editing rule library can include editing rules corresponding to each editing style, and the editing styles included in the editing rule library include but are not limited to: editing styles related to scenes, editing styles related to directors, and editing styles related to film and television works.

[0196] In some embodiments, the editing rule library may also include the above-mentioned sound materials, lens materials and special effect materials.

[0197] In an embodiment of the present application, the camera hardware abstraction layer includes a film editing module. The film editing module obtains the lens type of the image frame output by the lens recognition module, and obtains the scene type of the image frame output by the scene recognition module. The film editing module is used to divide new video clips, such as dividing lens clips from the image stream according to the lens types of multiple image frames, and / or dividing scene clips from the image stream according to the scene types of multiple image frames, and / or dividing wonderful lens clips from the image stream according to editing rules, lens tags (lens clips) and scene tags (scene clips). The film editing module is also used to combine new video clips according to corresponding combination logic to obtain new videos.

[0198] The film editing module can obtain the currently selected target editing rules from the memory storing the editing rules according to the instructions issued by the application, or the system can automatically match to obtain the editing rules with the highest matching degree from the memory storing the editing rules, or call the default editing rules. The present application does not specifically limit the way in which the electronic device obtains the editing rules.

[0199] The film editing module can also add one or more of the above-mentioned sound materials, lens materials and special effects materials to the new video according to actual conditions.

[0200] In some embodiments, the camera algorithm library may be integrated into the camera hardware abstraction layer.

[0201] In some embodiments, the camera hardware abstraction layer may provide virtual hardware for camera device 1 (first camera) and camera device 2 (second camera).

[0202] The kernel layer is the layer between hardware and software. The kernel layer includes drivers for various hardware.

[0203] In some embodiments, the kernel layer may include a camera device driver, a digital signal processor driver, an image processor driver, etc. The hardware layer includes a sensor, an image signal processor, a digital signal processor, and a graphics processor.

[0204] The camera device driver is used to drive the camera sensor to collect images and drive the image signal processor to pre-process the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0205] In some embodiments, the hardware abstraction layer may further include an audio hardware abstraction layer, and the audio hardware abstraction layer includes an audio input unit. Accordingly, the kernel layer also includes an audio driver, and the hardware layer includes a microphone.

[0206] Taking video recording as an example, the video processing method in the embodiment of the present application is specifically described in combination with the above hardware structure and system structure:

[0207] Step 1: The electronic device starts the video recording function to obtain the image stream and audio stream.

[0208] Step 1 is performed continuously. In response to the user's input (such as a click operation) of the recording start control on the shooting interface, the camera application calls the camera access interface of the application framework layer, starts the camera application, and then sends an instruction to start recording by calling the camera device 1 (first camera) in the camera hardware abstraction layer. The camera hardware abstraction layer sends the instruction to the camera device driver of the kernel layer. The camera device driver starts the sensor (sensor 1) of the first camera according to the instruction, and collects the image light signal through sensor 1. The camera device driver transmits the image light signal to the image signal processor for preprocessing to obtain an image stream, and then transmits the image stream to the camera hardware abstraction layer through the camera device driver. The camera application can also send an instruction to start recording to the audio driver of the kernel layer through the audio input unit in the audio hardware abstraction layer. According to the instruction, the audio driver can start the microphone to collect audio signals and obtain an audio stream.

[0209] Step 2: The electronic device detects the image frames in the image stream to determine the shot segments and / or the highlight shot segments.

[0210] Step 2 is continuous. The camera hardware abstraction layer can transmit the image stream to the camera algorithm library. The camera algorithm library calls the relevant detection algorithm inside it to detect and process the image stream to obtain the detection result. The camera hardware abstraction layer outputs the detection result to the lens recognition module. The lens recognition module can determine the lens segment and / or output the lens tag based on the detection result, the user's input information, and the data of the electronic device sensor. The camera hardware abstraction layer outputs the detection result to the scene recognition module. The scene recognition module processes the detection result, determines the scene segment and / or outputs the scene tag. The camera hardware abstraction layer determines the wonderful lens segment based on the editing rules, the lens tag (or the position of the lens segment) and the scene tag (or the position of the scene segment), such as determining the position of the wonderful lens segment in the original video.

[0211] Step 3: The electronic device mixes the image stream and the audio stream into the original video.

[0212] With the support of digital signal processors and image processors, the image stream and audio stream can be mixed into the original video based on the same time track.

[0213] Step 4: The electronic device may extract multiple new video segments from the video based on the positions of the new video segments. The electronic device may also combine the new video segments into a new video.

[0214] The camera algorithm library can call the editing algorithm to extract multiple lens clips from the image stream based on the position of the lens clips, and / or extract multiple scene clips from the image stream based on the position of the scene clips, and / or extract multiple wonderful lens clips from the image stream based on the position of the wonderful lens clips.

[0215] The camera algorithm library may also call a fusion algorithm to fuse multiple shot clips and / or multiple highlight shot clips and / or multiple scene clips into a new video.

[0216] Step 5, the electronic device can save one or more of the footage clips, the highlight footage clips, the new video and the original video.

[0217] The camera hardware abstraction layer can save one or more of the lens clips, the wonderful lens clips, the new video and the original video. The saved wonderful lens clips, the new video and the original video can be used for video editing to optimize the video editing effect.

[0218] The following takes the electronic device 100 as a mobile phone as an example to exemplarily introduce the application scenario of the video processing method provided in the embodiment of the present application.

[0219] Recording video scenes

[0220] During the process of recording a video by the electronic device 100, the electronic device 100 can automatically identify the wonderful shots and / or shots and / or scenes in the original video. The electronic device 100 can save the shot tags and / or scene tags in the original video in response to the user's input or at the end of recording the video (such as when the user clicks the recording end control). The electronic device 100 can also automatically crop the wonderful shots and / or shots and / or scenes in the original video in response to the user's input or at the end of recording the video, and present the cropped clips to the user. For details, please refer to FIG. 5A to FIG. 5C The electronic device 100 can also automatically merge the cropped multiple new video segments into a new video according to the editing rules in response to the user's input or at the end of recording the video, and present the new video to the user. For details, please refer to FIG. 6A to FIG. 6C After recording a video, the electronic device 100 can save the original video, shot tags, scene tags, shot clips, wonderful shot clips and new videos.

[0221] The user can instruct the electronic device 100 to open the camera application by touching a specific control on the display screen 130 of the electronic device 100, pressing a specific physical button or button combination, inputting voice, performing air gestures, etc. Figure 3AThe electronic device 100 may display a desktop 310 , which may display a plurality of application icons, such as a camera application icon 311 and a gallery application icon 312 .

[0222] like Figure 3A As shown, the electronic device 100 receives an input (eg, a click operation) from a user on the camera application icon 311, and in response to the input, the electronic device 100 may display a Figure 3B The shooting interface 320 is shown.

[0223] The shooting interface 320 may include an echo control 321, a shooting control 322, a camera conversion control 323, a preview box, a setting control 325, a zoom ratio control 326, and one or more shooting mode controls (e.g., a "night scene mode" control 327A, a "portrait shooting mode" control 327B, a "large aperture mode" control 327C, a "photographing mode" control 327D, a "video recording mode" control 327E, a "professional mode" control 327F, and a more mode control 327G). Among them, the preview box is used to display the image stream currently collected by the camera 140 of the electronic device 100. If a preview screen 324 is displayed in the preview box, the preview screen 324 is an image frame in the image stream currently collected by the camera 140 of the electronic device 100. The echo control 321 can be used to display the image that has been taken. The shooting control 322 can be used to trigger the saving of the image captured by the camera. The camera conversion control 323 can be used to switch the camera for shooting. The setting control 325 can be used to set the camera function. The zoom ratio control 326 can be used to set the zoom ratio of the camera. The shooting mode control can be used to trigger the image processing process corresponding to the shooting mode. For example, the "night scene mode" control 327A can be used to trigger the increase of brightness and color richness in the captured image. The "portrait mode" control 327B can be used to trigger the beautification of the portrait in the captured image.

[0224] like Figure 3B As shown, the shooting interface 320 indicates that the shooting mode currently selected by the user is "photographing mode". The electronic device 100 receives the input of the user acting on the "recording mode" control 327E, and in response to the input, the electronic device 100 switches from the "photographing mode" to the "recording mode". The electronic device 100 may display the following Figure 3C The video preview interface 330 is shown.

[0225] The difference between the video preview interface 330 and the shooting interface 320 is that the video preview interface 330 replaces the shooting control 322 with a recording start control 331. The preview screen 332 displayed on the video preview interface 330 is the image frame in the image stream currently captured by the camera 140 of the electronic device 100, and the preview screen 332 can be the same as or different from the preview screen 324.

[0226] like Figure 3C As shown, the electronic device 100 receives the input (ie, the sixth input) from the user on the setting control 325, and in response to the input, the electronic device 100 displays the following Figure 3D The settings interface 340 is shown.

[0227] The setting interface 340 displays a "record one get multiple results" option 341, an editing style option 342, and a return control 343. When the electronic device 100 receives the user's input, such as the seventh input, the "record one get multiple results" option 341 is turned on according to the seventh input. For example, the setting interface 340 indicates that the "record one get multiple results" option 341 is turned on. Below the "record one get multiple results" option 341, a prompt "If AI recognizes a wonderful clip, you can create a blockbuster with one click" may be prompted. In response to the "record one get multiple results" option 341 being turned on, the electronic device 100 enters the "record one get multiple results" mode. When the electronic device 100 is in the recording mode, it will automatically adopt the video processing method provided by the embodiment of the present application, such as intelligently identifying new video clips in the original video when recording a video, especially lens clips and / or wonderful lens clips. The new video clips are used to create a blockbuster with one click later, such as automatically combining the identified new video clips according to the editing rules to generate a new video. Of course, the user can also control the electronic device 100 to exit the "record one get multiple results" mode by turning off the "record one get multiple results" option 341. The return control 343 is used to instruct the electronic device 100 to exit the setting interface 340 and return to the previous interface. The editing style option 342 is used for the user to select an editing style. The editing styles that can be selected by the user include editing styles in the editing rule library. When the user does not select an editing style, such as when the user does not trigger the editing style option 342 and selects the following editing options, the default is to automatically match the editing style according to the original video shot by the user. Figure 3D The editing style option 342 in the setting interface 340 indicates that the editing style is currently automatically matched.

[0228] It is understood that the sixth input method is not limited to the above, and the editing style option 342 can be displayed in other ways, and the sixth input can also be used to instruct the electronic device 100 to display the editing style option 342 during video recording. The seventh input method is not limited to the above, and the electronic device 100 can be instructed to enter the one-record-multiple-receive mode in other ways, and the seventh input can also be used to instruct the electronic device 100 to enter the one-record-multiple-receive mode during video recording.

[0229] In some embodiments, the setting interface 340 may also include controls for setting the photo ratio, controls for voice-controlled photo taking, and controls for image frame rate setting, which are not specifically limited in this application.

[0230] In some embodiments, when the electronic device 100 enters the video recording mode, the electronic device 100 may enter the one-record-multiple-mode by default, and in response to the input indicating the start of recording the original video (i.e., the third input), the electronic device 100 enters the one-record-multiple-mode. The user may also control the electronic device 100 to enter the one-record-multiple-mode in other ways, which is not specifically limited in this application.

[0231] like Figure 3D As shown, the electronic device 100 may receive an input (ie, a fifth input) from a user acting on the editing style option 342, and in response to the input, the electronic device 100 displays Figure 3E Editing style interface 400 is shown.

[0232] It can be understood that the fifth input method is not limited to the above, and the editing style interface 400 can also be displayed in other ways.

[0233] The electronic device can set the editing style interface 400 according to the editing rule library, so that the editing style interface 400 includes each editing style in the editing rule library. The editing style interface 400 can provide editing styles related to scenes, editing styles related to directors, and editing styles related to film and television works, etc., for users to select the desired editing style. Film and television works include but are not limited to movie categories. The editing style interface 400 includes editing options for indicating the corresponding editing styles and a return option 409.

[0234] Take the three aspects of scene, director and movie as an example to classify editing styles. Figure 3E As shown, the editing options corresponding to the editing styles related to the scenes include editing options 401, 402, and 403. Editing option 401 is used to indicate the editing style of the birthday party scene. Editing option 402 is used to indicate the editing style of the sports scene. Editing option 403 is used to indicate the editing style of the wedding scene. Editing option 404 is used to indicate the editing style of the scenery punch-in. The editing options corresponding to the editing styles related to the directors include editing options 405 and 406. Editing option 405 is used to indicate the editing style of director Lucy. Editing option 406 is used to indicate the editing style of director Donny. The editing options corresponding to the editing styles related to the movies include editing options 407 and 408. Editing option 407 is used to indicate the editing style of movie A. Editing option 408 is used to indicate the editing style of movie B.

[0235] Figure 3EIn the editing style interface 400 shown, the editing option 401 is turned on, and the editing option 401 is the target editing option, and it is determined that the currently selected target editing style is birthday party. In response to the editing option 401 being turned on, the electronic device can obtain the editing rules corresponding to the target editing style of birthday party, and the editing rules corresponding to birthday party are the target editing rules.

[0236] Please refer to Table 4 for an example of the editing rules corresponding to the birthday party.

[0237] Table 4

[0238]

[0239]

[0240] The scene types shown in Table 4 include food and birthday scenes, among which wearing birthday hats, blowing candles and cheering can be understood as further subdivisions of the birthday scene type, that is, the division of events in the birthday scene. The shot types include close-up, medium close-up single subject, and medium close-up multiple subjects.

[0241] The combination logic of the target editing rules can be represented by the scene types and the sequence of shot types indicated in Table 4. For example, the order of the rows in Table 4 indicates the sequence of scene types, that is, the sequence is: food, wearing birthday hats, blowing candles, and cheering. The sequence of different shot types in the same row indicates the sequence of shot types, such as in the scene of wearing birthday hats, the single subject medium close-up is first, and then the multi-subject medium close-up.

[0242] It can be understood that the combination logic of the target clipping rules can also be expressed in other ways, which are not specifically limited here.

[0243] It is understood that Table 4 is only an exemplary introduction to the editing rules corresponding to the birthday party, and does not constitute a limitation on the editing rules of the embodiment of the present application. The editing rules corresponding to each editing style can be set according to actual conditions.

[0244] It can be understood that the editing style interface 400 may include more or fewer editing styles. Figure 3E The editing style interface 400 shown is only an example and does not constitute a limitation on the buttons on the interface and the interface layout style.

[0245] In some embodiments, the editing style interface 400 may be displayed in response to a fifth input from the user during the video recording process. For example, during the video recording process, the user long presses the volume button, or slides the video recording interface 350 downward, or sets a corresponding control on the video recording interface 35, and the editing style interface 400 is displayed in response to the input to the control.

[0246] like Figure 3E As shown, the electronic device 100 receives the user's input on the return control 409, and in response to the input, the electronic device 100 displays Figure 3F The settings interface 340 is shown.

[0247] Figure 3F The setting interface 340 shown is Figure 3D The differences of the settings interface 340 shown are: Figure 3F The editing style option 342 in the setting interface 340 shown no longer indicates the automatic matching editing rule, and the editing style option 342 indicates and displays that the currently selected target editing style is birthday party.

[0248] like Figure 3F As shown, the electronic device 100 receives the user's input on the return control 343, and in response to the input, the electronic device 100 displays Figure 3G The video preview interface 330 is shown.

[0249] Figure 3G The video preview interface 330 shown is Figure 3C The difference between the video preview interface 330 shown is that: Figure 3G The video preview interface 330 shown displays an editing style prompt interface 333 and a confirmation control 334. The prompt interface 333 is used to indicate the video sample (i.e., the required clip) required for the currently selected target editing style. For example, the prompt interface 333 may display the shot type and / or scene type of the required clip. For example, the birthday party editing style requires a close-up of food (i.e., a prompt to shoot the food in a close-up), a medium close-up of a person wearing a birthday hat (i.e., a prompt to shoot the subject wearing the birthday hat in a medium close-up), a medium close-up of blowing candles (i.e., a prompt to shoot the subject blowing candles in a medium close-up), and a close-up of cheering (i.e., a prompt to shoot the subject cheering in a close-up).

[0250] like Figure 3G As shown, the electronic device 100 receives the user's input on the confirmation control 334, and in response to the input, the electronic device 100 displays Figure 3H The video preview interface 330 is shown.

[0251] In some embodiments, the electronic device 100 may directly display the Figure 3H The video preview interface 330 is shown.

[0252] Figure 3H For the relevant contents of the video preview interface 330 shown, please refer to Figure 3C The video preview interface 330 is shown.

[0253] like Figure 3HAs shown, the electronic device 100 receives the input (ie, the third input) from the user on the recording start control 331, and in response to the input, the electronic device 100 starts recording the original video and displays the video as shown in FIG. Fig. 3I The video recording interface 350 is shown. The video recording interface 350 is used to display the image stream of the original video acquired by the camera 140 of the electronic device 100 in real time.

[0254] Fig. 3I The video recording interface 350 shown may display a recording end control 351, a shooting control 352, video recording time information 353, and a video recording screen 354. The recording end control 351 may be used to trigger the electronic device 100 to end recording the original video. The shooting control 352 may be used to trigger the electronic device 100 to save the first frame of the original video captured by the camera of the electronic device 100 when receiving the input as a picture in response to the user's input. The video recording screen 354 is used to display the image frames in the original video image stream.

[0255] For example, from 0 to 5 seconds after the start of recording, the birthday scene is filmed in full view, from 5 to 6 seconds, the birthday scene is filmed with friends, from 6 to 12 seconds, the friends are filmed in close-up, from 12 to 13 seconds, the friends are filmed with the birthday cake, from 13 to 20 seconds, the birthday cake is filmed in close-up, from 20 to 21 seconds, the birthday person wearing a birthday hat is filmed from the birthday cake, from 21 to 30 seconds, the birthday person wearing a birthday hat is filmed in medium and close-up, and from 30 to 31 seconds, the birthday person wearing the birthday hat is filmed from the birthday cake to the birthday person wearing the birthday hat. The birthday boy in the birthday hat turns to the crowd wearing birthday hats, the 31st to 40th seconds shoots the crowd wearing birthday hats in a medium close-up, the 40th to 41st seconds switches from the crowd wearing birthday hats to the birthday boy blowing out the candles, the 41st to 48th seconds shoots the birthday boy blowing out the candles in a medium close-up, the 48th to 49th seconds switches from the birthday boy blowing out the candles to the crowd cheering, the 49th to 59th seconds shoots the crowd cheering in a panoramic shot, the 59th to 60th seconds switches from the crowd cheering to the birthday cake, and the shooting ends at the 60th second.

[0256] The following shows the screen displayed on the video recording screen 354 on the video recording interface 350 during the video recording process.

[0257] At the 0th second of the start of recording, when the electronic device 100 is recording, the direction in which the camera of the electronic device 100 is pointed is the birthday scene. Fig. 3I The video screen 354 on the video interface 350 shows a panoramic shot of the birthday scene.

[0258] At the 7th second of the start of recording, Figure 3J The video screen 354 on the video interface 350 shows a close-up of the friend.

[0259] At 14 seconds after the start of recording, Figure 3K The video frame 354 on the video interface 350 shows a close-up of a birthday cake.

[0260] At 22 seconds after the start of recording, Figure 3L The video screen 354 on the video interface 350 is shown as a medium close-up shot of the birthday boy wearing a birthday hat.

[0261] At 33 seconds into the recording, Figure 3M The video screen 354 on the video interface 350 shows a medium close-up shot of two people wearing birthday hats.

[0262] At 44 seconds into the recording, Figure 3N The video screen 354 on the video interface 350 shows a medium close-up shot of the birthday person blowing out the candles.

[0263] At the 50th second of the start of recording, Fig.3O The video screen 354 on the video interface 350 is shown to display a panoramic shot of the cheering crowd.

[0264] At the 60th second of the start of recording, Figure 3P The video screen 354 on the video interface 350 shown shows a close-up of a birthday cake with the candles removed. Figure 3P As shown, at the 60th second after the start of recording, the electronic device 100 receives the user's input on the recording end control 351 (i.e., the fourth input), and in response to the input, the electronic device 100 ends the recording. It can identify and save the shot labels and scene labels in the original video, and can also save the recorded original video, the new video clips cropped from the original video, and the generated new video.

[0265] In an embodiment of the present application, during the video recording process of the electronic device 100, the electronic device can automatically and continuously identify and determine the shot tags, scene tags, shot scores and scene scores, and can automatically cut out shot clips, scene clips and wonderful shot clips. In some embodiments, the electronic device 100 can continuously identify and cut out new video clips from the original video according to the editing rules during the recording of the original video. After the electronic device 100 finishes recording the original video, the electronic device 100 can merge the cut new video clips into a new video according to the editing rules. The electronic device 100 can save and display the new video clips and the new video.

[0266] Optional, such as Figure 3Q As shown, after the video recording is finished, the electronic device 100 can display a video recording preview interface 330. For the text description in the video recording preview interface 330, reference can be made to the relevant content, which will not be repeated here.

[0267] After the electronic device 100 generates and saves a new video clip and / or a new video, the electronic device 100 may display a prompt message 355 on the video preview interface 330. The prompt message 355 is used to prompt the user that the electronic device 100 has generated and saved a lens clip and / or a wonderful lens clip and / or a scene clip and / or a new video from the recorded original video, and the user can view it by himself. The prompt message 355 may be a text prompt (for example, "a new video clip has been generated"), a pattern prompt, an animation prompt, etc. It can be understood that the "new video clip" in the prompt message 355 includes a lens clip and / or a scene clip and / or a wonderful lens clip and / or a new video.

[0268] Please also read Figure 4 ,right Figures 3A to 3P The processing of the original video shot in the scene is explained.

[0269] The contents of the original video can be referred to above and will not be repeated here.

[0270] The electronic device 100 can determine that the scene type of the 0-5.5s segment (segment a) in the original video is "birthday scene", the scene type of the 5.5s-12.5s segment (segment b) is "people", the scene type of the 12.5s-20.5s segment (segment c) is "food", the scene type of the 20.5s-30.5s segment (segment d) is "people", the scene type of the 30.5s-40.5s segment (segment e) is "multiple people", the scene type of the 40.5s-48.5s segment (segment f) is "people", the scene type of the 48.5s-59.5s segment (segment g) is "multiple people", and the scene type of the 59.5s-60s segment (segment h) is "food".

[0271] The electronic device 100 (shot size identification unit) can determine that the shot type corresponding to the segment from 0s to 5s (segment A) is panorama, the shot type corresponding to the segment from 6s to 12s (segment C) is close-up, the shot type corresponding to the segment from 13s to 20s (segment E) is close-up, the shot type corresponding to the segment from 21s to 30s (segment G) is medium close-up, the shot type corresponding to the segment from 31s to 40s (segment I) is medium close-up, the shot type corresponding to the segment from 41s to 48s (segment K) is medium close-up, and the shot type corresponding to the segment from 49s to 59s (segment M) is panorama.

[0272] The electronic device 100 (camera movement identification unit) can determine that the shot types corresponding to the 5th to 6th seconds segment (segment B), the 12th to 13th seconds segment (segment D), the 20th to 21st seconds segment (segment F), the 30th to 31st seconds segment (segment H), the 40th to 41st seconds segment (segment J), the 48th to 49th seconds segment (segment L), and the 59th to 60th seconds segment (segment N) are all fast camera movements.

[0273] The electronic device 100 (lens composition recognition unit) can determine that the lens type corresponding to the 6th to 12th segment is a single subject, the lens type corresponding to the 21st to 30th segment is a single subject, the lens type corresponding to the 31st to 40th segment is multiple subjects, the lens type corresponding to the 41st to 48th segment is a single subject, and the lens type corresponding to the 49th to 59th segment is multiple subjects.

[0274] The electronic device 100 can determine the shot segments including segments A to N, and determine the corresponding shot tags A1 to N1 according to the segments A to N. For example, the shot tag A1 corresponds to segment A, including the start time 0s, the end time 5s, and the shot type of the shot segment A is a panoramic shot. And so on. The shot types of segment C include close-up and single subject.

[0275] The scene segments that the electronic device 100 can determine include segments a to h, and determine corresponding scene tags a1 to h1 according to segments a to h. For example, scene tag a1 corresponds to scene segment a, including the start time 0s, end time 5.5s, and the scene type of birthday scene of scene segment a. And so on.

[0276] The target editing rules are shown in Table 4. The target editing rules indicate that the required wonderful shot clips include: clips with scene type of food and shot type of close-up, clips with scene type of wearing birthday hats and shot type of single subject and medium close-up, clips with scene type of wearing birthday hats and shot type of multiple subjects and medium close-up, clips with scene type of blowing candles and shot type of single subject and medium close-up, and clips with scene type of cheering and shot type of multiple subjects and panorama.

[0277] The 13s to 20s segments in the original video meet the scene type of food and the lens type of close-up, the 21s to 30s segments meet the scene type of wearing birthday hats and the lens type of single subject and medium close-up, the 31s to 40s segments meet the scene type of wearing birthday hats and the lens type of multiple subjects and medium close-up, the 41s to 48s segments meet the scene type of blowing candles and the lens type of single subject and medium close-up, and the 49s to 59s segments meet the scene type of cheering and the lens type of multiple subjects and panorama. According to the target editing rules, the wonderful lens segments that can be cut out from the original video include: segment 1 from 13s to 20s in the original video, segment 2 from 21s to 30s in the original video, segment 3 from 31s to 40s in the original video, segment 4 from 41s to 48s in the original video, and segment 5 from 49s to 59s in the original video.

[0278] When the electronic device 100 automatically creates a blockbuster with one click, the above-mentioned wonderful shot segments can be spliced ​​together according to the combination logic of the target editing rules. Therefore, the new video is spliced ​​in sequence: segment 1, segment 2, segment 3, segment 4 and segment 5. Splicing area 1 is used to connect the end of segment 1 with the beginning of segment 2, splicing area 2 is used to connect the end of segment 2 with the beginning of segment 3, splicing area 3 is used to connect the end of segment 3 with the beginning of segment 4, and splicing area 4 is used to connect the end of segment 4 with the beginning of segment 5, and finally a new video is obtained.

[0279] Among them, the specific process of the electronic device 100 identifying and cropping multiple new video segments from the original video, and merging the multiple new video segments into a new video can be referred to the subsequent embodiments of the present application and will not be repeated here.

[0280] Video synthesis scene

[0281] In an embodiment of the present application, after the electronic device 100 executes the video processing method of the present application on the original video, a new video clip in the original video can be identified. The electronic device 100 can save and display the new video clip, and edit the new video clip in response to the user's input to obtain a new video. The new video clip edited in the video synthesis scenario can be a new video clip that the electronic device 100 identifies and cuts out from the original video being shot in the above-mentioned video recording scenario. It can also be a new video clip that the electronic device 100 identifies and cuts out from the original video that has been shot in the following video editing scenario.

[0282] For example, a multiple-record album can be set in the gallery to display new video clips. When the electronic device 100 receives the user's input for the new video clip, the electronic device 100 can edit and synthesize the new video clip to obtain a new video. When the electronic device 100 receives the user's input for the new video clip, the electronic device 100 can play the corresponding video.

[0283] The following video synthesis scene is a new video clip edited by the electronic device 100. Figures 3A to 3P Take the original video shot in the scene as an example to identify and crop the shot fragments.

[0284] like Figure 5A As shown, the electronic device 100 can display a desktop 310. For the text description of the desktop 310, please refer to the aforementioned Figure 3A The illustrated embodiments will not be described in detail here.

[0285] The electronic device 100 receives the user's input on the gallery application icon 312, and in response to the input, the electronic device 100 may display the following Figure 5B The gallery application interface 410 is shown.

[0286] like Figure 5B As shown, the gallery application interface 410 can display one or more albums (for example, all photo albums, video albums 416, camera albums, multiple albums 417, etc.). The electronic device 100 can display a gallery menu 411 below the gallery application interface 410. The gallery menu 411 includes a photo control 412, an album control 413, a moment control 414, and a discovery control 415. The photo control 412 is used to trigger the electronic device 100 to display all local pictures in the form of picture thumbnails. The album control 413 is used to trigger the electronic device 100 to display the album to which the local picture belongs. Figure 5B As shown, the current album control 413 is in a selected state, and the electronic device 100 displays the gallery application interface 410. The moment control 414 can be used to trigger the electronic device 100 to display the selected pictures stored locally, such as images in a shot clip or images in a wonderful shot clip. The discovery control 415 can be used to trigger the electronic device 100 to display the classified albums of pictures.

[0287] The electronic device 100 may receive an input (i.e., the eighth input) from a user acting on the one-record-many-photos album 417, and in response to the input, the electronic device 100 may display the one-record-many-photos interface. The one-record-many-photos interface may include one or more video options, and the video options may be video options for the above-mentioned wonderful footage, or video options for the above-mentioned scene footage, or video options for the above-mentioned footage. In some embodiments, it may also be a video option for a new video synthesized from the above-mentioned new video. The video options may display a thumbnail of a specified frame in the video, a duration prompt for indicating the video time length information, and a selection control for selecting a video.

[0288] Taking the one-record-many-access interface displaying a portion of the lens clip as an example, the electronic device 100 responds to the input of the one-record-many-access album 417 and displays the following Figure 5C The one-entry-multiple-gets interface 420 is shown. Figure 5C As shown, option 421 corresponds to the above-mentioned segment 1, and option 421 displays a thumbnail of the first image frame in segment 1, a selected control 422, and a duration prompt 423. Option 424 corresponds to the above-mentioned segment 2, and option 424 displays a thumbnail of the first image frame in segment 2, a selected control 425, and a duration prompt 426. Option 427 corresponds to the above-mentioned segment 3, and option 427 displays a thumbnail of the first image frame in segment 3, a selected control 428, and a duration prompt 429. Option 430 corresponds to the above-mentioned segment 4, and option 430 displays a thumbnail of the first image frame in segment 4, a selected control 432, and a duration prompt 431. Option 433 corresponds to the above-mentioned segment 5, and option 433 displays a thumbnail of the first image frame in segment 5, a selected control 434, and a duration prompt 435.

[0289] like Figure 5C As shown, the one-record-multiple-receive interface 420 may also include a one-touch blockbuster control 436. The one-touch blockbuster control 436 is used to generate a new video based on the new video clip selected by the user. The electronic device 100 receives the user's input on the above-mentioned selection control, and in response to the input, determines the video selected by the user. Figure 5C As shown, the electronic device 100 receives input for selecting controls 422, 425, 428, and 432, and in response to the input, selects segments 1-4 corresponding to controls 422, 425, 428, and 432, and determines that segments 1-4 are target shot segments. Then, the electronic device 100 receives input from the user acting on the one-key blockbuster control 436 (i.e., the ninth input), and in response to the input, synthesizes the segments 1, 2, 3, and 4 (i.e., the target shot segments) selected by the user into a new video according to the target editing rule.

[0290] It is understandable that the ninth input method is not limited to the above, and other methods can also be used to instruct the electronic device to synthesize the clips selected by the user into a new video according to the target editing rules.

[0291] In an embodiment of the present application, the electronic device 100 can combine video clips from different original videos to generate a new video. Specifically, all new video clips identified and cropped by the electronic device 100 are displayed on the one-record-many-receive interface 420, which may include new video clips cropped from the recorded original video, or may include new video clips identified and cropped from the original video stored in the electronic device. The one-record-many-receive interface 420 can also display new videos generated from different original videos. The user can select any video clip on the one-record-many-receive interface 420 according to actual conditions.

[0292] In some embodiments, the Figure 5CThe above-mentioned editing style option 342 or the editing style control 721 described below can be set in the illustrated one-record-multiple-receive interface 420, so that the user can select the editing rule corresponding to the editing style. The relevant content of the editing style control 721 can be referred to below. The user can operate the editing style option 342 or the editing style control 721 to determine the target editing rule corresponding to the target editing style. After determining the target editing rule, the video clip selected by the user is edited according to the target editing rule to synthesize a new video.

[0293] In some embodiments, the electronic device 100 can automatically match the editing style according to the video clip selected by the user, and edit the video clip selected by the user according to the editing rule corresponding to the editing style with the highest matching degree. In some embodiments, the electronic device 100 can edit the video clip selected by the user according to the default editing rule. The above-mentioned editing of the selected video clip includes: adjusting the selected video clip and combining the selected video clip (or the adjusted video clip) according to the combination logic of the editing rule. The specific content can be referred to as follows.

[0294] In some embodiments, the electronic device 100 performs the video processing method of the present application on the target image selected by the user, and can identify the lens type of the target image, and then edit it according to the identified lens type of the target image to obtain a new video. In other embodiments, the electronic device 100 performs the video processing method of the present application on the target image selected by the user and the original video, and can identify the lens type of the target image and the new video segment of the original video, and then edit it according to the identified lens type of the target image and the new video segment of the original video, such as combining the target image and the new video segment to obtain a new video.

[0295] Exemplarily, the target editing rule indicates the lens type of the desired image and the combination logic of each desired image. In response to the user selecting multiple target images from all photo albums, the lens types of the multiple target images are identified, and then the target editing rule and the multiple target images are edited to obtain a new video.

[0296] Automatic video editing scenes

[0297] In the automatic video editing scenario, the electronic device 100 continuously identifies new video clips and their labels in the recorded video, and automatically generates a new video based on the new video clips, can save and display the new video, and can also store the labels in the video file of the original video. For example, after saving the new video clip and the new video, the electronic device 100 can simultaneously display the new video clip and / or the display area of ​​the new video in the display interface of the original video. When the electronic device 100 receives the user's input (such as a single click) for the display area of ​​the new video, the electronic device 100 can play the new video.

[0298] To Figures 3A to 3P The original video shot in the video recording scene identifies wonderful footage, and takes the editing rules corresponding to the birthday party as shown in the following Table 5 as an example.

[0299] Table 5

[0300]

[0301] The editing rules shown in Table 5 are different from those shown in Table 4 in that the editing rules shown in Table 5 also indicate the duration, especially the duration of the wonderful shot segments.

[0302] Then the electronic device 100 can cut out a 2-second continuous wonderful shot segment A from segment 1, cut out a 3-second continuous wonderful shot segment B from segment 2, cut out a 3-second continuous wonderful shot segment C from segment 3, cut out a 3-second continuous wonderful shot segment D from segment 4, and cut out a 4-second continuous wonderful shot segment E from segment 5. The combination logic shown in Table 5 is similar to that in Table 4. The electronic device 100 sequentially combines the wonderful shot segments A, B, C, D, and E to obtain a new video with a duration of 15 seconds.

[0303] like Fig. 6A As shown, the electronic device 100 can display a gallery application interface 410. For a text description of the gallery application interface 410, please refer to the aforementioned Figure 5B The text part of the illustrated embodiment will not be repeated here.

[0304] The electronic device 100 may receive user input on the video album 416, and in response to the input, the electronic device 100 may display the video album 416. Figure 6B Video album interface 510 is shown.

[0305] like Figure 6B As shown, the video album interface 510 may include video options of one or more videos, which may be original videos shot by the user through the camera 140 of the electronic device 100, original videos downloaded by the user, or new videos obtained by editing. Figures 3A to 3P The original video of 60 seconds recorded in the video recording scene, the video option 511 displays the thumbnail of the first image frame in the original video and the duration prompt 512. The video option 513 corresponds to the original video downloaded from the network by the electronic device 100, and the video option 513 displays the thumbnail of the first image frame in the original video and the duration prompt 514.

[0306] The electronic device 100 receives the user's input on the video option 511 (ie, the video option 511 is the target video option), and in response to the input, the electronic device 100 obtains the original video corresponding to the video option 511, and can display the original video as shown in FIG. Figure 6C The video display interface 610 is shown.

[0307] The video display interface 610 may include a display area 611 of the original video (i.e., the original video corresponding to the video option 511), a display area 613 of the new video generated from the original video, a menu 616, etc. Among them, the display area 611 of the original video may display the frame picture and time information 612 (e.g., the time length is 60 seconds) in the original video. When the display area 611 of the original video receives the input of the user (e.g., single click), the electronic device 100 may play or pause the original video. The display area 613 of the new video may display the frame picture in the new video and the time information 615 of the new video. The time information 615 indicates that the time length is 15 seconds. The new video displayed on the display area 613 of the new video is the new video obtained by editing the original video in the display area 611 of the original video according to the target editing rule of Table 5. The content of the new video obtained by editing the original video may refer to the following embodiments.

[0308] Optionally, a highlight mark 614 may be displayed on the display area 613 of the new video, and the highlight mark 614 may be used to prompt the user that what is displayed in the display area 613 is a new video generated from the original video. The menu 616 may include a share button, a favorite button, an edit button, a delete button, and a more button. The share button may be used to trigger the sharing of the original video and / or the new video. The favorite button may be used to trigger the collection of the original video and / or the new video to a favorite folder. The edit button may be used to trigger editing functions such as rotation, trimming, adding filters, blurring, etc. of the original video and / or the new video. The delete button may be used to trigger the deletion of the original video and / or the new video. The more button may be used to trigger the opening of more functions related to the original video and / or the new video, such as generating a new video by using the video processing method provided in the embodiment of the present application to the original video and / or the new video.

[0309] The electronic device 100 may receive an input from a user acting on a more button, and in response to the input, the electronic device 100 may display the following Figure 7C The list 720 shown, the related contents of the list 720 can be referred to as follows, and then the user can select the target editing rule corresponding to the target editing style. Then, the new video in the new video display area 613 or the original video in the original video display area 611 can be edited according to the target editing rule.

[0310] Video editing scenes

[0311] In an embodiment of the present application, a user can edit an original video saved in the electronic device 100 (such as a video shot or a new video edited, etc.), including identifying, cropping and combining new video clips to obtain a new video, and can display the saved video after saving the new video clip and / or the new video.

[0312] The electronic device 100 may display Fig. 7A The video album interface 510 shown in FIG. Fig. 7A The text description of the video album interface 510 shown can refer to the aforementioned Figure 6B The text part of the illustrated embodiment will not be repeated here.

[0313] The electronic device 100 receives the user's input on the video option 513 (ie, the video option 513 is the target video option), and in response to the input, the electronic device 100 obtains the original video indicated by the video option 513, and can display the original video as shown in FIG. Figure 7B Video display interface 710 is shown.

[0314] The video display interface 710 may include a display area 711 of the original video (i.e., the original video corresponding to the video option 513), a menu 713, a one-key blockbuster control 714, and a progress bar 715. Among them, the display area 711 may display image frames and time information 712 (for example, the time length is 20 seconds) in the original video. When the display area 711 receives the user's input, the electronic device 100 can play or pause the original video. The one-key blockbuster control 714 can be used to trigger the electronic device 100 to generate a new video from the original video displayed in the display area 711. The progress bar 715 can be used to indicate the video playback status, such as playing or not playing or pausing. When the video has been played, the progress of the video playback can be displayed. The menu 713 may include a share button, a favorite button, an edit button, a delete button, and more buttons 716. The relevant contents of the share button, the favorite button, the edit button, the delete button, and the more buttons 716 can refer to the above.

[0315] The electronic device 100 receives an input from a user on the more button 716, and in response to the input, the electronic device 100 may display Figure 7C Video display interface 710 is shown. Figure 7C The video display interface 710 shown in the figure displays a list 720. The list 720 includes a screen casting control, a clipping style control 721, a move to album control, and a copy to album control. The clipping style control 721 is used for the user to select a clipping style. For related content, please refer to the clipping style option 342. The difference between the clipping style control 721 and the clipping style option 342 is that the clipping style control 721 cannot be displayed or indicate automatic matching of the clipping style when the user has not selected the clipping style.

[0316] In some embodiments, the editing style control 721 can be set to display and indicate automatic matching of editing styles when the user has not selected an editing style. For details, please refer to the editing style option 342. Then the input of the user on the more button 716 is the sixth input, and the editing style control 721 is also relative to the editing style option 342.

[0317] It is understandable that the list 720 may include more or fewer controls under the editing style control 721 , and the present application does not make any specific limitation on this list 720 .

[0318] The electronic device 100 receives the user input (ie, the second input) on the editing style control 721, and in response to the input, the electronic device 100 may display the editing style interface 400 shown in FIG7D. The content of the editing style interface 400 is as described above and will not be repeated here.

[0319] It is understandable that the second input method is not limited to the above, and other methods can also be used to provide the editing style interface 400 when editing the recorded video.

[0320] Fig.7D The editing style interface 400 shown is similar to Figure 3E The differences of the editing style interface 400 shown are: Fig.7D In the editing style interface 400 shown, the editing option 403 indicating the wedding editing style is selected, while the editing option 401 indicating the birthday party editing style is not selected, that is, the editing option 403 is the target editing option, thereby determining that the target editing style is the wedding editing style.

[0321] like Fig.7D As shown, the electronic device 100 receives the user's input on the return control 409, and in response to the input, the electronic device 100 may display Fig. 7E Video display interface 710 is shown.

[0322] like Fig. 7E As shown, the electronic device 100 receives the user's input (i.e., the first input) on the one-key blockbuster control 714, and in response to the input, the electronic device 100 identifies and crops multiple wonderful shot segments in the original video according to the target editing rules corresponding to the wedding editing style, and merges the multiple wonderful shot segments into a new video according to the target editing rules corresponding to the wedding editing style.

[0323] It is understandable that the first input method is not limited to the above, and other methods can also be used to instruct the electronic device to synthesize the original video selected by the user into a new video according to the target editing rule.

[0324] Optional, such as Figure 7F As shown, in the process of the electronic device 100 generating a new video, the electronic device 100 can display the generation progress 720 of the new video on the video display interface 710. The generation progress 720 is used to indicate the progress of the new video being generated, and it can also prompt text, such as "generating a wonderful video", which is not specifically limited in this application.

[0325] In some embodiments, the user has not selected a target editing style, that is, the user has not input the more button 716 and the editing options on the editing style interface 400, and the user directly inputs the one-key blockbuster control 714. The electronic device 100 responds to the input to the one-key blockbuster control 714 and can automatically match the editing rules, such as matching the editing rules in the editing rule library according to the original video displayed in the display area 711, or the electronic device 100 obtains the default editing rules, and then edits the original video according to the obtained editing rules (matched or default editing rules) to obtain a new video.

[0326] The specific process of the electronic device 100 identifying and cutting out multiple new video segments from a video and merging them into a new video can be referred to the subsequent embodiments of the present application and will not be described in detail here.

[0327] like Figure 7G As shown, after the electronic device 100 generates a new video, the electronic device 100 displays the display area 717, menu 740, and progress bar 731 of the generated new video on the video display interface 710. The image frames and time information 730 (for example, the time length is 13 seconds) in the new video can be displayed on the display area 717 of the new video. The synthesis time of the new video is 13 seconds.

[0328] Menu 740 may include a music button 741, an edit button 743, and a save button 742. Music button 741 may be used to trigger the soundtracking of a new video. Edit button 743 may be used to trigger editing functions such as rotation, trimming, adding filters, blurring, etc. of the new video. Save button 742 may be used to trigger the deletion of the currently generated new video. The saved new video may be a new video generated by a one-key blockbuster, or may be a video processed from the new video generated by a one-key blockbuster, such as a video with soundtracking, such as an edited video, etc. Progress bar 731 indicates that the new video is paused and is currently playing to the 2nd second.

[0329] It can be understood that in the video editing scene, if the video tag is stored in the video file of the original video, the video tag can be directly obtained, and the clips can be cut out according to the target editing rule and the video tag, and the cut clips can be combined. The video tag includes one or more of the shot tag and the scene tag.

[0330] If in the video editing scene, the video file of the original video does not store a video tag, such as the video shot by the camera of an electronic device but the one-record-multiple-record mode is not activated, or the video shot by a third-party application and no relevant detection (lens detection and / or scene detection) is performed, or the video is downloaded and no relevant detection (lens detection and / or scene detection) is performed, then the image stream of the original video can be detected, such as calling a relevant detection algorithm for detection, and then each new video segment in the original video can be determined, and then the corresponding segment is cropped according to the target editing rule, and the cropped segment is combined. In some embodiments, after the original video is detected, the obtained tag is stored in the video file of the original video, so that when the original video is subsequently edited, the tag can be directly obtained and edited according to the tag.

[0331] Video call scenario

[0332] During a video call, the electronic device 100 can identify and crop new video clips in the image stream during the video call, and merge the cropped new video clips into a new video. In particular, the highlight clips can be cropped according to the editing rules, and the highlight clips can be combined according to the editing rules to obtain a new video. After the video call ends, the electronic device 100 can save the new video clips and / or the generated new video. Optionally, the electronic device 100 can also share the new video clips and / or the generated new video with the other party of the video call.

[0333] Live broadcast scene

[0334] During the live video broadcast, the electronic device 100 can identify and crop new video clips in the image stream during the live video broadcast, and synthesize the cropped new video clips into a new video. In particular, the wonderful shot clips can be cropped according to the editing rules, and the wonderful shot clips can be combined according to the editing rules to obtain a new video. After the live broadcast ends, the electronic device 100 can save the new video clips and / or the generated new video. Optionally, the electronic device 100 can also synchronize the new video clips and / or the generated new video to the server of the live broadcast application and bind it to the live broadcast account, and share it to the public viewing area for other accounts that follow the live broadcast account to watch.

[0335] It can be understood that the video processing method provided in the embodiment of the present application can be applied to scenes of video processing, such as identifying and / or editing clips related to the lens type, including but not limited to scenes of video recording, video synthesis, automatic video editing, video editing, live broadcast, video call, etc. Figure 3H to Figure 3PThe video synthesis scenario refers to the new video clips and / or new videos obtained based on the video processing method provided in the embodiment of the present application, and the obtained new video clips and / or new videos can be synthesized into a large movie with one click, such as synthesizing according to the target editing rules with one click. FIG. 5A to FIG. 5C The automatic video editing scenario refers to the electronic device 100 entering a one-record-multiple-record mode during the video recording process. When the original video is captured, the original video can also be automatically edited to obtain a new video. The automatic video editing scenario can refer to the above FIG. 6A to FIG. 6C . The video editing scene refers to the user automatically editing the video in the electronic device 100. The difference between the video editing scene and the automatic video editing scene is that the automatic video editing scene generates a new video during the video recording process. The video editing scene is to edit the recorded video to generate a new video. The original video processed in the video editing scene may be a video that the electronic device has not yet performed relevant detection, that is, the original video may be a video shot in the one-record-many mode, or a video shot by a third-party application or a downloaded video. The video editing scene can refer to the above FIG. 7A to FIG. 7G .

[0336] A video processing method provided by an embodiment of the present application is introduced below in conjunction with a flow chart and a functional module diagram.

[0337] It should be noted that, in the methods disclosed in the embodiments of the present application or the methods shown in the flowcharts, the execution order of multiple steps can be interchangeable with each other, and some of the steps can also be deleted.

[0338] See also Figure 8 , a video processing method provided in an embodiment of the present application is exemplarily introduced, and the video processing method is applied to an electronic device and may include steps S801 to S804.

[0339] Step S801, obtaining the image stream of the original video.

[0340] In an embodiment of the present application, the electronic device can obtain the image stream of the original video in response to the user's input.

[0341] In a video recording scene, a video automatic editing scene, a live broadcast scene, or a video call scene, the electronic device can automatically perform the video processing method of the present application on the original video obtained when entering the one-record-multiple-acquisition mode. Based on the fact that the electronic device has entered the one-record-multiple-acquisition mode, the electronic device can respond to the input indicating the start of recording, acquire an image stream by acquiring an image in real time through the camera, and the electronic device executes step S801 to acquire the image stream acquired in real time by the camera. For example, in a video recording scene, the electronic device receives the input of the user acting on the recording start control 331, that is, receives the input indicating the start of recording.

[0342] In other embodiments, in a video recording scene, a video automatic editing scene, a live broadcast scene, and a video call scene, the electronic device has started recording. During the recording process, the electronic device can respond to an input indicating that the electronic device enters a one-record-many-acquire mode, and the electronic device executes step S801 to obtain an image stream captured by the camera. For example, after starting to record a video, the user turns on the one-record-many-acquire option 341.

[0343] In the video editing scene, the electronic device may receive an editing instruction or an original video selection instruction input by the user, wherein the editing instruction indicates editing the original video, and the original video selection instruction may indicate the selected original video. In response to the editing instruction, the electronic device may execute step S801 to obtain the image stream of the original video indicated by the editing instruction from the memory storing the video to be edited. For example, in the video editing scene, the electronic device receives the input of the user acting on the one-key blockbuster control 714, i.e., receives the editing instruction. Receiving the input acting on the video option 513, i.e., receiving the original video selection instruction.

[0344] Step S802: Perform shot detection on the image frames in the image stream to determine the shot type of each image frame in the image stream.

[0345] In the embodiment of the present application, the electronic device may perform shot detection on each image frame in the image stream obtained in step S801, or may perform shot detection on each image frame in the preprocessed image stream to obtain a detection result, and then determine the shot type of each image frame in the image stream based on the detection result and other information.

[0346] In an embodiment of the present application, the electronic device can perform lens detection on the image stream according to the lens types included in the lens type library to determine the lens type included in each image frame in the image stream. Taking the lens type library as an example, which divides the lens types from four dimensions of lens size, lens movement, lens composition, and lens angle and position, for image frame A, the lens size recognition unit can recognize that the image frame A is a long-range lens, the lens composition recognition unit can recognize that the image frame A is a three-body lens, and the lens angle position recognition unit can recognize that the image frame A is a ground-level height lens, and then it can be recognized that the lens types of image frame A include long-range lenses, three-body lenses, and ground-level height lenses.

[0347] In the embodiment of the present application, the number of shot types included in one image frame may be greater than 1. In some embodiments, at least one shot type can be determined for each image frame.

[0348] The specific contents of performing shot detection on the image stream or the preprocessed image stream to determine the shot type of each image frame in the image stream may refer to the relevant contents of the above-mentioned shot recognition module, which will not be described in detail here.

[0349] Step S803: determining shot segments according to the shot types of the multiple image frames.

[0350] In an embodiment of the present application, the electronic device determines a shot segment according to the shot types of multiple image frames, including determining a start time, an end time, and a shot type corresponding to the shot segment.

[0351] In the embodiment of the present application, when the lens types of a plurality of consecutive image frames in the image stream are the same, it can be determined that the plurality of consecutive image frames are part or all of a lens segment. When the electronic device determines all the image frames included in the lens segment, it can determine the start time and the end time of the lens segment. The lens type that is consistent with all the image frames in the lens segment is the lens type of the lens segment.

[0352] In the embodiment of the present application, after the electronic device determines the start time, end time and shot type of the shot segment, the shot tag of the shot segment can be determined. The shot tag of the shot segment includes the start time, end time and shot type of the shot segment. In some embodiments, the shot tag may also include the shot score of the shot segment.

[0353] In an embodiment of the present application, the electronic device determines all shot tags included in the image stream and writes all shot tags to the video file of the original video, so that when the original video is processed (such as editing) next time, the corresponding shot tags can be directly obtained from the video file of the original video, and the original video can be edited according to the shot tags.

[0354] It can be understood that a video file includes a video header and video content. The video header is provided with a custom extended information field. The shot tag and / or scene tag can be written into the custom extended information field in the video header in JSON (JavaScript Object Notation) format.

[0355] In the embodiment of the present application, different shot tags may indicate different shot types, and the shot segments indicated by different shot tags may have overlapping parts. For specific content, please refer to the above content about shot tags and shot segments, which will not be repeated here.

[0356] Step S804, multiple shot segments in the original video are edited according to the target editing rule to obtain a new video.

[0357] In the embodiment of the present application, multiple shot segments in the image stream can be determined in step S803, and the electronic device can cut out multiple shot segments from the image stream according to the positions of the multiple shot segments in the image stream. The multiple shot segments can be edited, such as screening and adjusting the multiple shot segments, and then combined into a new video. Specifically, the electronic device can cut and combine multiple shot segments in the original video according to the target editing rule to obtain a new video.

[0358] The method for the electronic device to obtain the target clipping rule may be referred to as follows.

[0359] In the embodiment of the present application, the target clipping rule may be a default clipping rule. The electronic device may clip all the lens segments in the original video according to the default clipping rule to combine the lens segments of different lens types to obtain a new video.

[0360] For another example, the target editing rule is selected by the user. The electronic device determines the target editing style selected by the user in response to the user's input, and the editing rule corresponding to the target editing style is the target editing rule. The electronic device cuts out multiple shot segments of the required shot type from the image stream according to the shot type indicated by the target editing rule.

[0361] For another example, the target editing rule is the editing rule with the highest matching degree. The electronic device matches the editing style with the highest matching degree according to the original video, and cuts multiple shot segments of the required shot type from the image stream according to the shot type indicated by the editing rule corresponding to the editing style with the highest matching degree.

[0362] Exemplarily, taking the target editing style (or the editing style with the highest matching degree) as birthday party, and the target editing rules for the birthday party as shown in Table 4, the electronic device can determine the required shot types including close-up, medium close-up single-subject shot, and medium close-up multi-subject shot according to the target editing rules, and then can filter out the shot types of close-up, medium close-up single-subject shot, or medium close-up multi-subject shot from all the shot segments of the original video, and according to the positions of the filtered shot segments, cut out the corresponding shot segments from the image stream of the original video.

[0363] In some embodiments, when the electronic device cuts the clips according to the target clipping rule, the clips with high scores can be screened out according to the scores of the clips, and then the clips with high scores can be cut out according to the target clipping rule. For example, after the electronic device screens out the clips with high scores, it cuts the screened clips according to the default clipping rule.

[0364] In some embodiments, when the electronic device crops a segment according to the target editing rule, it can crop a new video segment of corresponding duration according to the duration. For specific content, reference can be made to the wonderful shot segment AE cropped according to Table 5 above.

[0365] In some embodiments, when the electronic device crops a segment according to a target editing rule, the electronic device may adjust the cropped segment according to the total duration requirement of the new video, such as adjusting the start time and / or end time of the segment.

[0366] In an embodiment of the present application, the electronic device can combine the above-mentioned multiple cropped shot segments according to the target editing rule to obtain a new video.

[0367] In the embodiment of the present application, the electronic device can extract the corresponding shot segment from the image stream of the video according to the shot tag. For example, for shot tag A, shot tag A indicates that the shot type of shot segment a is a close-up shot, the start frame of shot segment a is the first frame in the original video, and the end frame of shot segment a is the 1200th frame in the original video. Then, according to the shot tag, the first frame to the 1200th frame in the original video are extracted to obtain shot segment a.

[0368] It can be understood that when the electronic device enters the one-record-multiple-receive mode, the electronic device can automatically execute steps S800 to S803. In some embodiments, after the electronic device enters the one-record-multiple-receive mode, the shot tags of the original video can be obtained. In some embodiments, in the automatic video editing scene, the electronic device can also execute step S804 to automatically edit a new video. For another example, in the video editing scene, the electronic device receives the editing instruction input by the user, and the electronic device can also execute step S804 to automatically edit a new video.

[0369] See also Fig. 9 , exemplarily introduces a video processing method provided in an embodiment of the present application, which is applied to an electronic device and may include steps S901 to S908, wherein the contents of steps S901 to S903 may refer to the contents of the above-mentioned steps S801 to S803, and will not be repeated here.

[0370] Step S901, obtaining the image stream of the original video.

[0371] Step S902: Perform shot detection on the image frames in the image stream to determine the shot type of each image frame in the image stream.

[0372] Step S903: determining shot segments according to the shot types of the multiple image frames.

[0373] In the embodiment of the present application, the electronic device may obtain a shot tag of the shot segment after determining the shot segment.

[0374] Step S904: Perform scene detection on the image stream to determine the scene type of each image frame in the image stream.

[0375] In the embodiment of the present application, the electronic device may perform scene detection on each image frame in the image stream obtained in step S902, or may perform scene detection on each image frame in the preprocessed image stream to obtain a detection result, and then determine the scene type of the image frame according to the detection result.

[0376] In some embodiments, the electronic device may perform lens detection and scene detection simultaneously, or perform scene detection first and then lens detection, which is not specifically limited in this application.

[0377] The electronic device can perform scene detection on the image stream through a scene detection algorithm. For specific content, please refer to the relevant content of the above-mentioned scene recognition module, which will not be repeated here.

[0378] Step S905: determining scene segments according to the scene types of the multiple image frames.

[0379] In an embodiment of the present application, the electronic device determines a scene segment according to scene types of multiple image frames, including determining a start time, an end time, and a scene type corresponding to the scene segment.

[0380] In the embodiment of the present application, when the scene types of a plurality of consecutive image frames in the image stream are the same, it can be determined that the plurality of consecutive image frames are part or all of the scene segment. When the electronic device determines all the image frames included in the scene segment, it can determine the start time and the end time of the scene segment. The scene type that is consistent with all the image frames in the shot segment is the scene type of the scene segment.

[0381] In an embodiment of the present application, after the electronic device determines the start time, end time and scene type of the scene segment, the scene tag of the scene segment can be determined. The scene tag of the scene segment includes the start time, end time and scene type of the scene segment. In some embodiments, the scene tag may also include the scene score of the scene segment.

[0382] In an embodiment of the present application, the electronic device determines all scene tags included in the image stream and records all scene tags. For example, all shot tags, scene tags, shot scores and scene scores can be written into the video file of the original video, so that when the original video is processed (such as editing) next time, the corresponding shot tags, scene tags, shot scores and scene scores can be directly obtained from the video file of the original video, and then the original video can be edited according to the shot tags, scene tags, shot scores and scene scores.

[0383] Step S906, obtaining target clipping rules.

[0384] The content of the target clipping rule may refer to the above clipping rule, which will not be described in detail here.

[0385] In the embodiment of the present application, the electronic device obtains the target clipping rule in the following ways, including but not limited to:

[0386] First, the electronic device can respond to the user's input and obtain the target editing rule indicated by the input according to the user's input. For example, if the user selects the editing option 401 in the editing style interface 400, the birthday party editing style corresponding to the editing option 401 can be determined as the target editing style, and the camera application sends an instruction to the electronic device to instruct the acquisition of the target editing rule corresponding to the target editing style.

[0387] Second, the electronic device can also match the editing rules for the electronic device according to the detection result obtained by detecting the image stream, and obtain the editing rule with the highest matching degree from the memory storing the editing rules, and the editing rule with the highest matching degree is the target editing rule. For example, in the above-mentioned recording video scene, the user does not operate the editing style interface 400, and the editing style option 342 indicates automatic matching of the editing style. The electronic device can match the scene type and / or lens type of the image stream with the editing rules in the editing rule library according to the scene detection and / or lens detection of the image stream, and determine the editing rule with the highest matching degree. For example, the editing rule with the most lens types and scene types in the image stream is the editing rule with the highest matching degree, and then obtain the editing rule with the highest matching degree from the memory and determine it as the target editing rule.

[0388] In some embodiments, when the electronic device enters the one-record-multiple-receive mode and no editing style is selected, the editing rules may be automatically matched first. When the editing rule with the highest matching degree does not meet the preset requirements, the editing rules may be determined in the following third manner. For example, if the number of scene types and / or shot types required by the editing rules that the original video can provide is lower than a preset threshold, such as 50%, it may be determined that the preset requirements are not met. The preset requirements may be set according to actual conditions and are not specifically limited thereto.

[0389] The third method is that the electronic device can obtain a default clipping rule. When the user does not indicate a clipping rule, or the electronic device does not automatically match the clipping rule, or the clipping rule with the highest matching degree does not meet the preset requirements, the electronic device can obtain a default clipping rule and determine it as the target clipping rule.

[0390] The timing of obtaining the target clipping rule can be determined according to actual conditions and is not specifically limited thereto.

[0391] In some embodiments, the electronic device preferentially uses the editing rule corresponding to the editing style selected by the user as the target editing rule. When the electronic device does not obtain the editing style selected by the user, it determines the editing rule with the highest matching degree in the editing rule library according to the shot type in the original video; determines whether the editing rule with the highest matching degree meets the preset requirements; if so, uses the editing rule with the highest matching degree as the target editing rule; if not, uses the default editing rule as the target editing rule.

[0392] In an embodiment of the present application, determining whether the editing rule with the highest matching degree meets the preset requirements includes: calculating the number of pending lens segments of the original video to obtain a first value, where the lens type of the pending lens segment is the lens type of the required segment; calculating the number of required segments to obtain a second value; calculating the ratio of the first value to the second value; when the ratio is less than a preset threshold, determining that the editing rule with the highest matching degree does not meet the preset requirements; when the ratio is greater than or equal to the preset threshold, determining that the editing rule with the highest matching degree meets the preset requirements.

[0393] For example, the original video includes 3 lens clips, and the corresponding lens types are close-up, medium close-up single subject, and close single subject. Based on the lens type of the original video, the automatic matching is performed, and the editing rule with the highest matching degree is shown in Table 5. It can be determined that the number of undetermined lens clips in the original video is 2, and the corresponding lens types are close-up and medium close-up single subject. The number of clips required for the editing rule with the highest matching degree is 5 (that is, corresponding to 5 rows in Table 5), and the ratio of the first value to the second value is calculated to be 2 / 5. The ratio is judged by comparing the ratio with the preset threshold to determine whether the editing rule with the highest matching degree is the preset requirement.

[0394] Step S907, determining a plurality of wonderful shot segments in the image stream according to the positions of the shot segments, the shot types of the shot segments, the positions of the scene segments, the scene types of the scene segments and the target editing rules.

[0395] In the embodiment of the present application, the electronic device can determine the scene type and shot type of the wonderful shot clip according to the scene type and shot type indicated by the target editing rule, that is, the scene type and shot type of the wonderful shot clip are the scene type and shot type indicated by the target editing rule. The electronic device determines the wonderful shot clip in the image stream according to the position of the shot clip, the position of the scene clip, and the scene type and shot type of the wonderful shot clip, including determining the scene type and shot type of the wonderful shot clip and the position of the wonderful shot clip in the original video (that is, the starting time and the ending time).

[0396] Taking the target editing rule as Table 4 as an example, the target editing rule indicates that the scene type needs to be food and the lens type needs to be close-up, such as Figure 4 As shown, the scene type of the scene clip from 12.5 seconds to 20.5 seconds in the original video is food, and the shot type of the lens clip from 13 seconds to 20 seconds in the original video is close-up. According to the position of the scene clip and the lens clip, the clip from 13 seconds to 20 seconds in the original video is cut out, that is, the wonderful shot clip.

[0397] In the embodiment of the present application, the method of determining multiple wonderful shot clips in the image stream according to the position of the shot clip, the shot type of the shot clip, the position of the scene clip, the scene type of the scene clip and the target editing rule includes but is not limited to the following:

[0398] The first method is to determine candidate scene segments according to the scene type of the required segment, the position of the scene segment and the scene category of the scene segment, and then determine the wonderful shot segments from the candidate scene segments according to the shot type of the required segment, the position of the shot segment and the shot type of the shot segment.

[0399] The second method is to determine candidate shot segments according to the shot type of the required segment, the position of the shot segment and the shot type of the shot segment, and then determine the wonderful shot segments from the candidate shot segments according to the scene type of the required segment, the position of the scene segment and the scene type of the scene segment.

[0400] In some embodiments, the electronic device may also determine multiple wonderful shot segments in the image stream according to the target editing rules, scene tags and shot tags. The specific implementation method may refer to the above content of determining multiple wonderful shot segments in the image stream according to the target editing rules, shot segments and scene segments.

[0401] Step S908, editing multiple wonderful shot segments according to the target editing rule to obtain a new video.

[0402] In the embodiment of the present application, the electronic device edits the multiple wonderful shot segments determined in step S907, including cutting the multiple wonderful shot segments according to the target cutting rule and combining the cut segments according to the target cutting rule to obtain a new video. Alternatively, the electronic device can directly combine the multiple wonderful shot segments obtained in step S907 according to the combination logic of the target cutting rule to obtain a new video.

[0403] Among them, when cutting multiple wonderful shot clips: first, the wonderful shot clips with high scores can be screened out according to the scores, and then the required clips can be selected from the wonderful shot clips with high scores. Alternatively, the start time and the end time of the wonderful shot clip can be adjusted according to the duration of the target editing rule, such as shortening the duration of the wonderful shot clip or lengthening the duration of the wonderful shot clip. The content of cutting multiple wonderful shot clips can refer to the above step S804, which will not be repeated here.

[0404] In the embodiment of the present application, the electronic device combines the multiple wonderful shot clips, including splicing the multiple wonderful shot clips together according to the combination logic indicated by the target editing rule. The contents of combining the multiple wonderful shot clips can refer to the above step S804, which will not be repeated here.

[0405] For example, Fig.10 As shown, the original video uses an aerial camera to shoot the city appearance from 0 to 5s, switches from the city appearance to person A from 5s to 7s, shoots person A walking in the crowd with a medium close-up shot from 7s to 17s, switches from person A to the city appearance from 17s to 19s, shoots the city appearance with a panning shot from 19s to 30s, switches from the city appearance to person A from 30s to 32s, shoots person A with a close-up shot from 32s to 37s, switches from person A to the city from 37s to 39s, and shoots person A and the city streets with a long shot from 39s to 60s.

[0406] The recognition results of scene types in the image stream can be: the scene type of the 0-6s segment in the image stream (scene segment 1) is "city", the scene type of the 6s-18s segment in the image stream (scene segment 2) is "people", the scene type of the 18s-31s segment in the image stream (scene segment 3) is "city", the scene type of the 31s-38s segment in the image stream (scene segment 4) is "people", and the scene type of the 38s-60s segment in the image stream (scene segment 5) is "city".

[0407] The recognition results of transitions in the image stream can be: the transition category of the 5s to 7s segment in the image stream is "city to person", the transition category of the 17s to 19s segment in the image stream is "person to city", the transition category of the 30s to 32s segment in the image stream is "city to person", and the transition category of the 37s to 39s segment in the image stream is "person to city".

[0408] The recognition result of the shot type in the image stream can be: the shot type of the 0th to 5th s segment in the image stream is "aerial shot", the shot type of the 5th to 7th s segment in the image stream is "fast camera movement", the shot type of the 7th to 17th s segment in the image stream is "medium close shot", the shot type of the 17th to 19th s segment in the image stream is "fast camera movement", the shot type of the 19th to 30th s segment in the image stream is "shaking shot", the shot type of the 30th to 32th s segment in the image stream is "fast camera movement", the shot type of the 32th to 37th s segment in the image stream is "close-up shot", the shot type of the 37th to 39th s segment in the image stream is "fast camera movement", and the shot type of the 39th to 60th s segment in the image stream is "long shot".

[0409] The lens segments determined in the image stream may include: segment a from T1 to T2 in the image stream, the lens type of segment a is "aerial lens", T1 is greater than 0s and less than 5s, and T2 is greater than 5s and less than 6s. Segment b from T3 to T4 in the image stream, the lens type of segment b is "medium close-up", T3 is greater than 7s and less than 17s, and T4 is greater than 17s and less than 18s. Segment c from T5 to T6 in the image stream, the lens type of segment c is "panning lens", T5 is greater than 19s and less than 30s, and T6 is greater than 30s and less than 31s. Segment d from T7 to T8 in the image stream, the lens type of segment d is "close-up lens", T7 is greater than 32s and less than 37s, and T8 is greater than 37s and less than 38s. Segment e from T9 to 60s in the image stream, the lens type of segment e is "long lens", and T9 is greater than 39s and less than 60s. The following will specifically describe the method of determining the lens segments.

[0410] The shot types of the T2-T3 segment in the image stream, the T4-T5 segment in the image stream, the T6-T7 segment in the image stream, and the T8-T9 segment in the image stream are all “fast camera movement”.

[0411] Taking the editing style currently selected or matched by the electronic device as the editing style of the movie "A" in the editing style interface (ie, the target editing style), the target editing rules corresponding to the editing style of the movie "A" are shown in Table 6 below as an example.

[0412] Table 6

[0413]

[0414]

[0415] According to the target editing rules shown in Table 6, it can be determined that the wonderful shot segments in the image stream include: segment b1, segment c1, segment d1 and segment e1. Among them, segment b1 is a continuous segment with a duration of 3s in the image stream T3-T4, segment c1 is a continuous segment with a duration of 5s in the image stream T5-T6, segment d1 is a continuous segment with a duration of 2s in the image stream T7-T8, and segment e1 is a continuous segment with a duration of 10s in the image stream T9-60s.

[0416] The electronic device splices the wonderful shot segments according to the combination logic indicated by the target editing rule shown in Table 6. The combination logic of the target editing rule shown in Table 6 is: splice the wonderful shot segments in sequence according to the rows of the list. Specifically, splicing area 1 splices the end position of segment c1 with the start position of segment d1, splicing area 2 splices the end position of segment d1 with the start position of segment b1, and splicing area 3 splices the end position of segment b1 with the start position of segment e1.

[0417] In other embodiments, if the target editing style is not determined and there is no editing style with the highest matching degree, the electronic device can crop all the shot segments of the original video according to the default editing rules, remove invalid shot segments, such as shot segments with fast camera movements, and then splice the shot segments together in chronological order to obtain a new video 1. For example, the end position of segment a is spliced ​​with the start position of segment b, the end position of segment b is spliced ​​with the start position of segment c, the end position of segment c is spliced ​​with the start position of segment d, and the end position of segment d is spliced ​​with the start position of segment e.

[0418] In an embodiment of the present application, when an electronic device determines a lens segment according to the lens type of a plurality of image frames or determines a scene segment according to the scene type of a plurality of image frames, an observation queue may be set, and the start time and end time of the segment may be determined based on the observation queue. The electronic device may store the lens type (or scene type) of the image frame into the observation queue, and then output an invalid state and a first lens type (or a first scene type) according to the lens type (or scene type) in the observation queue. And the start time and end time of the lens segment (or scene segment) are determined according to the invalid state and the first lens type (or the first scene type). When the first lens type (or the first scene type) appears for the first time after the invalid state is output, the image frame a stored in the observation queue when the first lens type (or the first scene type) appears is determined to be the start frame. When the invalid state appears for the first time after the first lens type (or the first scene type) is output, the image frame b stored in the observation queue when the invalid state appears is determined to be the end frame. Thus, it can be determined that the starting frame of the shot segment (scene segment) is image frame a, the ending frame is image frame b, and the corresponding shot type (or scene type) is the first shot type (or first scene type), and thus the corresponding label can also be determined. Among them, the first shot type is the shot type that appears most times in the observation queue. Accordingly, the first scene type is the scene type that appears most times in the observation queue.

[0419] The following takes a certain shot segment as an example.

[0420] See also Fig.11 The method for determining a shot segment provided by an embodiment of the present application is exemplarily introduced. The method for determining a shot segment is applied to an electronic device, and can be specifically applied to a shot recognition module. For example, the above-mentioned shot size recognition unit, the camera movement recognition unit, the shot angle position recognition unit, and the shot composition recognition unit can each execute Fig.11 The method shown may include steps S101 to S106.

[0421] Step S101, setting an observation queue and setting a preset storage length of the observation queue.

[0422] In an embodiment of the present application, for each dimension of lens type, the electronic device can set a corresponding observation queue. For example, the lens size recognition unit is set with its corresponding observation queue A, and the camera movement and composition module is set with its corresponding observation queue B, and so on. Specifically, taking the original video C including image frames from the 1st frame to the 100th frame as an example, the lens size recognition unit stores the lens type of the image frame in the original video C identified to the observation queue A, and continues to perform the following steps to determine the lens segment related to the lens size in the original video C. Correspondingly, the camera movement and composition unit stores the lens type of the image frame in the original video C identified to the observation queue B, and continues to perform the following steps to determine the lens segment related to the lens composition in the original video C. The same applies to other recognition units.

[0423] Among them, the observation queue can be implemented as a stack, such as a First In First Out (FIFO) stack. The preset storage length refers to the data capacity that the observation queue can store at the same time. For example, when the observation queue is a First In First Out stack, the length of the First In First Out stack is the preset storage length. The length of the First In First Out stack indicates the number of elements that the First In First Out stack can store at the same time. For example, the length of the First In First Out stack is 10, that is, the First In First Out stack can store 10 elements (such as 10 data structures), and when the 11th element continues to be stored, the first stored element is popped out of the stack.

[0424] Step S102, obtaining the lens type of the image frame, and storing the lens type of the image frame in an observation queue.

[0425] In an embodiment of the present application, after obtaining the lens type of the image frame, the unit in the lens recognition module can store the lens type of the image frame and the frame number of the image frame (or the timestamp of the image frame (such as the timestamp of generating the image frame)) in the corresponding observation queue at the same time. Exemplarily, if the lens size recognition unit recognizes that the lens type of the 77th image frame is a close-up, the frame number "77" and the lens type "close-up" are bound to a data structure (i.e., a first structure), and the data structure is stored as an element in the corresponding observation queue (e.g., observation queue A). If the lens composition recognition unit recognizes that the lens type of the 77th image frame is a single subject, the frame number "77" and the lens type "single subject" are bound to a data structure, and the data structure is stored as an element in the corresponding observation queue (e.g., observation queue B).

[0426] Step S103, determining whether the preset storage length has been reached.

[0427] In the embodiment of the present application, when the lens type (or data structure) of an image frame is stored in the observation queue, it is determined whether the preset storage length is reached. If not, step S104 is executed, and if yes, step S105 is executed. Reaching the preset storage length means that the observation queue is full of data. For example, in the above example, the first-in-first-out stack stores 10 elements.

[0428] Step S104, outputting an invalid state, and continuing to execute step S102, that is, continuing to store the shot type of the next image frame in the image stream into the observation queue until all the image frames in the image stream are polled.

[0429] Step S105, determining whether the proportion of the first shot type in the observation queue reaches a preset ratio.

[0430] The content of the first lens type may refer to the above. Reaching the preset ratio may be that the proportion of the first lens type is greater than or equal to the preset ratio. The preset ratio may be set according to actual conditions, such as 60%, 80%, etc.

[0431] In the embodiment of the present application, it is determined whether the proportion of the first lens type in the observation queue reaches a preset ratio. If so, step S106 is executed; if not, step S104 is executed.

[0432] Step S106, outputting the first lens type, and continuing to execute step S102, that is, continuing to store the lens type of the next image frame in the image stream into the observation queue until all the image frames in the image stream are polled.

[0433] Step S107: determining a shot segment according to the first shot type and the invalid state.

[0434] In the embodiment of the present application, after step S101 and step S106, each recognition unit can obtain the first lens type and the invalid state. When the invalid state is converted to the output lens type, the start time of the lens segment can be determined, and when the output lens type is converted to the invalid state, the end time of the lens segment can be determined.

[0435] It can be understood that for the same original video, each shot recognition unit finally obtains a different first shot type.

[0436] For example, the observation queue is a first-in-first-out stack, the length of the first-in-first-out stack is 10, and the preset ratio is 60%. Fig.10 The process of dividing the shot segments is shown.

[0437] Taking the 0-5s segment having 125 frames as an example, this article describes how to extract the T1-T2 lens segment based on the 0-T2 segment.

[0438] The lens type of the image frame of the 0-5s segment is an aerial lens, and the lens type of the image frame of the 5s-6s segment is a fast camera movement. The first structure corresponding to the image frame of the 0-5s segment is stored in the first-in-first-out stack in sequence. When the first structure corresponding to the first frame of the image frame in the 0-5s segment is stored in the first-in-first-out stack, the preset storage length is not reached at this time, and an invalid state is output. When the first structure corresponding to the second frame of the image frame in the 0-5s segment is stored in the first-in-first-out stack, the preset storage length is still not reached at this time, and an invalid state is output. Similarly, when the first structure corresponding to the 10th frame of the image frame in the 0-5s segment is stored in the first-in-first-out stack, the preset storage length is reached at this time, and it is determined whether the proportion of the first lens type with the most appearances in the current first-in-first-out stack in the first-in-first-out stack reaches the preset ratio. Based on the fact that the lens types of the first frame and the 10th frame are both "aerial lens", the first lens type is "aerial lens", and the proportion of the first lens type in the first-in-first-out stack reaches the preset ratio, and the first lens type "aerial lens" is output. Similarly, the first structure corresponding to the 125th image frame in the 0-5s segment is continuously stored in the FIFO stack, and the first lens type "aerial lens" is output. The first structure corresponding to the 1st image frame of the 5s-6s segment is continuously input into the FIFO stack, based on the lens type of the 1st image frame of the 5s-6s segment being fast camera movement, at this time, there are 9 image frames in the FIFO stack whose lens type is "aerial lens", and 1 image frame whose lens type is "fast camera movement", and the proportion of the first lens type in the FIFO stack reaches the preset ratio, and the first lens type "aerial lens" is continuously output. Similarly, the first structure corresponding to the 5th image frame in the 5s-6s segment is continuously stored in the FIFO stack, and there are 5 image frames in the FIFO stack whose lens type is "aerial lens", and the lens type of 5 image frames is "fast camera movement", and the first lens type can be "aerial lens" or "fast camera movement", and the proportion of the first lens type in the FIFO stack does not reach the preset ratio of 60%, and an invalid state is output.

[0439] In an embodiment of the present application, based on the previous output result being an invalid state, the current image frame that first meets the proportion of the first lens type that reaches a preset ratio is used as the starting frame of the lens segment. As in the above example, the 10th image frame in the 0-5s segment is used as the starting frame of the lens segment a, and the timestamp of the 10th image frame corresponds to the T1 moment in the image stream. Based on the previous output result being the first lens type, the current image frame that first meets the proportion of the first lens type that does not reach a preset ratio is used as the ending frame of the lens segment. As in the above example, the 5th image frame in the 5s-6s segment is used as the ending frame of the lens segment a, and the timestamp of the 5th image frame in the 5s-6s segment corresponds to T2 in the image stream. In this way, the starting and ending times of the lens segment a can be determined, and the lens type of the lens segment a is the first lens type.

[0440] In an embodiment of the present application, when an electronic device determines a shot segment based on the shot types of multiple image frames or determines a scene segment based on the scene types of multiple image frames, it can determine the start and end times of the shot segment (or scene segment) by comparing the scene types or shot types of N adjacent image frames, according to the comparison results of the N adjacent image frames, where N is an integer greater than or equal to 2.

[0441] The following example takes the scene type as a motion scene clip, and N is 2. Fig. 12A , a method for determining a scene segment provided in an embodiment of the present application is exemplarily introduced. The method for determining a scene segment is applied to an electronic device, and specifically can be applied to a scene recognition module.

[0442] Exemplarily, the electronic device (scene recognition module) obtains two adjacent frames of images A and B in an image stream, where image frame A precedes image frame B in the image stream, and determines whether the scene types of image frame A and image frame B both include motion.

[0443] When the scene types of image frame A and image frame B both include motion, state 1 is output. State 1 indicates that it is possible to continue to determine whether the scene type of the image frame is motion. In response to state 1, the electronic device may continue to acquire the scene type of the next image frame C to continue to determine whether the scene types of image frame B and image frame C both include motion. Alternatively, in response to state 1, the electronic device continues to acquire two adjacent image frames C and D after image frame B in the image stream to continue to determine whether the scene types of image frame C and image frame D both include motion.

[0444] When the scene type of image frame A does not include motion, and the scene motion type of image frame B includes motion, state 2 is output. State 2 may indicate that image frame B is the starting frame of a scene segment. In some embodiments, the electronic device continues to acquire the scene type of the image frame after image frame B in the image stream in response to outputting state 2. When the scene type of the image frame after image frame B is motion, image frame B may be determined to be the starting frame of the scene segment. When the scene type of the image frame after image frame B is not motion, it may be determined that there is a detection error in image frame B, and image frame B may be ignored.

[0445] When the scene motion type of image frame B does not include motion, and the scene motion type of image frame A includes motion, state 3 is output. State 3 may indicate that image frame A is the end frame of the scene segment. In some embodiments, the electronic device responds to output state 3 and detects that the scene type of the image frame before image frame A in the image stream is motion, and image frame A may be determined to be the end frame of the scene segment. When the scene type of the image frame before image frame A in the image stream is not motion, it may be determined that there is a detection error in image frame A, and image frame A may be ignored.

[0446] When the scene motion types of both image frames A and B do not include motion, state 4 is output. State 4 may indicate that the judgment of the scene segment whose scene type is motion is ended, that is, the judgment of the scene type as motion is ended. State 4 may indicate that the current scene type is not motion.

[0447] The following is an example of a shot clip that determines the shot type as a moving shot, with N being 2. Fig. 12B , a method for determining a shot segment provided in an embodiment of the present application is exemplarily introduced. The method for determining a shot segment is applied to an electronic device, and specifically can be applied to a shot size recognition unit.

[0448] Exemplarily, the electronic device obtains two adjacent image frames A and B in an image stream, where image frame A is before image frame B in the image stream, and determines whether the shot types of image frame A and image frame B both include camera movement.

[0449] When the shot types of image frame A and image frame B both include camera movement, state 1 is output. State 1 indicates that it is possible to continue to determine whether the shot type of the image frame is camera movement. In response to state 1, the electronic device may continue to obtain the shot type of the next image frame C to continue to determine whether the shot types of image frame B and image frame C both include camera movement. Alternatively, in response to state 1, the electronic device may continue to obtain two adjacent image frames C and D after image frame B in the image stream to continue to determine whether the shot types of image frame C and image frame D both include camera movement.

[0450] When the shot type of image frame A does not include camera movement, and the shot type of image frame B includes camera movement, state 2 is output. State 2 may indicate that image frame B is the starting frame of a shot segment. In some embodiments, the electronic device continues to acquire the shot type of the image frame after image frame B in the image stream in response to outputting state 2. When the shot type of the image frame after image frame B is camera movement, image frame B may be determined to be the starting frame of the shot segment. When the shot type of the image frame after image frame B is not camera movement, it may be determined that there is a detection error in image frame B, and image frame B may be ignored.

[0451] When the shot type of image frame B does not include camera movement, and the shot type of image frame A includes camera movement, state 3 is output. State 3 may indicate that image frame A is the end frame of the shot segment. In some embodiments, the electronic device responds to output state 3 and detects that the shot type of the image frame before image frame A in the image stream is camera movement, and image frame A may be determined to be the end frame. When the shot type of the image frame before image frame A in the image stream is not camera movement, it may be determined that there is a detection error in image frame A, and image frame A may be ignored.

[0452] When the shot types of both image frames A and B do not include camera movement, state 4 is output. State 4 may indicate that the determination of the shot segment whose shot type is camera movement is ended, that is, the determination of the shot type as camera movement is ended. State 4 may indicate that the current shot type is not camera movement.

[0453] The following takes determining a specific scene segment (wonderful shot segment) where the shot type is camera movement and the scene type is character, and N is 2 as an example. For example, the electronic device can determine the wonderful shot segment according to the target editing rule indicating that the shot type is camera movement and the scene type is character. Fig. 12C .

[0454] The electronic device obtains two adjacent frames of images A and B in an image stream, where image frame A is before image frame B in the image stream, and determines whether scene types of image frame A and image frame B both include people.

[0455] When the scene types of the image frame A and the image frame B both include people, it is determined whether the shot types of the image frame A and the image frame B both include camera movement.

[0456] When the shot types of both the image frame A and the image frame B include camera movement, the output state is 1. The state 1 indicates that it is possible to continue to determine whether the scene type of the image frame is a person and whether the shot type is camera movement.

[0457] When the shot types of image frame A and image frame B do not both include camera movement, and the shot type of image frame B includes camera movement, state 2 is output. State 2 may indicate that image frame B is the starting frame of a wonderful shot clip (i.e., the scene type is character and the shot type is camera movement).

[0458] When the shot types of image frame A and image frame B do not both include camera movement, and the shot type of image frame A includes camera movement, state 3 is output. State 3 may indicate that image frame A is the end frame of a wonderful shot clip (i.e., the scene type is character and the shot type is camera movement).

[0459] When the shot types of both image frame A and image frame B do not include camera movement, state 4 is output. State 4 may indicate that the judgment of the wonderful shot clips whose scene type is character and shot type is ended, that is, the judgment of the scene type is character and the shot type is camera movement is ended. State 4 may indicate that the current scene type is not character and the shot type is not camera movement.

[0460] When it is determined that the scene types of image frame A and image frame B do not both include people, and the scene type of image frame B includes people, determine whether the shot type of image frame B includes camera movement. If so, output state 2, otherwise, output state 4.

[0461] When it is determined that the scene types of image frame A and image frame B do not both include people, and the scene type of image frame A includes people, determine whether the shot type of image frame A includes camera movement. If so, output state 3, otherwise, output state 4.

[0462] When it is determined that the scene types of image frame A and image frame B do not include a person, state 4 is output.

[0463] The following exemplarily introduces another specific implementation of determining scene segments according to scene types of multiple image frames.

[0464] The electronic device performs transition detection on the image stream, determines the transition position and transition category of the scene transition occurring in the image stream, and then divides the image stream into multiple scene segments based on the scene type of each image frame in the image stream and the transition position and transition category of the scene transition occurring in the image stream, thereby determining the scene type of each scene segment, and determining the start frame and end frame of the scene segment according to the transition position.

[0465] Accordingly, the electronic device pre-trains a detection model for performing shot conversion detection, and the training of the detection model can refer to the above-mentioned transition recognition model. Based on the detection model, the image stream can be subjected to shot conversion detection, and the transition position and conversion category of the shot conversion in the image stream can be determined. Then, the electronic device divides the image stream into a plurality of shot segments based on the shot type of each image frame in the image stream and the transition position and transition category of the shot conversion in the image stream, thereby determining the shot type of each shot segment, and determining the start frame and end frame of the shot segment according to the transition position of the shot conversion.

[0466] It can be understood that different ways of determining shot segments (shot tags) or scene segments (scene tags) may result in differences in the determined shot segments (shot tags) or scene segments (scene tags), such as Fig.10The determined shot segments are different from the actual shot segments by several frames, while the determined scene segments are consistent with the actual scene segments. The method of determining the shot segments (shot tags) or scene segments (scene tags) can be similar or different, and this application does not make specific limitations on this.

[0467] See also Fig.13 , exemplarily introduces another video processing method provided in an embodiment of the present application, which is applied to an electronic device and may include steps S130 to S138, wherein the contents of steps S133 to S138 may refer to the above.

[0468] Step S130, obtaining the processing method of the original video.

[0469] In an embodiment of the present application, the electronic device obtains the original video indicated by the input in response to the user's input, so as to process the original video to obtain a new video. When the original video is obtained, the processing method of the original video is obtained. When a video tag is stored in the video file of the original video, it can be determined that the processing method of the original video is an online mode. When a video tag is not stored in the video file of the original video, it can be determined that the processing method of the original video is an offline mode. Among them, the video tag includes one or more of a shot tag and a scene tag.

[0470] Step S131, obtaining the target clipping rule and the video tag of the original video.

[0471] In the embodiment of the present application, when it is determined that the processing mode of the original video is the online mode, the electronic device obtains the video tag from the video file of the original video. The method for the electronic device to obtain the target clipping rule can be referred to above and will not be repeated here.

[0472] Step S132: determining a plurality of new video segments in the original video according to the target editing rule and the video tag.

[0473] In the embodiment of the present application, when the video tag only includes the shot tag, the electronic device can determine multiple shot segments in the original video according to the target editing rule and the shot tag. When the video tag includes the shot tag and the scene tag, the electronic device can determine multiple wonderful shot segments in the original video according to the target editing rule, the shot tag and the scene tag. When the video tag includes the scene tag, the electronic device can determine multiple scene segments in the original video according to the target editing rule and the scene tag.

[0474] In other embodiments, the electronic device determines multiple new video segments in the original video based on the required segments indicated by the target editing rule. When the video tag does not include a scene tag, but the target editing rule indicates the scene type of the required segment, the electronic device can perform scene detection on the image frames in the image stream to determine the scene segments or scene tags in the image stream, and then determine multiple new video segments in the original video based on the target editing rule, the shot tags and the scene segments (or scene tags). Correspondingly, when the video tag does not include a scene tag, but the target editing rule indicates the shot type of the required segment, the electronic device can perform shot detection on the image frames in the image stream to determine the shot segments or shot tags in the image stream, and then determine multiple new video segments in the original video based on the target editing rule, the scene tags and the shot segments (or shot tags).

[0475] The electronic device determines the multiple new video segments in the original video according to the target editing rule and the video tag, which can be referred to above and will not be described in detail here.

[0476] Step S133, editing multiple new video clips according to the target editing rule to obtain a new video.

[0477] Offline mode, step S134, obtaining target clipping rules.

[0478] Step S135, obtaining the image stream in the original video.

[0479] Step S136: Detect the image stream to determine the type of each image frame in the image stream.

[0480] The type of the image frame may include one or more of a shot type and a scene type.

[0481] In some embodiments, the image stream is detected to determine the type and score of each image frame in the image stream, such as a shot score and a scene score.

[0482] Step S137, determining a new video segment in the original video according to the types of the multiple image frames.

[0483] In an embodiment of the present application, the electronic device can determine a new video segment in the original video based on the types and / or scores of multiple image frames. The content of the new video segment can be determined by referring to the above-mentioned determination of the shot segment based on the shot types of multiple image frames, or the determination of the content of the scene segment based on the scene types of multiple image frames.

[0484] In some embodiments, the electronic device may identify the type of the image frame while determining the new video segment. Specifically, the following steps may be included: Step 1, the electronic device obtains the lens type of the image frame. Step 2, the electronic device determines whether the image frame meets the lens type required in the target editing rule. If so, execute step 3, if not, execute step 1 to continue to obtain the lens type of the next image frame. Step 3, the electronic device marks the image frame. Step 4, the electronic device determines multiple continuous image frames with consistent lens types in the marked image frames as corresponding marked video segments. Step 5, the electronic device determines whether the marked video segment meets the scene type required by the target editing rule. If so, execute step 6, if not, continue to execute step 5 to obtain the next marked video segment. Step 6, remove the mark of the marked video segment. Step 7, determine that the marked video segment is a new video segment in the original video.

[0485] In some embodiments, when it is determined that the processing mode of the original video is offline mode, several image frames are randomly selected from the original video, and the several image frames are detected to determine the types of the several image frames (such as one or more of the lens type and the scene type). Key frames are selected according to the types of the several image frames, such as selecting image frames including the lens type and / or the scene type indicated by the target editing rule, and the selected image frames are used as key frames. Then, other image frames adjacent to the key frame in the original video are detected to determine the types of other image frames adjacent to the key frame. After determining the types of other image frames adjacent to the key frame, lens segments are determined according to the types of other image frames adjacent to the key frame and the key frame. By analogy, multiple lens segments in the original video are determined. When the determined multiple lens segments meet the required segments indicated by the target editing rule, the multiple lens segments are combined according to the combination logic of the target editing rule.

[0486] When the determined multiple shot segments do not meet the required segments indicated by the target editing rule, such as the duration of the determined shot segment is shorter than the duration of the required segment, then continue to detect other adjacent image frames of the shot segment, such as shot segment A is from the 50th frame to the 200th frame, then continue to detect other adjacent image frames of the shot segment A, such as the 49th frame, the 201st frame, etc. Then, according to the types of other adjacent image frames of the shot segment A, the image frames included in the shot segment A are expanded, thereby obtaining a new shot segment. The duration of the new shot segment is longer than the original shot segment. Subsequent editing can be performed based on the new shot segment.

[0487] When the determined multiple shot segments do not meet the required segments indicated by the target editing rule, such as the duration of the determined shot segments is longer than the duration of the required segments, then according to the duration indicated by the target editing rule, high-scoring and adjacent image frames are selected from the shot segments to form a new shot segment, and the duration of the new shot segment is shorter than the duration of the original shot segment. The new shot segment can be subsequently edited.

[0488] In some embodiments, after the original video is detected, when the original video does not include the shot segment indicated by the target editing rule, and it is detected that the editing rule library includes shot materials of the shot type indicated by the target editing rule, when the new video segment of the original video is combined according to the combination logic of the target editing rule, the shot material is added to the corresponding position to obtain a new video.

[0489] In some embodiments, when the target editing rule indicates a corresponding sound material, the corresponding sound material is obtained from the editing rule library and added to the new video.

[0490] In some embodiments, when the target editing rule indicates the corresponding special effect material and the method of adding the special effect material, the corresponding special effect material is obtained from the editing rule library, and the special effect material is added to the new video according to the indicated adding method.

[0491] Step S138, editing multiple new video clips according to the target editing rule to obtain a new video.

[0492] In an embodiment of the present application, when processing the original video, the processing method of the original video is first obtained, so that when the original video includes a video tag, the detection operation on the original video can be reduced, thereby improving the video processing efficiency.

[0493] See also Fig.14 , which exemplarily introduces the functional module diagram of the video processing system provided in an embodiment of the present application.

[0494] like Fig.14 As shown, the video processing system 200 may include: a data module 201 , a perception module 202 , a fusion module 203 , and a video processing module 204 .

[0495] The data module 201 is used to obtain the image stream (such as the image stream when recording the video) and the audio stream of the original video. The data module 201 can pass the image stream to the perception module 202 and pass the image stream and the audio stream to the video processing module 204.

[0496] The perception module 202 can perform video understanding on the image stream, wherein the video understanding includes shot detection, transition detection and scene detection. Specifically, the perception module 202 can perform shot detection on the image stream, and identify the shot type and / or shot score of each image frame in the image stream. The perception module 202 can perform scene detection on the image stream, and identify the scene type and / or scene score of each image frame in the image stream. The perception module 202 can perform transition detection on the image stream, and identify the transition position and transition type of the scene conversion in the image stream. Among them, the specific contents of the shot detection, transition detection and scene detection of the image stream can be referred to the above, and will not be repeated here.

[0497] The perception module 202 may pass the shot type and / or shot score of each image frame to the fusion module 203. The perception module 202 may pass the scene type, scene score, and transition position and transition category of each image frame in the image stream where the scene transition occurs to the fusion module 203.

[0498] The fusion module 203 may determine a shot label (or shot segment) based on the shot types of the multiple image frames. The fusion module 203 may divide the image stream into multiple screen segments based on the transition position where the scene transition occurs in the image stream. The fusion module 203 may determine the scene type of each screen segment in the multiple screen segments based on the transition position and transition type where the scene transition occurs and the scene type of each image frame, thereby determining a scene label (or scene segment).

[0499] The fusion module 203 may pass the shot tags (or shot segments), scene tags (or scene segments), shot scores, and scene scores to the video processing module 204 .

[0500] The video processing module 204 can mix the audio stream and the image stream into the original video. The video processing module 204 can determine multiple new video segments according to the editing rules, scene tags, shot tags, shot scores and scene scores, and then splice the multiple new video segments according to the editing rules to obtain a new video. For the specific content, please refer to the above, which will not be repeated here.

[0501] The video processing module 204 can splice multiple new video clips into a new video according to the editing rules, wherein the splicing process includes: combining multiple new video clips according to the combination logic, adding special effect materials, adding music, etc. For the specific content, please refer to the above, which will not be repeated here.

[0502] The video processing module 204 can output the original video and the wonderful video.

[0503] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0504] An embodiment of the present application also provides an electronic device, comprising at least a processor and a memory, wherein the processor and the memory are coupled, and the memory is used to store a computer program. When the computer program is executed by the processor, the electronic device can indicate the video processing method of the above embodiment.

[0505] An embodiment of the present application also provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the video processing method of the above embodiment.

[0506] An embodiment of the present application further provides a computer storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the video processing method as described in the above embodiment.

[0507] Among them, the electronic device, computer storage medium or chip system provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0508] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0509] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0510] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0511] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0512] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0513] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A video processing method, characterized in that: Applied to electronic equipment, the method comprises: Displaying a video album interface, the video album interface including video options of one or more videos stored in the electronic device; In response to an input of selecting a target video option from the video options on the video album interface, obtaining an original video indicated by the target video option; Obtaining shot tags and scene tags in the original video, wherein the shot tags include the shot type, start time, and end time of the shot segment, and the scene tags include the scene type, start time, and end time of the scene segment; In response to the first input, the original video is edited according to a target editing rule, the shot tag and the scene tag to obtain a wonderful shot segment, wherein the target editing rule indicates a shot type and a scene type of a desired segment; The wonderful footage segments are combined according to the target editing rule to obtain a new video.

2. The method according to claim 1, characterized in that The method further comprises: In response to the second input, displaying an editing style interface, the editing style interface including editing options corresponding to each editing style; In response to an input of selecting a target editing option from the editing options of the editing style interface, taking the editing style corresponding to the target editing option as the target editing style; The editing rule corresponding to the target editing style is used as the target editing rule.

3. The method according to claim 1, characterized in that: The electronic device stores an editing rule library, wherein the editing rule library stores editing rules corresponding to various editing styles, and the method further includes: When no input for indicating the user to select the target editing style is obtained, determining the editing rule with the highest matching degree in the editing rule library according to the shot type in the original video; Determining whether the editing rule with the highest matching degree meets the preset requirements; If yes, taking the clipping rule with the highest matching degree as the target clipping rule; If not, the default clipping rule is used as the target clipping rule.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When the original video does not include the shot tag, performing shot detection on the image frames in the original video to obtain the shot tag of the original video, and saving the shot tag; When the original video does not include the scene label, performing scene detection on the image frames in the original video to obtain the scene label of the original video, and saving the scene label; When the original video does not include the shot tag and the scene tag, shot detection and scene detection are performed on the image frames in the original video to obtain the shot tag and the scene tag of the original video, and the shot tag and the scene tag are saved.

5. A video processing method, characterized in that: Applied to electronic equipment, the method comprises: In response to the third input, start recording the original video and display a video recording interface, wherein the video recording interface is used to display an image stream of the original video acquired in real time by a camera of the electronic device; Performing shot detection and scene detection on the image frames in the image stream to obtain a plurality of shot segments, shot tags and scene tags, wherein each shot tag corresponds to one shot segment, the shot tag includes the shot type, start time and end time of the corresponding shot segment, and the scene tag includes the scene type, start time and end time of the scene segment; In response to a fourth input, ending recording of the original video; According to the target editing rule, the shot tag and the scene tag, the plurality of shot segments in the original video are edited to obtain wonderful shot segments, wherein the target editing rule indicates the shot type and scene type of the required segment; Combining the wonderful footage segments according to the target editing rule to obtain a new video; The original video, the new video and the shot label are saved.

6. The method according to claim 5, characterized in that The method further comprises: In response to a fifth input, displaying an editing style interface, the editing style interface including editing options corresponding to each editing style; In response to an input of selecting a target editing option from the editing options of the editing style interface, taking the editing style corresponding to the target editing option as the target editing style; The editing rule corresponding to the target editing style is used as the target editing rule.

7. The method according to claim 6, characterized in that After taking the editing rule corresponding to the target editing style as the target editing rule, the method further includes: Acquire the shot type and the scene type of the required segment; A prompt interface is displayed, wherein the prompt interface includes the lens type and the scene type.

8. The method according to claim 5, characterized in that The electronic device stores an editing rule library, wherein the editing rule library stores editing rules corresponding to various editing styles, and the method further includes: In response to a sixth input, displaying editing style options, wherein the editing style options are used for the user to select an editing style; When the editing style option indicates to automatically match the editing style, determining the editing rule with the highest matching degree in the editing rule library according to the shot type in the original video; Determining whether the editing rule with the highest matching degree meets the preset requirements; If yes, taking the clipping rule with the highest matching degree as the target clipping rule; If not, the default clipping rule is used as the target clipping rule.

9. The method according to any one of claims 5 to 8, characterized in that The performing shot detection and scene detection on the image frames in the image stream comprises: In response to a seventh input, the shot detection and the scene detection are performed on the image frames in the image stream, and the seventh input instructs the electronic device to enter a one-record-many-acquire mode.

10. The method according to any one of claims 5 to 8, characterized in that The saving of the original video, the new video and the shot tag comprises: Saving the plurality of shot segments in the original video; Saving the shot tag to the video file of the original video; The original video, the new video and the footage are displayed to receive user input on any one or more of the original video, the new video and the footage.

11. The method according to claim 10, characterized in that The method further comprises: In response to an eighth input, displaying a one-record-multiple-acquisition interface, wherein the one-record-multiple-acquisition interface is used to display the footage clips acquired by the electronic device; In response to a selection of the shot segment in the one-record-multiple-receive interface, determining the selected shot segment as a target shot segment; In response to the ninth input, the target shot segment is edited according to a target editing rule to obtain a new video.

12. A video processing method, characterized in that: Applied to electronic equipment, the method comprises: Get the image stream of the original video; Performing shot detection on the image frames in the image stream to determine the shot type of the image frames; Determining a shot segment according to shot types of the plurality of image frames; Performing scene detection on the image frames in the image stream to determine the scene type of the image frames; Determining a scene segment according to scene types of the plurality of image frames; Editing multiple shot segments in the original video according to a target editing rule, the shot type and the scene type to obtain a wonderful shot segment, wherein the target editing rule indicates the shot type and scene type of the required segment; The wonderful footage segments are combined according to the target editing rule to obtain a new video.

13. The method according to claim 12, characterized in that The electronic device stores a lens type library, wherein the lens type library includes lens types in one or more dimensions: a lens type related to lens size, a lens type related to lens composition, a lens type related to lens movement, a lens type related to lens angle and position, and a lens type related to lens duration; The performing shot detection on the image frames in the image stream to determine the shot type of the image frames comprises: Shot detection is performed on the image frames in the image stream according to the shot type library to determine the shot type of the image frame.

14. The method according to claim 13, characterized in that The electronic device stores a camera algorithm library, and performing lens detection on the image frame in the image stream according to the lens type library to determine the lens type of the image frame includes: Calling the detection algorithm in the camera algorithm library to perform lens detection on the image frames in the image stream to obtain the detection results of the image frames; The lens type of the image frame is determined according to the lens type library and the detection result of the image frame.

15. The method according to any one of claims 12 to 14, characterized in that The electronic device stores an editing rule library, the editing rule library includes editing rules corresponding to various editing styles, the editing styles include one or more of the following: editing styles related to scenes, editing styles related to directors, and editing styles related to film and television works, and the method further includes: In response to an input of selecting a target editing style from the editing styles in the editing rule library, taking the editing rules corresponding to the target editing style as the target editing rules; or, When no input for indicating the user to select the target editing style is obtained, determining the editing rule with the highest matching degree in the editing rule library according to the shot type in the original video; Determining whether the editing rule with the highest matching degree meets the preset requirements; If yes, taking the clipping rule with the highest matching degree as the target clipping rule; If not, the default clipping rule is used as the target clipping rule.

16. The method according to claim 15, characterized in that The editing rule with the highest matching degree indicates the shot type of the required segment, and judging whether the editing rule with the highest matching degree meets the preset requirements includes: Calculating the number of pending shot segments of the original video to obtain a first value, wherein the shot type of the pending shot segment is the shot type of the required segment; Calculating the number of the required fragments to obtain a second value; Calculating a ratio of the first value to the second value; When the ratio is less than a preset threshold, it is determined that the clipping rule with the highest matching degree does not meet the preset requirement; When the ratio is greater than or equal to the preset threshold, it is determined that the clipping rule with the highest matching degree meets the preset requirement.

17. The method according to any one of claims 12 to 14 and 16, characterized in that The step of editing the plurality of shot segments in the original video according to the target editing rule, the shot type and the scene type to obtain the wonderful shot segments comprises: According to the position of the shot fragment, the shot type of the shot fragment, the position of the scene fragment, the scene type of the scene fragment and the target editing rule, a plurality of wonderful shot fragments in the image stream are determined, the wonderful shot fragments including the shot type and scene type indicated by the target editing rule; and the plurality of wonderful shot fragments are edited according to the target editing rule.

18. The method according to claim 17, characterized in that Determining a plurality of wonderful shot segments in the image stream according to the position of the shot segment, the shot type of the shot segment, the position of the scene segment, the scene type of the scene segment and the target editing rule comprises: Determining candidate scene segments according to the scene type of the required segment, the position of the scene segment, and the scene type of the scene segment, wherein the candidate scene segments include the scene type of the required segment; Determine the wonderful shot segment from the candidate scene segments according to the shot type of the required segment, the position of the shot segment and the shot type of the shot segment; or, determining candidate shot segments according to the shot type of the required segment, the position of the shot segment and the shot type of the shot segment, wherein the candidate shot segments include the shot type of the required segment; The wonderful shot segment is determined from the candidate shot segments according to the scene type of the required segment, the position of the scene segment and the scene type of the scene segment.

19. The method according to claim 17, characterized in that The combination logic of the target editing rule indicates the position of the required segment in the new video, and combining the wonderful shot segments according to the target editing rule to obtain the new video includes: Combine multiple wonderful shot clips according to the combination logic of the target editing rule to obtain the new video; or, Adjusting a plurality of the wonderful shot segments according to the target editing rule; The adjusted wonderful shot segments are combined according to the combination logic of the target editing rule to obtain the new video.

20. The method according to any one of claims 12 to 14, 16, 18 and 19, characterized in that: The step of determining a shot segment according to the shot types of the plurality of image frames comprises: Setting an observation queue and setting a preset storage length of the observation queue; storing the lens type of the image frame in the observation queue; When the number of lens types stored in the observation queue reaches the preset storage length and the proportion of the first lens type in the observation queue reaches a preset ratio, outputting the first lens type, where the first lens type is the lens type with the largest proportion in the observation queue; When the number of lens types stored in the observation queue does not reach the preset storage length, or when the number of lens types stored in the observation queue reaches the preset storage length and the proportion of the first lens type that does not exist reaches the preset ratio, an invalid state is output; A shot segment is determined according to the invalid state and the first shot type.

21. The method according to claim 20, characterized in that The setting of the observation queue comprises: The corresponding observation queue is set for the lens type of each dimension.

22. The method according to claim 20, characterized in that Determining a shot segment according to the invalid state and the first shot type comprises: In response to switching from outputting the invalid state to outputting the first shot type, determining a start time of the shot segment; In response to the state of switching from outputting the first shot type to outputting an invalid state, determining a termination time of the shot segment; The first shot type is used as the shot type of the shot segment.

23. The method according to any one of claims 12 to 14, 16, 18, 19, 21, 22, characterized in that The step of determining a shot segment according to the shot types of the plurality of image frames comprises: Comparing shot types of N adjacent image frames in the image stream to obtain a comparison result, where N is an integer greater than or equal to 2; The shot segment is determined according to the comparison result of the adjacent N image frames.

24. The method according to claim 23, characterized in that When N is 2, determining the shot segment according to the comparison result of the adjacent N image frames includes: When the comparison result indicates that both the front frame and the rear frame of the adjacent N frames include the second shot type, a first state is output, the front frame is an image frame before the rear frame of the adjacent N frames, and the first state is used to indicate to continue to determine the second shot type in the image stream; When the comparison result indicates that the latter frame among the adjacent N image frames includes the second shot type, and the former frame does not include the second shot type, determining the latter frame as the start frame of the shot segment; When the comparison result indicates that the first frame in the adjacent N image frames includes the second shot type and the second frame does not include the second shot type, determining that the first frame is the end frame of the shot segment; The second shot type is used as the shot type of the shot segment.

25. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the processor and the memory are coupled, and the memory is used to store a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 24.

26. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 24.

27. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 24.

Images