Video processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a video processing method and device, electronic equipment and a storage medium, and the method comprises the steps: displaying a candidate item corresponding to a candidate main body in a current interface, and enabling the candidate main body to be a main body recognized for an initial image frame in an image sequence; in response to a selection operation for a candidate item corresponding to a target subject in the current interface, generating a target video corresponding to the target subject for the image sequence, the target subject being any one of the candidate subjects, a target image frame including the target subject in the target video is obtained by cutting the target subject in a to-be-processed image frame, and the to-be-processed image frame is an image frame including the target subject in the image sequence. According to the method, the video can be conveniently generated for the target subject required by the user.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and more specifically, to a video processing method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid advancement of science and technology and living standards, electronic devices (such as smart phones, tablet computers, etc.) have become one of the commonly used electronic products in people's lives. Users can use electronic devices to shoot videos, edit videos, etc. In some scenarios, there may be multiple subjects (such as multiple people, etc.) in the captured video screen, and there may also be multiple subjects in the edited video screen, and the user may only need to focus on the screen content of one of the subjects, which requires the user to edit the image frames in the video to obtain the video corresponding to the required subject, which makes the user operation more cumbersome. Summary of the invention

[0003] The present application proposes a video processing method, device, electronic device and storage medium, which can conveniently generate a video for a target subject according to user needs.

[0004] In a first aspect, an embodiment of the present application provides a video processing method, the method comprising: displaying candidate items corresponding to candidate subjects in a current interface, the candidate subjects being subjects identified for an initial image frame in an image sequence; in response to a selection operation of the candidate items corresponding to a target subject in the current interface, generating a target video corresponding to the target subject for the image sequence, the target subject being any one of the candidate subjects, and a target image frame including the target subject in the target video being obtained after cropping the target subject in an image frame to be processed, and the image frame to be processed being an image frame including the target subject in the image sequence.

[0005] In a second aspect, an embodiment of the present application provides a video processing device, comprising: a candidate item display module and a video generation module, wherein the candidate item display module is used to display candidate items corresponding to candidate subjects in a current interface, and the candidate subject is a subject identified for an initial image frame in an image sequence; the video generation module is used to generate a target video corresponding to the target subject for the image sequence in response to a selection operation of the candidate items corresponding to the target subject in the current interface, and the target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained after cropping the target subject in the image frame to be processed, and the image frame to be processed is an image frame including the target subject in the image sequence.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the video processing method provided in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the video processing method provided in the first aspect above.

[0008] The solution provided by the present application is to display candidate items corresponding to candidate subjects in the current interface, where the candidate subjects are subjects identified for the initial image frame in the image sequence, and in response to the selection operation of the candidate items corresponding to the target subject in the current interface, generate a target video corresponding to the target subject for the image sequence, where the target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained by cropping the target subject in the image frame to be processed, where the image frame to be processed is the image frame including the target subject in the image sequence. Thus, after the user selects the subject in the video image required, the target subject in the image frame in the image sequence can be cropped according to the target subject selected by the user to generate a video corresponding to the target subject, thereby conveniently realizing the generation of a video for the target subject required by the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic flow chart of a video processing method according to an embodiment of the present application is shown.

[0011] Figure 2 A schematic flow chart of a video processing method according to another embodiment of the present application is shown.

[0012] Figure 3 A schematic diagram showing a principle of a video processing method provided in an embodiment of the present application is shown.

[0013] Figure 4 A schematic diagram showing a principle of a video processing method provided in an embodiment of the present application is shown.

[0014] Figure 5A schematic diagram showing a principle of a video processing method provided in an embodiment of the present application is shown.

[0015] Figure 6 A schematic diagram showing a principle of a video processing method provided in an embodiment of the present application is shown.

[0016] Figure 7 A schematic flow chart of a video processing method according to another embodiment of the present application is shown.

[0017] Figure 8 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0018] Fig. 9 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0019] Fig.10 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0020] Fig.11 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0021] Fig.12 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0022] Fig.13 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0023] Fig.14 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0024] Fig.15 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0025] Fig.16 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0026] Fig.17 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0027] Fig.18 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0028] Fig.19 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0029] Fig. 20 Another interface schematic diagram provided in an embodiment of the present application is shown.

[0030] Fig.21A schematic flow chart of a video processing method according to another embodiment of the present application is shown.

[0031] Fig. 22 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0032] Fig.23 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0033] Fig.24 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0034] Fig.25 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0035] Fig.26 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0036] Fig. 27 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0037] Fig.28 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0038] Fig.29 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0039] Fig.30 A schematic flow chart of a video processing method according to yet another embodiment of the present application is shown.

[0040] Fig.31 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0041] Fig.32 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0042] Fig.33 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0043] Fig.34 A schematic diagram of an interface provided in an embodiment of the present application is shown.

[0044] Fig.35 A block diagram of a video processing device according to an embodiment of the present application is shown.

[0045] Fig.36 It is a block diagram of an electronic device according to an embodiment of the present application for executing the video processing method according to an embodiment of the present application.

[0046] Fig.37It is a storage unit of an embodiment of the present application for storing or carrying a program code for implementing a video processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0048] In daily life, there are some video scenes with multiple subjects. For example, in many cases, there are video scenes with multiple characters. For example, in the video recording of a multi-person performance, the picture taken during recording is a panoramic picture, and the obtained video picture includes multiple character subjects. If it is necessary to crop the image frame of one of the character subjects in the video picture as a new image frame in the video later, the user needs to perform tedious editing operations; for example, in a live broadcast, there may be many characters in the live broadcast scene, so that multiple character subjects will be captured within the camera's field of view, but the anchor may only need to shoot a video picture that mainly includes the anchor's picture content. At this time, the anchor can only adjust the shooting position, focal length, etc. to make the live broadcast picture mainly include the anchor.

[0049] In view of the above problems, the inventors have proposed the video processing method, device, electronic device and storage medium provided in the embodiments of the present application, which can realize that after the user selects the subject in the video image required, the target subject in the image frame in the image sequence can be cropped according to the target subject selected by the user to generate a video corresponding to the target subject, thereby conveniently realizing the generation of a video for the target subject required by the user. Among them, the specific video processing method is described in detail in the subsequent embodiments.

[0050] The video processing method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0051] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a video processing method provided by an embodiment of the present application. In a specific embodiment, the video processing method is applied to Fig.35 The video processing device 600 and the electronic device 100 ( Fig.36 ). The following will take an electronic device as an example to illustrate the specific process of this embodiment. Of course, it can be understood that the electronic device used in this embodiment can be a smart phone, a tablet computer, an e-book, a laptop computer, etc., which is not limited here. Figure 1 The process shown is described in detail, and the video processing method may specifically include the following steps:

[0052] Step S110: displaying candidate items corresponding to candidate subjects in the current interface, where the candidate subjects are subjects identified for the initial image frame in the image sequence.

[0053] In an embodiment of the present application, the current interface may be a video shooting interface corresponding to a video shooting application, a video editing interface corresponding to a video editing application, etc., and the specific application scenario may not be limited. For example, the current interface may also be a live broadcast interface corresponding to the anchor end of a live broadcast application. In the case where it is necessary to generate a video corresponding to one of the subjects for the corresponding image sequence, the electronic device may display candidate items corresponding to the candidate subjects in the current interface so that the user can select the required subject. The candidate subject is a subject identified for the initial image frame in the image sequence. The subject may be a person, animal, plant, vehicle, food, etc. The specific type of the subject in the embodiment of the present application may not be limited.

[0054] In some embodiments, the electronic device can obtain the above initial image frame, identify all subjects existing in the initial image frame, and use the identified subjects as candidate subjects; based on the determined candidate subjects, candidate items corresponding to each candidate subject can be generated for each candidate subject in the current interface.

[0055] In a possible implementation, the electronic device may segment the initial image frame using an image segmentation algorithm, and determine the subject in the initial image frame according to the segmentation result.

[0056] In some embodiments, the above image sequence may be a sequence of image frames arranged in a temporal order from early to late. For example, in a video shooting scenario, the above image sequence may be a sequence of video image frames captured during video shooting, that is, the image frames in the image sequence are video frames saved during video shooting under normal circumstances; in a video editing scenario, the above image sequence may be a sequence of image frames in a video to be edited (i.e., the video being edited).

[0057] In a possible implementation, the above initial image frame may be the first image frame in an image sequence. For example, in a video shooting scene, the initial image frame may be the first image frame captured after the video shooting starts, so that after selecting the target subject from the first image frame, the subsequently captured image frames may be processed to crop the target subject in the subsequently captured image frames, thereby obtaining the video frames in the required video.

[0058] In a possible implementation, the above initial image frame may be any image frame in an image sequence. For example, in a video editing scenario, the initial image frame may be any image frame in the video to be edited that includes the target subject required by the user, so that the corresponding image frame may be selected as the initial image frame in the video to be edited, and then after the target subject is selected for the image frame, the target subject in each image frame including the target subject in the video to be edited may be cropped to obtain the video frame in the required video.

[0059] Step S120: In response to the selection operation of the candidate item corresponding to the target subject in the current interface, a target video corresponding to the target subject is generated for the image sequence, the target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained after cropping the target subject in the image frame to be processed, and the image frame to be processed is the image frame including the target subject in the image sequence.

[0060] In an embodiment of the present application, after the electronic device displays the candidate items corresponding to the above candidate subjects in the current interface, it can detect the operation in the current interface. After detecting the selection operation of the candidate items for the target subject in the current interface, it can generate a video corresponding to the above target subject for the above image sequence in response to the selection operation, so that the image frame including the target subject in the generated target video is obtained after cropping the target subject in the image frame to be processed in the image sequence, so that the generated target video can meet the needs of the user.

[0061] In some embodiments, the electronic device may crop the target subject in image frames (image frames to be processed) in which the target subject exists in an image sequence, thereby obtaining the above target image frames; replace these image frames to be processed in the image sequence with the obtained target image frames, and then generate a video corresponding to the target subject based on the image sequence.

[0062] In a possible implementation, when generating a video corresponding to the target subject based on the above image sequence, the video corresponding to the target subject may be generated based on the order of the image frames in the image sequence. It should be noted that since the target subject may not exist in some of the image frames in the above image sequence, these image frames without the target subject may be deleted or retained in the image sequence.

[0063] In one possible implementation, the target subject in the image frame to be processed is cropped by cropping an image region where the target subject is located from the image frame to be processed, and using the cropped image region as a new image frame to reduce image content other than the target subject in the image frame, thereby obtaining the target image frame required by the user.

[0064] In some embodiments, before generating the above target video, or after generating the target video, a preset image processing may be performed on the above target image frame, and the preset image processing is used to improve the image quality of the target image frame, thereby improving the video quality. The preset image processing may include image enhancement, image noise reduction, AI deblurring, AI de-reflection, image super-resolution processing, etc.

[0065] The video processing method provided in the embodiment of the present application allows the user to select the required subject by displaying candidate items corresponding to the subject identified in the image frame in the interface. After the user selects the required subject, the target subject in the image frame in the image sequence can be cropped according to the target subject selected by the user to generate a video corresponding to the target subject, thereby conveniently realizing the generation of a video for the target subject required by the user.

[0066] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of a video processing method provided by another embodiment of the present application. The video processing method is applied to the above electronic device. Figure 2 The process shown is described in detail, and the video processing method may specifically include the following steps:

[0067] Step S210: displaying candidate items corresponding to candidate subjects in the current interface, where the candidate subjects are subjects identified for the initial image frame in the image sequence.

[0068] In the embodiment of the present application, step S210 can refer to the contents of other embodiments and will not be repeated here.

[0069] Step S220: In response to the selection operation of the candidate item corresponding to the target subject in the current interface, each image frame in the image sequence containing the target subject is determined as an image frame to be processed, and the target subject is any one of the candidate subjects.

[0070] In an embodiment of the present application, when the electronic device generates a video corresponding to the target subject for the above image sequence in response to the selection operation of the candidate items for the target subject in the current interface, it can determine each image frame in the image sequence in which the target subject exists as the image frame to be processed, so as to crop each image frame in the image sequence that includes the target subject to obtain the image frame in the video required by the user.

[0071] In some embodiments, the above initial image frame is the first image frame in the image sequence. When the electronic device determines that each image frame in the above image sequence contains the target subject in response to the above selection operation, since the above initial image frame must include the target subject, the above initial image frame can be determined as the image frame to be processed, and each subsequent image frame in the image sequence can be sequentially identified to determine whether the target subject exists in the subsequent image frame, and these subsequent image frames are image frames that are located after the above initial image frame in time sequence; after each identification of whether the above target subject exists in the subsequent image frame, based on the identification result, if it is identified that the above target subject exists in the subsequent image frame, the subsequent image frame can be determined as the image frame to be processed.

[0072] In a possible implementation, when identifying whether the target subject exists in the above subsequent image frames, it may be identified whether the target subject exists in the subsequent image frames based on reference image features corresponding to the target subject.

[0073] In the above implementation, the electronic device can identify all subjects present in the above subsequent image frames; based on the result of matching the image features of each subject in the subsequent image frames with the above reference image features, determine whether the target subject exists in the subsequent image frame. Among them, the above image features include low-level features and high-level features of the subject; low-level features are also called basic features or underlying features of the image, which can be basic information directly extracted from the original data of the image, such as edge features, texture features, shape features, spatial relationship features, etc.; high-level features are abstract information further extracted from low-level features or mid-level features, which are usually closely related to the semantic content of the image and have a higher level of abstraction, such as the semantic label of the subject (object name, category, etc.), scene information, shape and posture of the subject, contextual information, etc.

[0074] In a possible implementation, the electronic device can segment the subsequent image frames through an image panoramic segmentation algorithm model based on artificial intelligence (AI), extract all subjects in the segmentation results, and obtain the image area corresponding to each subject in the subsequent image frame. For example, the subject in the subsequent image frame can be identified and segmented through a generative artificial intelligence (AIGC) model, a pre-trained neural network model for segmenting subjects, etc. It should be noted that this implementation can also be used to identify the existing subject in the initial image frame.

[0075] Among them, the above model can be trained based on a large number of sample images including multiple subjects. These sample images cover different shooting themes, shooting environments (scenes) and shooting subjects. In addition, these sample images can also have different shooting parameters (such as lighting conditions, color system, shooting angle) and other factors, so as to subdivide the training scene of the model and ensure that the sample images cover a wide range, thereby improving the model's recognition ability, generalization ability and anti-interference ability, and learning about complex backgrounds and subject types that are not required to be identified, so that the model can more effectively focus on the subject types that need to be identified, eliminate noise, and thus improve the recognition effect. The above shooting environments (scenes) may include indoor scenes (such as classrooms, stadiums, restaurants, museums, shopping malls, etc.), outdoor scenes (such as park lawns, squares, campus playgrounds, roads, courts, etc.), etc.; the types of subjects may include people (such as adults, children, single people, multiple people, etc.), food (such as Chinese food, Western food, solids, liquids, vegetables, fruits, etc.), plants (such as trees, flowers, grass, etc.), animals (such as dogs, cats, cows, sheep, birds, fish, etc.), buildings (such as high-rise buildings, bungalows, villas, oriental buildings, western buildings, modern buildings, etc.), objects (such as dolls, calligraphy and paintings, tables, chairs, etc.), scenery (such as the sky, the ocean, the river, etc.); shooting parameters include shooting hardware parameters (such as shutter, aperture, sensitivity, depth of field, white balance, focus, shutter, etc.), shooting manual parameters (such as shooting angle, shooting time, brightness, contrast, color temperature, hue, exposure parameters, etc.).

[0076] Optionally, before segmenting the subsequent image frames to determine the subject present therein, the subsequent image frames may be preprocessed. Figure 3 , the electronic device can perform image enhancement, image noise reduction and other pre-processing on the image frame, and then segment the image frame through the AI ​​image panoramic segmentation algorithm model, extract all the subjects in the segmentation result, and obtain the image area corresponding to each subject in the subsequent image frame.

[0077] In a possible implementation, the above reference image features may be image features currently recorded for the target subject. Figure 4, the reference image features can be obtained after feature extraction and feature processing based on the image area corresponding to the target subject cropped from the above initial image frame (i.e., the area where the image content of the target subject is located) and the image area corresponding to the target subject in each subsequent image frame identified. That is to say, when any subsequent image frame is not identified to determine whether there is a target subject, the image area corresponding to the target subject in the initial image frame can be obtained at this time, and then image enhancement, image noise reduction and other data preprocessing are performed on the image area, and image features (such as low-level features, high-level features) are extracted through a deep learning model, and then the extracted image features are subjected to feature processing such as dimensionality reduction (reducing feature dimensions) and normalization, so as to obtain the reference image features of the target subject (i.e., target subject features); and thereafter, for any subsequent image frame, if the target subject is identified in the subsequent image frame, the image features of the target subject in the subsequent image frame can be obtained based on the method of obtaining the reference image features of the above initial image frame, and the reference image features can be updated according to the latest acquired image features, so that the image features of the target subject can be continuously updated and optimized, thereby ensuring the recognition accuracy of the target subject.

[0078] In one possible implementation, see Figure 5 , the electronic device can crop the image area corresponding to each subject (that is, the area where the image content of each subject is located) from the subsequent image frame; then, through the feature extraction model, feature extraction and processing are performed on the image area corresponding to each subject, so as to obtain the image features of each subject; then, the image features of each subject are matched with the above reference image features; if the image features of any subject match the above reference image features, it can be determined that the target subject exists in the subsequent image frame, and the area where the target subject is located in the subsequent image frame can be determined; if the image features of any subject do not match the above reference image features, it can be determined that the target subject does not exist in the subsequent image frame.

[0079] In some embodiments, if the video processing method provided by the embodiment of the present application is applied to a video editing scenario, that is, the above image sequence is a sequence of image frames in the video to be edited, the electronic device may also first determine the above initial image frame as the image frame to be processed for the image sequence; then, for other image frames in the image sequence except the initial image frame, identify in parallel whether the above target subject exists in multiple image frames in other frames, and based on the identification results, determine the image frame with the above target subject as the image frame to be processed. Among them, when identifying whether the above target subject exists in the above image frame, the method of identifying whether the target subject exists in the subsequent image frame in the previous embodiment can be referred to, and no further description is given here.

[0080] Step S230: cropping the target subject in each of the image frames to be processed to obtain a target image frame corresponding to each of the image frames to be processed.

[0081] In an embodiment of the present application, after determining the image frames to be processed that include the target subject in the above image sequence, the target subject in each image frame to be processed can be cropped to obtain the target image frame corresponding to each image frame to be processed.

[0082] In some embodiments, see Figure 6 , the electronic device can, for the above image frame to be processed, crop the image region where the target subject is located according to the region of the target subject in the image frame to be processed, as the target image frame corresponding to the image frame to be processed. The electronic device can determine the cropping region according to the image region where the target subject is located, and then crop the above image region based on the cropping region, and the above target subject is at least completely contained in the cropping region.

[0083] In a possible implementation, after cropping the above image area, the electronic device may also perform image processing and optimization such as image enhancement, image noise reduction, AI deblurring, AI de-reflection, and image super-resolution processing on the cropped image area, and use the processed image area as the above target image frame.

[0084] In some embodiments, the electronic device crops the target subject in each image frame to be processed, which may be after all the image frames to be processed in the image sequence are determined, and then crops the target subject in each image frame to be processed; or it may be that each time a frame to be processed is determined, the target subject in the determined image frame to be processed is cropped. Of course, the electronic device may also determine N frames to be processed, and then crop the target subject in each of the determined N frames to be processed, where N is a positive integer.

[0085] In a possible implementation, considering that the electronic device determines N frames of image frames to be processed and then crops the target subject in each of the N frames of image frames to be processed, these N frames of image frames to be processed will continue to occupy memory when being cropped. Therefore, the electronic device can determine a target number of frames that matches the memory occupancy state based on the memory occupancy state, and the target number of frames can be negatively correlated with the memory occupancy rate; the electronic device can determine a target number of image frames to be processed and then crop the target subject in each of the target number of image frames to be processed.

[0086] In a possible implementation, the electronic device may also determine the cropping size set by the user based on the operation for setting the size of the cropping area detected. When the electronic device crops the above image to be processed, it may crop the image area where the target subject of the cropping size is located based on the cropping size and the position of the target subject in the image to be processed, thereby obtaining a target image frame corresponding to the image frame to be processed.

[0087] Step S240: Based on the target image frame corresponding to each of the image frames to be processed, a target video corresponding to the target subject is generated for the image sequence.

[0088] In the embodiment of the present application, after obtaining the target image frame corresponding to each frame of the image frame to be processed, the electronic device can generate a target video corresponding to the target subject for the above image sequence based on the target image frame corresponding to each frame of the image frame to be processed. Among them, each frame of the image frame to be processed in the image sequence can be replaced with its corresponding target image frame, and then a video corresponding to the target subject is generated based on the image sequence.

[0089] The video processing method provided in the embodiment of the present application allows the user to select the required subject by displaying candidate items corresponding to the subject identified in the image frame in the interface. After the user selects the required subject, the image frame to be processed in which the target subject exists in the image sequence can be identified according to the target subject selected by the user, and the target subject in the image frame to be processed can be cropped to obtain the required target video frame, so that the image frame only includes the image area where the target subject is located, so that the generated video can meet the user's needs, reduce user operations, and improve the user experience.

[0090] See also Figure 7 , Figure 7 FIG. 1 is a flow chart of a video processing method provided by another embodiment of the present application. The video processing method is applied to the above electronic device. Figure 7 The process shown is described in detail, and the video processing method may specifically include the following steps:

[0091] Step S310: displaying candidate items corresponding to candidate subjects in the current interface, wherein the candidate subjects are subjects identified for the initial image frames in the image sequence, and the image sequence is a sequence consisting of image frames of the video to be edited.

[0092] In the embodiment of the present application, the video processing method provided by the embodiment of the present application can be applied to a video editing scenario, in which the above image sequence can be a sequence of image frames in a video to be edited (i.e., the video being edited). Optionally, the above image sequence can be a sequence of image frames in the video to be edited arranged in a chronological order from early to late.

[0093] In some embodiments, when the video processing method provided in the embodiment of the present application is used in a video editing scenario, the above current interface may be a video editing interface. Exemplarily, the electronic device may display a video editing interface corresponding to the video when an editing operation for the video to be edited is detected. In the video editing interface, a preview screen for the video to be edited may be displayed; the electronic device may also control the display of image frames corresponding to different playback progress in the preview screen in response to the progress adjustment operation in the video editing interface, so that the user can edit the video in different time periods and the image frames at different progress.

[0094] Step S320: In response to the selection operation, the initial image frame is determined as the image frame to be processed, and for each subsequent image frame in the image sequence, it is identified whether the target subject exists in the subsequent image frame.

[0095] The target subject is any one of the candidate subjects, and the subsequent image frame is located after the initial image frame in terms of time sequence.

[0096] In the embodiment of the present application, the initial image frame is the first image frame in the image sequence, that is, the image frame that is located at the first frame in the time sequence in the video to be edited. After the electronic device detects the above selection operation, since the above initial image frame includes the target subject, the above initial image frame can be determined as the image frame to be processed, and each subsequent image frame that is located after the above initial image frame in the time sequence in the image sequence can be sequentially identified to determine whether the target subject exists in the subsequent image frame.

[0097] If yes, execute step S330: determine the subsequent image frame as the image frame to be processed.

[0098] In an embodiment of the present application, for the image sequence corresponding to the video to be edited, after each identification of whether the above target subject exists in a subsequent image frame, based on the identification result, if the above target subject is identified in the subsequent image frame of the frame, the subsequent image frame of the frame can be determined as the image frame to be processed.

[0099] If so, execute step S340: display a first prompt message in the current interface, where the first prompt message is used to prompt whether to retain the subsequent image frame.

[0100] In an embodiment of the present application, for the image sequence corresponding to the video to be edited, each time after identifying whether the target subject exists in the subsequent image frame, based on the identification result, if it is identified that the target subject does not exist in the subsequent image frame, a prompt message can be output to prompt the user whether to keep the subsequent image frame. It can be understood that since the target subject does not exist in the subsequent image frame, the image content required by the user may not exist in the subsequent image frame, so the electronic device can output the first prompt message so that the user can choose whether to keep the subsequent image frame according to the needs.

[0101] In some embodiments, the electronic device may display the above first prompt information in the form of a pop-up window prompt. Optionally, the electronic device displays the above first prompt information to prompt the user whether to retain the text content of the subsequent image frame of the frame, and may also display a first control for determining whether to retain the subsequent image frame of the frame, and a second control for determining whether to retain the subsequent image frame of the frame, so that the user can operate the first control according to the prompt text content and needs, and then choose to retain the subsequent image frame of the frame, or operate the second control to choose not to retain the subsequent image frame of the frame.

[0102] Step S350: If a first confirmation operation for the first prompt information is detected, the subsequent image frames in the video to be edited are deleted.

[0103] In an embodiment of the present application, after the above first prompt information is displayed, if a first confirmation operation for the above first prompt information is detected, the first confirmation operation is used to indicate that the subsequent image frames of the frame should not be retained, and the electronic device can delete the subsequent image frames of the frame from the above video to be edited in response to the first confirmation operation.

[0104] In some embodiments, when the electronic device displays the above first prompt information, when the above first control and the second control are displayed, the electronic device detects a trigger operation for the above second control, for example, detects a click operation on the second control, then it can be determined that the above first confirmation operation is detected, and in response to the first confirmation operation, the subsequent image frames of the frame in the video to be edited are deleted.

[0105] Step S360: If a second confirmation operation for the first prompt information is detected, retaining the subsequent image frames in the video to be edited.

[0106] In an embodiment of the present application, after the above first prompt information is displayed, if a second confirmation operation for the above first prompt information is detected, the second confirmation operation is used to indicate that the subsequent image frames of the frame are retained. The electronic device can respond to the second confirmation operation and retain the subsequent image frames in the above video to be edited, which can also be understood as not deleting the subsequent image frames of the above video to be edited.

[0107] In some embodiments, when the electronic device displays the above first prompt information, when the above first control and the second control are displayed, the electronic device detects a trigger operation for the above first control, for example, detects a click operation on the first control, then it can be determined that the above second confirmation operation is detected, and in response to the second confirmation operation, retains the subsequent image frames of the frame in the video to be edited.

[0108] Step S370: cropping the target subject in each of the image frames to be processed to obtain a target image frame corresponding to each of the image frames to be processed.

[0109] In the embodiment of the present application, step S370 can refer to the content of the aforementioned embodiment and will not be repeated here.

[0110] Step S380: Replace each of the to-be-processed image frames in the to-be-edited video with the target image frame corresponding to each of the to-be-processed image frames, to obtain a target video corresponding to the target subject.

[0111] In an embodiment of the present application, after the target subject is cropped to obtain the target image frame for each frame to be processed in the image sequence corresponding to the video to be edited, each frame to be processed in the video to be edited can be replaced by the target image frame corresponding to each frame to be processed, thereby obtaining the target video corresponding to the target subject.

[0112] In some embodiments, the electronic device can replace the to-be-processed image frame in the video to be edited with its corresponding target image frame after each to-be-processed image frame is determined and the target subject in the to-be-processed image frame is cropped to obtain the target image frame corresponding to the to-be-processed image frame. After all the to-be-processed image frames are cropped and the to-be-processed image frames in the video to be edited are replaced, the processing of the entire video to be edited is completed, so that all the image frames including the target subject in the video to be processed are replaced with the target image frames obtained after cropping the target subject therein, thereby obtaining the target video required by the user.

[0113] In some embodiments, the electronic device may also crop the target subject in all the image frames to be processed to obtain the target image frame corresponding to each image frame to be processed, and then replace each image frame to be processed in the video to be edited with its corresponding target image frame in turn, thereby completing the processing of the entire video to be edited, so that all image frames including the target subject in the video to be processed are replaced with the target image frames obtained after cropping the target subject therein, thereby obtaining the target video required by the user.

[0114] In some embodiments, if any image frame in the video to be edited does not contain a target subject, the image frame can be deleted. If the audio frame corresponding to the image frame is deleted synchronously, it may cause the video and audio playback in the edited video to be not smooth enough, and the audio and video to be out of sync. Therefore, if there are deleted image frames in the video to be edited, the audio frames corresponding to the deleted image frames can be retained, and the image frames and audio frames in the video to be edited can be optimized and matched, or the audio playback speed can be accelerated when playing the target video, or the image frame playback speed can be slowed down, so that the duration of the audio and video are equal.

[0115] The video processing method provided in the embodiment of the present application is introduced below by giving examples.

[0116] For example, if the subject is a person, please refer to Figure 8 When the electronic device displays the video editing interface A1, for the video to be edited, a candidate frame A2 corresponding to the human subject identified in the initial image frame can be displayed in the video editing interface A1, and prompt information can also be displayed in the video editing interface A1 to prompt the user to select the corresponding target human subject; please refer to Fig. 9 After the electronic device detects the selection operation of the candidate frame A1 corresponding to the target person subject in the video editing interface A1, it can output prompt information to prompt that the target person subject has been selected.

[0117] After selecting the target person, you can start tracking video editing for the target person. For details, please refer to Fig.10 and Fig.11 For the above initial image frame, since the above target person subject has been identified, the target video image frame corresponding to the target person subject can be cropped, and the initial image frame in the video to be edited is replaced with the cropped target video image frame and displayed in the video frame list A3 below.

[0118] See also Fig.12 and Fig.13For each subsequent image frame after the initial image frame, after identifying all existing human subjects, combined with the image features of the target human subject, when it is identified that there is a target human subject in the subsequent image frame, the target video image frame corresponding to the target human subject can be cropped, and the subsequent image frame in the video to be edited can be replaced with the cropped target video image frame and displayed in the video frame list A3.

[0119] See also Fig.14 When it is identified that there is no target person subject in any subsequent image frame, a prompt message can be displayed in the video editing interface A1 to prompt whether to keep the frame because there is no target person subject in the subsequent image frame. Fig.15 , Fig.16 and Fig.17 In the case of detecting that the user inputs an operation of selecting to retain the subsequent image frame, the subsequent image frame in the video to be edited can be retained, and the subsequent image frame is directly retained in the video frame list A3, and the processing of the next subsequent video frame continues. Fig.18 and Fig.19 In the case of detecting that the user inputs an operation of not retaining the subsequent image frame, the subsequent image frame in the video to be edited can be deleted, and the subsequent image frame will not be retained in the video frame list A3, and the processing of the next subsequent video frame will continue. Fig. 20 After completing the processing of each subsequent video frame, the processing of the entire video to be edited is realized, thereby obtaining the video corresponding to the target person subject.

[0120] The video processing method provided by the embodiment of the present application allows the user to select the required subject by displaying candidate items corresponding to the subject identified in the initial image frame of the video to be edited in the interface in the video editing scene. After the user selects the required subject, the image frame to be processed in the image sequence where the target subject exists can be identified according to the target subject selected by the user, and the target subject in the image frame to be processed can be cropped to obtain the required target video frame, so that the image frame only includes the image area where the target subject is located, thereby meeting the user's need to edit the video to be edited to obtain the video of the target subject, and can reduce user operations and improve the user's usage experience; in addition, for the image frame in the video to be edited where the target subject does not exist, prompt information is output so that the user can choose whether to retain the subsequent image frame according to the demand, thereby better meeting the user's different editing needs for the video.

[0121] See also Fig.21 , Fig.21FIG. 1 is a flow chart of a video processing method provided by another embodiment of the present application. The video processing method is applied to the above electronic device. Fig.21 The process shown is described in detail, and the video processing method may specifically include the following steps:

[0122] Step S401: displaying candidate items corresponding to candidate subjects in the current interface, wherein the candidate subjects are subjects identified for an initial image frame in an image sequence, and the image sequence is a sequence of video image frames captured during the video shooting process.

[0123] In an embodiment of the present application, the video processing method provided in the embodiment of the present application can be applied to a video shooting scene, in which the above image sequence can be a sequence of video image frames captured during the video shooting process. Optionally, the above image sequence can be a sequence of image frames captured arranged in a chronological order from early to late.

[0124] In some embodiments, when the video processing method provided in the embodiment of the present application is used in a video shooting scene, the above current interface may be a video shooting interface. For example, the electronic device may display a video editing interface corresponding to the video when a selection operation of a video mode in the shooting interface of the shooting application is detected. In the video shooting interface, a preview screen may be displayed to display the image captured by the camera in the preview screen.

[0125] In a possible implementation, when the electronic device displays the above video shooting interface, it can detect the operation in the video shooting interface. When the shooting operation in the video shooting interface is detected, the first image frame obtained by shooting can be used as the above initial image frame, and candidate items corresponding to the candidate subject can be displayed in the video shooting interface for the initial image frame.

[0126] Step S402: In response to a selection operation of a candidate item corresponding to a target subject in the current interface, the initial image frame is determined as an image frame to be processed, and the target subject is any one of the candidate subjects.

[0127] In an embodiment of the present application, after detecting the above selection operation, the electronic device can perform video shooting. Since the above initial image frame includes the target subject, the above initial image frame can be determined as the image frame to be processed.

[0128] Step S403: sequentially identifying, for each subsequent image frame in the image sequence, whether the target subject exists in the subsequent image frame, wherein the subsequent image frame is located after the initial image frame in terms of time sequence.

[0129] In an embodiment of the present application, for subsequent image frames captured after the above initial image frame, the electronic device can identify whether a target subject exists in each subsequent image frame in turn.

[0130] If yes, execute step S404: determine the subsequent image frame as the image frame to be processed.

[0131] In an embodiment of the present application, each time the subsequent image frame obtained by taking the image is identified to determine whether the above target subject exists, based on the recognition result, if the above target subject is identified to exist in the subsequent image frame of the frame, the subsequent image frame can be determined as the image frame to be processed.

[0132] If not, executing step S405: displaying a second prompt message in the current interface, wherein the second prompt message is used to prompt the user to adjust the shooting angle of the video.

[0133] In the embodiment of the present application, after each identification of whether the above target subject exists in the subsequent image frame obtained by shooting, based on the identification result, if it is identified that the above target subject does not exist in the subsequent image frame of the frame, a prompt message can be output to prompt the user to adjust the shooting angle of the video shooting. It can be understood that since the above target subject does not exist in the subsequent image frame of the frame, the content of the target subject required by the user may not exist in the subsequent image frame of the frame, so the electronic device can output the above second prompt information, so that after the user adjusts the shooting angle according to the second prompt information, the target subject can be located in the field of view of the camera, so that the captured image frame can include the target subject.

[0134] Step S406: Pause the video shooting.

[0135] In an embodiment of the present application, when it is currently recognized that the target subject does not exist in the subsequent image frames, it means that the above target subject does not exist within the field of view of the camera. Therefore, the electronic device can also pause video shooting to avoid the presence of image frames in the captured video that do not include the target subject.

[0136] Step S407: Identify whether the target subject exists in the preview image frame currently captured by the camera.

[0137] In an embodiment of the present application, after displaying the above second prompt information and pausing video shooting, the electronic device can identify whether the target subject exists in the preview image frame currently captured by the camera, so that the video shooting can continue after the user adjusts the shooting angle so that the target subject is back in the field of view of the camera.

[0138] In some embodiments, the electronic device can identify all subjects present in the preview image frame and obtain image features for each subject; then match the image features of each subject with the reference image features of the above target subject; if the image features of any subject match the reference image features of the above target subject, it can be determined that the target subject exists in the preview image frame; if the image features of each subject do not match the reference image features of the above target subject, it can be determined that the target subject does not exist in the preview image frame.

[0139] If yes, execute step S408: continue the video shooting, and return to step S403.

[0140] In the embodiment of the present application, after identifying whether the above target subject exists in the preview image frame currently captured by the camera, based on the recognition result, if the above target subject exists in the above preview image frame, the video shooting can be continued, and the above step S430 is returned to continue processing the captured image frame. If the above target subject does not exist in the above preview image frame, the above step S407 can be returned to continue identifying whether the above target subject exists in the newly captured preview image frame.

[0141] Step S409: cropping the target subject in each of the image frames to be processed to obtain a target image frame corresponding to each of the image frames to be processed.

[0142] In the embodiment of the present application, step S409 can refer to the content of the aforementioned embodiment and will not be repeated here.

[0143] Step S410: Based on the target image frame corresponding to each of the image frames to be processed, a target video corresponding to the target subject is generated for the image sequence.

[0144] In an embodiment of the present application, when the electronic device detects the end of shooting operation, it can generate a target video corresponding to the target subject based on the target image frame corresponding to the image frame to be processed in the current image sequence in response to the end of shooting operation. It can be understood that a video is a continuous dynamic image composed of a series of image frames (i.e., static pictures) and audio in a time sequence. Therefore, when generating the above target video, the target image frame corresponding to the image frame to be processed in the above image sequence and the recorded audio data can be encoded into a video file to obtain the above target video.

[0145] The video processing method provided in the embodiment of the present application is introduced below by giving examples.

[0146] For example, if the subject is a person, please refer to Fig. 22When the electronic device displays the video shooting interface A4, the candidate frame A2 corresponding to the human subject identified in the initial image frame obtained by shooting can be displayed in the video shooting interface A4, and prompt information can also be displayed in the video shooting interface A4 to prompt the user to select the corresponding target human subject; please refer to Fig.23 After the electronic device detects the selection operation of the candidate frame A4 corresponding to the target person subject in the video shooting interface A4, it can output prompt information to prompt that the target person subject has been selected.

[0147] After selecting the target person, you can start tracking and shooting the target person. For details, please refer to Fig.24 , Fig.25 and Fig.26 For the above initial image frame, since the above target person subject has been identified, the target video image frame corresponding to the target person subject can be cropped; for each subsequent image frame after the initial image frame, after all existing person subjects are identified, combined with the image features of the target person subject, when it is identified that the target person subject exists in the subsequent image frame, a preview frame A5 can be displayed for the identified target person subject in the preview area displayed in the video shooting interface A4, and the target video image frame corresponding to the target person subject can be cropped, and the image content in the preview frame A5 is the image area that needs to be cropped.

[0148] See also Fig. 27 If the target person subject is not found in any subsequent image frame, a prompt message may be displayed in the video shooting interface A4 to prompt that the target person subject does not exist in the subsequent image frame and the shooting angle needs to be adjusted. Fig.28 , when the target person is re-identified in the image frame captured by the camera, the video can be shot and the captured image frames can be processed. Fig.29 After detecting the operation on the shooting control A6, the video shooting can be stopped, and a video of the target subject can be generated based on the target video image frame corresponding to the recorded target person subject.

[0149] The video processing method provided in the embodiment of the present application allows a user to select a desired subject by displaying candidate items corresponding to the subject identified in the initial image frame captured in the interface in the video shooting scene. After the user selects the desired subject, the method can track and shoot the target subject selected by the user, identify the target subject in subsequent image frames, and crop the target subject in subsequent image frames. Finally, a video of the target subject is generated based on the cropped image frames, thereby satisfying the user's shooting needs for shooting a video of the target subject, reducing user operations, and improving the user experience.

[0150] See also Fig.30 , Fig.30 FIG. 1 is a flow chart of a video processing method provided by another embodiment of the present application. The video processing method is applied to the above electronic device. Fig.30 The process shown is described in detail, and the video processing method may specifically include the following steps:

[0151] Step S510: In response to the first trigger operation in the current interface, the candidate subjects existing in the initial image frame of the image sequence are identified, and the area where each candidate subject existing in the initial image frame is located is obtained as the subject identification result.

[0152] In an embodiment of the present application, the electronic device can detect a first trigger operation in the current interface, and in response to the first trigger operation, identify the candidate subjects existing in the initial image frame of the image sequence, obtain the area where each candidate subject existing in the initial image frame is located, and use it as the subject recognition result. Among them, the above first trigger operation can be an operation for triggering the display of candidate items corresponding to the subject in the above initial image frame. The way in which the electronic device identifies the candidate subjects existing in the initial image frame and the area where they are located can refer to the contents of the aforementioned embodiment, which will not be repeated here.

[0153] In some embodiments, the current interface may include a third control for triggering subject tracking. The electronic device detects the operation in the current interface, and when a trigger operation for the third control is detected, such as a click operation on the third control, it may be determined that the first trigger operation is detected, and in response to the trigger operation, the candidate subject present in the initial image frame of the image sequence is identified.

[0154] For example, see Fig.31 In the video shooting scene, the electronic device displays a video shooting interface A4, which includes a switch control A7 for the subject tracking shooting mode. If a click operation on the switch control A7 is detected, the subject tracking shooting mode can be controlled to be enabled, and the candidate subject existing in the captured initial image frame can be identified. In addition, please refer to Fig.32 After the subject tracking shooting mode is enabled, the display state of the above switch control A7 can be changed, for example, the display grayscale, brightness, etc. can be changed.

[0155] For example, see Fig.33In the video editing scene, the electronic device displays a video editing interface A1, which includes a switch control A8 for the subject tracking editing mode. If a click operation on the switch control A8 is detected, the subject tracking editing mode can be controlled to be enabled to identify the candidate subjects in the initial image frame of the video to be edited. In addition, please refer to Fig.34 After the subject tracking editing mode is enabled, the display state of the above switch control A8 can be changed, such as changing the display grayscale, brightness, etc.

[0156] Step S520: Based on the subject recognition result, candidate items corresponding to each candidate subject are displayed in the current interface.

[0157] In an embodiment of the present application, after obtaining the above subject recognition results, that is, obtaining the area where each candidate subject existing in the initial image frame is located, the electronic device can display candidate items corresponding to each candidate subject in the current interface based on the subject recognition results.

[0158] In some implementations, the electronic device displays the initial image frame in the current interface, and the electronic device may display a candidate frame corresponding to the area where each candidate subject is located in the displayed initial image frame based on the area where each candidate subject is located. Figure 8 In a video editing scenario, the electronic device may display a candidate frame A2 corresponding to a human subject identified in an initial image frame in a video editing interface A1.

[0159] Step S530: In response to the selection operation of the candidate item corresponding to the target subject in the current interface, a target video corresponding to the target subject is generated for the image sequence, the target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained after cropping the target subject in the image frame to be processed, and the image frame to be processed is the image frame including the target subject in the image sequence.

[0160] In the embodiment of the present application, step S530 can refer to the contents of the aforementioned embodiment and will not be described again here.

[0161] The video processing method provided in the embodiment of the present application identifies the candidate subjects in the initial image frame according to the detected trigger operation, obtains the area where each candidate subject is located, and then displays the candidate items corresponding to each candidate subject in the interface based on the area where each candidate subject is located, so that the user can select the required subject. After the user selects the required subject, the target subject in the image frame in the image sequence can be cropped according to the target subject selected by the user to generate a video corresponding to the target subject, thereby conveniently realizing the generation of a video for the target subject required by the user.

[0162] See also Fig.35 , which shows a structural block diagram of a video processing device 600 provided in an embodiment of the present application. The video processing device 600 applies the above-mentioned electronic device, and the video processing device 600 includes: a candidate item display module 610 and a video generation module 620. Among them, the candidate item display module 610 is used to display candidate items corresponding to candidate subjects in the current interface, and the candidate subject is a subject identified for the initial image frame in the image sequence; the video generation module 620 is used to respond to the selection operation of the candidate item corresponding to the target subject in the current interface, and generate a target video corresponding to the target subject for the image sequence. The target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained after cropping the target subject in the image frame to be processed, and the image frame to be processed is an image frame including the target subject in the image sequence.

[0163] In some embodiments, the video generation module 620 can be specifically used to: in response to the selection operation of the candidate item corresponding to the target subject in the current interface, determine each image frame in the image sequence that contains the target subject as an image frame to be processed; crop the target subject in each image frame to be processed to obtain a target image frame corresponding to each image frame to be processed; and generate a target video corresponding to the target subject for the image sequence based on the target image frame corresponding to each image frame to be processed.

[0164] In a possible implementation, the video generation module 620 can also be specifically used to: in response to the selection operation, determine the initial image frame as the image frame to be processed, and identify whether the target subject exists in each subsequent image frame in the image sequence, and the subsequent image frame is located after the initial image frame in time sequence; if the target subject is identified in the subsequent image frame, determine the subsequent image frame as the image frame to be processed.

[0165] Optionally, the image sequence is a sequence of image frames of the video to be edited, and the video processing device 600 may also include a first prompt module, an image deletion module and an image retention module. The first prompt module is used to, after identifying whether the target subject exists in the subsequent image frames, display a first prompt message in the current interface if it is identified that the target subject does not exist in the subsequent image frames, and the first prompt message is used to prompt whether to retain the subsequent image frames; the image deletion module is used to delete the subsequent image frames in the video to be edited if a first confirmation operation for the first prompt message is detected; and the image retention module is used to retain the subsequent image frames in the video to be edited if a second confirmation operation for the first prompt message is detected.

[0166] Optionally, the video generation module 620 may be specifically configured to replace each of the to-be-processed image frames in the video to be edited with the target image frame corresponding to each of the to-be-processed image frames, to obtain the target video.

[0167] In a possible implementation, the image sequence is a sequence of video image frames captured during the video shooting process, and the video processing device 600 may further include a second prompt module. The second prompt module is used to display a second prompt message in the current interface after identifying whether the target subject exists in the subsequent image frame, if it is identified that the target subject does not exist in the subsequent image frame, and the second prompt message is used to prompt the user to adjust the shooting angle of the video shooting.

[0168] Optionally, the video processing device 600 may also include a shooting pause module, a subject recognition module and a video shooting module. The shooting pause module is used to pause the video shooting if it is recognized that the target subject does not exist in the subsequent image frame; the subject recognition module is used to identify whether the target subject exists in the preview image frame currently captured by the camera; the video shooting module is used to continue the video shooting if the target subject exists in the preview image frame, and continue to execute the above video generation module 620 in turn for each subsequent image frame in the image sequence, to identify whether the target subject exists in the subsequent image frame.

[0169] Optionally, the video generation module 620 may be specifically configured to: in response to a shooting end operation, generate a target video corresponding to the target subject based on a target image frame corresponding to the image frame to be processed in the current image sequence.

[0170] In a possible implementation, the video generation module 620 may also be used to: identify whether the target subject exists in the subsequent image frame based on reference image features corresponding to the target subject.

[0171] Optionally, the video generation module 620 may also be configured to update the reference image features based on image features of the target subject in the subsequent image frame if it is recognized that the target subject exists in the subsequent image frame.

[0172] In some implementations, the video processing device 600 may further include an image processing module. The image processing module is used to perform a preset image processing on the target image frame, and the preset image processing is used to improve the image quality of the target image frame.

[0173] In some embodiments, the candidate item display module 610 can be specifically used to: in response to a first trigger operation in the current interface, identify the candidate subjects existing in the initial image frame, and obtain the area where each candidate subject existing in the initial image frame is located as a subject recognition result; based on the subject recognition result, display the candidate items corresponding to each candidate subject in the current interface.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0175] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0176] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0177] In summary, the solution provided by the present application is to display candidate items corresponding to candidate subjects in the current interface, where the candidate subjects are subjects identified for the initial image frame in the image sequence, and in response to the selection operation of the candidate items corresponding to the target subject in the current interface, generate a target video corresponding to the target subject for the image sequence, where the target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained by cropping the target subject in the image frame to be processed, and the image frame to be processed is the image frame including the target subject in the image sequence. Thus, after the user selects the subject in the video image required, the target subject in the image frame in the image sequence can be cropped according to the target subject selected by the user to generate a video corresponding to the target subject, thereby enabling the generation of a video for the target subject required by the user to be achieved conveniently.

[0178] Please refer to Fig.36 , which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 may be an electronic device capable of running applications, such as a smart phone, a tablet computer, an e-book, a laptop computer, etc. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more applications are configured to execute the method described in the aforementioned method embodiment.

[0179] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0180] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0181] Please refer to Fig.37 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0182] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video processing method, characterized in that: The method comprises: Displaying candidate items corresponding to candidate subjects in the current interface, wherein the candidate subjects are subjects identified for the initial image frame in the image sequence; In response to the selection operation of the candidate item corresponding to the target subject in the current interface, a target video corresponding to the target subject is generated for the image sequence, the target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained after cropping the target subject in the image frame to be processed, and the image frame to be processed is the image frame including the target subject in the image sequence.

2. The method according to claim 1, characterized in that In response to the selection operation of the candidate item corresponding to the target subject in the current interface, generating a target video corresponding to the target subject for the image sequence includes: In response to a selection operation of a candidate item corresponding to a target subject in the current interface, determining each image frame in the image sequence in which the target subject exists as an image frame to be processed; Cropping the target subject in each of the image frames to be processed to obtain a target image frame corresponding to each of the image frames to be processed; Based on the target image frame corresponding to each of the image frames to be processed, a target video corresponding to the target subject is generated for the image sequence.

3. The method according to claim 2, characterized in that In response to the selection operation of the candidate item corresponding to the target subject in the current interface, determining each image frame in the image sequence in which the target subject exists as an image frame to be processed, includes: In response to the selection operation, the initial image frame is determined as the image frame to be processed, and each subsequent image frame in the image sequence is sequentially identified to determine whether the target subject exists in the subsequent image frame, wherein the subsequent image frame is located after the initial image frame in terms of time sequence; If it is recognized that the target subject exists in the subsequent image frame, the subsequent image frame is determined as the image frame to be processed.

4. The method according to claim 3, characterized in that The image sequence is a sequence of image frames of the video to be edited. After identifying whether the target subject exists in the subsequent image frames, the method further includes: If it is recognized that the target subject does not exist in the subsequent image frame, displaying first prompt information in the current interface, wherein the first prompt information is used to prompt whether to retain the subsequent image frame; If a first confirmation operation for the first prompt information is detected, deleting the subsequent image frames in the video to be edited; If a second confirmation operation for the first prompt information is detected, the subsequent image frames in the video to be edited are retained.

5. The method according to claim 4, characterized in that The generating a target video corresponding to the target subject for the image sequence based on the target image frame corresponding to each of the image frames to be processed includes: Each of the to-be-processed image frames in the to-be-edited video is replaced with the target image frame corresponding to each of the to-be-processed image frames to obtain the target video.

6. The method according to claim 3, characterized in that The image sequence is a sequence of video image frames captured during the video shooting process. After identifying whether the target subject exists in the subsequent image frames, the method further includes: If it is recognized that the target subject does not exist in the subsequent image frame, a second prompt information is displayed in the current interface, and the second prompt information is used to prompt to adjust the shooting angle of the video shooting.

7. The method according to claim 6, characterized in that If it is recognized that the target subject does not exist in the subsequent image frame, the method further includes: Pause the video shooting; Identify whether the target subject exists in the preview image frame currently captured by the camera; If the target subject exists in the preview image frame, the video shooting is continued, and the process returns to the step of sequentially identifying whether the target subject exists in each subsequent image frame in the image sequence.

8. The method according to claim 6, characterized in that The generating a target video corresponding to the target subject for the image sequence based on the target image frame corresponding to each of the image frames to be processed includes: In response to the shooting end operation, a target video corresponding to the target subject is generated based on the target image frame corresponding to the image frame to be processed in the current image sequence.

9. The method according to claim 3, characterized in that: The identifying whether the target subject exists in the subsequent image frame comprises: Based on the reference image features corresponding to the target subject, it is identified whether the target subject exists in the subsequent image frame.

10. The method according to claim 9, characterized in that The method further comprises: If it is recognized that the target subject exists in the subsequent image frame, the reference image feature is updated based on the image feature of the target subject in the subsequent image frame.

11. The method according to any one of claims 1 to 10, characterized in that: The method further comprises: The target image frame is subjected to a preset image processing, wherein the preset image processing is used to improve the image quality of the target image frame.

12. The method according to any one of claims 1 to 10, characterized in that: The candidate items corresponding to the candidate subject are displayed in the current interface, including: In response to a first trigger operation in the current interface, identifying candidate subjects in the initial image frame, and obtaining a region where each candidate subject in the initial image frame is located as a subject identification result; Based on the subject recognition result, candidate items corresponding to each candidate subject are displayed in the current interface.

13. A video processing device, characterized in that: The device comprises: a candidate item display module and a video generation module, wherein: The candidate item display module is used to display candidate items corresponding to candidate subjects in the current interface, where the candidate subjects are subjects identified for the initial image frame in the image sequence; The video generation module is used to respond to the selection operation of the candidate items corresponding to the target subject in the current interface, and generate a target video corresponding to the target subject for the image sequence, wherein the target subject is any one of the candidate subjects, and the target image frame including the target subject in the target video is obtained after cropping the target subject in the image frame to be processed, and the image frame to be processed is the image frame including the target subject in the image sequence.

14. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the method according to any one of claims 1 to 12.

Images