Video processing method and apparatus, computer device, and storage medium

Abstract

The present application provides a video processing method and apparatus, a computer device, and a storage medium. The method comprises: acquiring a video to be processed; determining initial focus information of said video on the basis of a preset focusing mode, wherein the initial focus information comprises a focus area in each video frame; in response to an editing operation for the focus area of a video frame, correcting the focus area of the video frame; on the basis of the corrected focus area, updating the focus areas of the rest video frames in said video to obtain target focus information; and performing focusing processing on said video on the basis of the target focus information. According to embodiments of the present application, automatic focusing processing is performed on a video, and then a focus area editing instruction sent by a user is received, so as to determine target focusing information; thus, quick focusing processing of videos to be processed can be achieved, and focusing processing results can meet user requirements.

Description

Video processing method, device, computer equipment and storage medium

[0001] This application claims priority to Chinese patent application No. 202310975336.9 filed on August 3, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby cited in their entirety as part of this application. Technical Field

[0002] The present disclosure relates to a video processing method, apparatus, computer equipment, and storage medium. Background Art

[0003] When shooting a video, focus processing is required to improve the presentation of specific subjects in the video. Generally, video shooting requires the use of devices with high processing performance, such as cameras or camcorders, to achieve better focus effects. However, in most cases, when shooting videos, devices with poor focus performance, such as mobile phones, can only be used, resulting in poor focus effects in the videos.

[0004] Summary of the Invention

[0005] The embodiments of the present disclosure at least provide a video processing method, apparatus, computer equipment, and storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides a video processing method, including:

[0007] Get the video to be processed;

[0008] Based on a preset focusing method, determining initial focusing information of the video to be processed; wherein the initial focusing information includes a focus area in each video frame;

[0009] In response to an editing operation on the focus area of ​​the video frame, revising the focus area of ​​the video frame;

[0010] Based on the revised focus area, updating the focus areas of the remaining video frames in the to-be-processed video to obtain target focus information;

[0011] Focus processing is performed on the video to be processed based on the target focus information.

[0012] In a possible implementation, determining the initial focus information of the video to be processed based on a preset focus mode includes:

[0013] Determining a video frame to be detected in the video to be processed, and determining a focus subject in the video frame to be detected based on the preset focusing method;

[0014] Tracking the position information of the focused subject in the video to be processed, resetting the video frame where tracking is interrupted as the video frame to be detected, and returning to the step of determining the focused subject;

[0015] Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.

[0016] In a possible implementation, determining the focused subject in the to-be-detected video frame based on the preset focusing mode includes:

[0017] Detecting candidate subjects contained in the to-be-detected video frame based on a pre-trained subject detection model;

[0018] The focused subject is determined according to the priority order corresponding to each candidate subject.

[0019] In a possible implementation, determining the focused subject in the to-be-detected video frame based on the preset focusing mode includes:

[0020] According to the priority order of each preset subject, detecting in sequence whether the video frame to be detected contains the preset subject;

[0021] The detected preset subject with the highest priority is used as the focus subject.

[0022] In a possible implementation manner, the method further includes:

[0023] When it is detected based on the preset focus mode that the video frame to be detected does not contain a focus subject, determining a target focus depth of the video frame to be detected, and determining an image position corresponding to the target focus depth as the focus area in the initial focus information of the video frame to be detected; and

[0024] The next video frame of the video frame to be detected is used again as the video frame to be detected, and the focus subject in the updated video frame to be detected is determined.

[0025] In a possible implementation, determining the target focus depth of the to-be-detected video frame includes:

[0026] Determining a central focus area of ​​the video frame to be detected;

[0027] The average depth of the central focus area is used as the target focus depth.

[0028] In a possible implementation, responding to an editing operation on the focus area of ​​the video frame and re-determining the focus area of ​​the video frame includes:

[0029] In response to a deletion operation on the focus area of ​​the video frame, re-determining the focus area of ​​the video frame based on the preset focus mode; or,

[0030] In response to a setting operation for the focus area of ​​the video frame, a target position corresponding to the setting operation is used as the focus area of ​​the video frame.

[0031] In a possible implementation, the initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize blur strength;

[0032] The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of the user.

[0033] In a possible implementation, the performing focus processing on the to-be-processed video based on the target focus information includes:

[0034] For any video frame, determine a focus area and a blur area based on the focus area in the target focus information of the video frame;

[0035] The blurred area is divided into a plurality of area blocks, and blurring is performed on the blurred area according to a depth difference between each area block and the focus area.

[0036] In a second aspect, an embodiment of the present disclosure further provides a video processing device, including:

[0037] An acquisition module is used to obtain the video to be processed;

[0038] A determination module, configured to determine initial focus information of the video to be processed based on a preset focus mode; wherein the initial focus information includes a focus area in each video frame;

[0039] an adjusting module, configured to modify the focus area of ​​the video frame in response to an editing operation on the focus area of ​​the video frame;

[0040] An updating module, configured to update the focus areas of the remaining video frames in the to-be-processed video based on the revised focus area to obtain target focus information;

[0041] A focusing module is used to perform focusing processing on the video to be processed based on the target focus information.

[0042] In one possible implementation, when determining the initial focus information of the video to be processed based on a preset focus method, the determination module is configured to:

[0043] Determining a video frame to be detected in the video to be processed, and determining a focus subject in the video frame to be detected based on the preset focusing method;

[0044] Tracking the position information of the focused subject in the video to be processed, resetting the video frame where tracking is interrupted as the video frame to be detected, and returning to the step of determining the focused subject;

[0045] Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.

[0046] In one possible implementation, the determination module, when determining the focused subject in the to-be-detected video frame based on the preset focusing mode, is configured to:

[0047] Detecting candidate subjects contained in the to-be-detected video frame based on a pre-trained subject detection model;

[0048] The focused subject is determined according to the priority order corresponding to each candidate subject.

[0049] In one possible implementation, the determination module, when determining the focused subject in the to-be-detected video frame based on the preset focusing mode, is configured to:

[0050] According to the priority order of each preset subject, detecting in sequence whether the video frame to be detected contains the preset subject;

[0051] The detected preset subject with the highest priority is used as the focus subject.

[0052] In a possible implementation manner, the determining module is further configured to:

[0053] When it is detected based on the preset focus mode that the video frame to be detected does not contain a focus subject, determining a target focus depth of the video frame to be detected, and determining an image position corresponding to the target focus depth as the focus area in the initial focus information of the video frame to be detected; and

[0054] The next video frame of the video frame to be detected is used again as the video frame to be detected, and the focus subject in the updated video frame to be detected is determined.

[0055] In a possible implementation, the adjustment module, when determining the target focus depth of the video frame to be detected, is configured to:

[0056] Determining a central focus area of ​​the video frame to be detected;

[0057] The average depth of the central focus area is used as the target focus depth.

[0058] In one possible implementation, the adjustment module, in response to an editing operation on the focus area of ​​the video frame and when revising the focus area of ​​the video frame, is configured to:

[0059] In response to a deletion operation on the focus area of ​​the video frame, re-determining the focus area of ​​the video frame based on the preset focus mode; or,

[0060] In response to a setting operation for the focus area of ​​the video frame, a target position corresponding to the setting operation is used as the focus area of ​​the video frame.

[0061] In a possible implementation, the initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize blur strength;

[0062] The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of the user.

[0063] In a possible implementation, the focusing module, when performing focus processing on the to-be-processed video based on the target focus information, is configured to:

[0064] For any video frame, determine a focus area and a blur area based on the focus area in the target focus information of the video frame;

[0065] The blurred area is divided into a plurality of area blocks, and blurring is performed on the blurred area according to a depth difference between each area block and the focus area.

[0066] In a third aspect, an embodiment of the present disclosure further provides a computer device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are performed.

[0067] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0068] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0070] FIG1 shows a flow chart of a video processing method provided by an embodiment of the present disclosure;

[0071] FIG2 shows a flowchart of a specific method for determining initial focus information in the video processing method provided by an embodiment of the present disclosure;

[0072] FIG3 shows a schematic diagram of a video processing device provided by an embodiment of the present disclosure;

[0073] FIG4 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0075] Research has found that video capture requires focus processing to enhance the presentation of specific subjects. Generally, video capture requires the use of high-performance devices such as cameras or camcorders to achieve good focus. However, in most cases, video capture is limited to devices with poor focus performance, such as mobile phones, resulting in poor focus in the captured videos.

[0076] Based on the above research, the present disclosure provides a video processing method, device, computer equipment and storage medium, which can automatically determine the initial focus information of the video to be processed based on a preset focus method after the video shooting is completed, so as to realize automatic focus processing of the video; then the user can edit the focus area of ​​the video frame, and the target focus information determined in this way can realize fast focus processing of the video to be processed, and the focus processing result can meet the user's focus requirements.

[0077] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0078] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0079] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0080] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0081] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0082] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0083] To facilitate understanding of this embodiment, a video processing method disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the video processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities. The computer device includes, for example, a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, the video processing method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0084] The video processing method provided by the embodiment of the present disclosure is described below by taking the execution subject as a terminal device as an example.

[0085] 1 is a flowchart of a video processing method according to an embodiment of the present disclosure. The method includes steps 101 to 105, wherein:

[0086] Step 101: Obtain the video to be processed.

[0087] Step 102: Determine initial focus information of the video to be processed based on a preset focus method.

[0088] Step 103: In response to the editing operation on the focus area of ​​the video frame, modify the focus area of ​​the video frame.

[0089] Step 104 : Based on the revised focus area, the focus areas of the remaining video frames in the to-be-processed video are updated to obtain target focus information.

[0090] Step 105: Perform focus processing on the video to be processed based on the target focus information.

[0091] The following is a detailed description of steps 101 to 105.

[0092] Regarding step 101:

[0093] The video to be processed may refer to a video to be focused, and none of the videos in the video to be processed may have been focused; or, the video to be processed may include some videos that have been focused during shooting, and may also include some videos that have not been focused during shooting.

[0094] For example, the length of the video to be processed is 10 minutes, and the 0th to 5th minutes may be the video that has been manually focused during shooting, so the 0th to 5th minutes may be the video that has been focused; the 6th to 10th minutes were not manually focused during shooting, so the 6th to 10th minutes may be the video that has not been focused.

[0095] Optionally, the method embodied in the present disclosure may be applied to a user end. In this case, when obtaining a video to be processed, a shooting instruction may be responded to, and an image acquisition device may be called to shoot a video to obtain the video to be processed.

[0096] Alternatively, when obtaining the video to be processed, a video uploaded by the user may be received and used as the video to be processed. Here, the video uploaded by the user may be a video uploaded locally by the user, or may be a video on the Internet selected by the user.

[0097] Regarding step 102:

[0098] Here, determining the initial focus information of the video to be processed based on a preset focus method may refer to automatically determining the initial focus information of each video frame of the video to be processed based on a preset focus method.

[0099] The initial focus information may include a focus area in each video frame. In addition, the initial focus information may also include a blur pattern and / or blur parameters.

[0100] The blurring style is used to indicate the style of blurring the video to be processed, and may include, for example, a heart shape, an ellipse, etc. The blurring styles of different video frames in the same video to be processed may be different.

[0101] The blur parameter is used to characterize the blur strength. For example, the blur strength of complete blur is 100%, and the blur strength of partial blur can be 10%, 20%, 50%, etc.

[0102] The blur style and / or blur parameters in the initial focus information may adopt default settings, may be manually set based on user instructions, or may be automatically set based on the video display content.

[0103] When automatically setting the blur style and / or blur parameters based on the video content, the blur style and / or blur parameters may be set to match the characteristics of the video content. For example, if the video content includes animals, an elliptical blur style may be used and the blur strength may be set to 70%; if the video content includes a little girl, a heart-shaped blur style may be used and the blur strength may be set to 20%.

[0104] In a possible implementation, different blurring styles and blurring parameters may be set for different video frames, and the same video frame may include multiple blurring styles and blurring parameters.

[0105] Specifically, when manually setting based on user instructions, the blur style and blur parameters can be set by any of the following two methods:

[0106] Method 1: Set the blur style and blur parameters according to time.

[0107] The user instructions include blur styles and blur parameters set for different time periods of the video to be processed. For example, if the length of the video to be processed is 20 minutes, the blur style from the 0th to the 15th minute can be set to a circle and the blur parameter can be 80%, and the blur style from the 16th to the 20th minute can be set to a diamond and the blur parameter can be 50%.

[0108] Method 2: Set the blur style and blur parameters according to the focused subject.

[0109] The user instructions may include blur styles and blur parameters set for different focused subjects in the video to be processed. For example, when the focused subject in the video to be processed is A, the blur style may be set to a square and the blur intensity may be set to 10%; when the focused subject in the video to be processed is B, the blur style may be set to a heart and the blur intensity may be set to 30%.

[0110] In another possible implementation, the initial focus information may further include a focus switching method. Because the focused subject in the processed video may change, when the focused subject changes, the focus also changes between adjacent video frames. The focus switching method refers to the method for switching the focus between adjacent video frames when the focused subject changes.

[0111] Optionally, the focus switching mode in the initial focus information may be a default preset value, or may be set in response to a setting instruction of the user.

[0112] Exemplarily, the focus switching manner may be switching at a certain speed and / or in a certain direction.

[0113] In another possible implementation, when determining the initial focus information of the video to be processed based on a preset focus method, the following steps may be used:

[0114] Step 201: For a video frame to be detected in the video to be processed, determine a focus subject in the video frame to be detected based on the preset focus mode.

[0115] Step 202 : Track the position information of the focused subject in the video to be processed, and use the video frame where tracking is interrupted as the video frame to be detected again, and return to step 201 .

[0116] Step 203: Determine initial focus information of the video to be processed based on the tracking results of each focused subject in the video to be processed.

[0117] In step 201 , the focused subject in the to-be-detected video frame may be, for example, an entity contained in the to-be-detected video frame, such as a person, an animal, or some objects.

[0118] Optionally, when determining the focused subject in the video frame to be detected, any one of the following methods may be used:

[0119] Method 1: Based on a pre-trained subject detection model, the candidate subjects contained in the video frame to be detected are detected, and then the focused subject is determined according to the priority order corresponding to each candidate subject.

[0120] When detecting the candidate subjects contained in the video frame to be detected based on a pre-trained subject detection model, exemplarily, the video frame to be detected can be input into the subject detection model, and the subject detection model can output all the candidate subjects contained in the video frame to be detected.

[0121] The priority order corresponding to each candidate subject may be preset, or may be set based on a user instruction.

[0122] Specifically, after detecting the candidate subjects contained in the video frame to be detected based on the subject detection model, the candidate subject identifiers can be displayed to the user, and then the priority order corresponding to each candidate subject can be determined in response to the user's sorting operation on the candidate subject identifiers.

[0123] In another possible implementation, the priority order corresponding to the candidate subjects may also be automatically determined based on the video content.

[0124] Specifically, after detecting the candidate subjects contained in the video frame to be detected, the appearance time of each candidate subject in the video frame to be detected can be determined respectively, and then the priority order corresponding to each candidate subject can be determined based on the appearance time of each candidate subject in the video frame to be detected.

[0125] Exemplarily, if the candidate subject appears for a longer time in the video frame to be detected, it means that the candidate subject accounts for a larger proportion in the video frame to be detected, and therefore, the candidate subject has a higher priority.

[0126] In an optional implementation, when determining the focus subject according to the priority order corresponding to each candidate subject, the candidate subject with the highest priority may be used as the focus subject.

[0127] Method 2: According to the priority order of each preset subject, the to-be-detected video frame is sequentially detected to see whether the preset subject is included, and the detected preset subject with the highest priority is used as the focus subject.

[0128] Here, the preset subject is different from the candidate subject in the above-mentioned method one. The candidate subject in method one refers to all subjects detected from the video frame to be detected and contained in the video frame to be detected, while here, the preset subject refers to all possible subjects pre-set before detecting the video frame to be detected.

[0129] The priority order of the preset subjects may be determined in the same manner as the priority order of the candidate subjects, which will not be described in detail here.

[0130] In an optional embodiment, when determining the focus subject, the video frames to be detected can be detected according to the priority ranking of each preset subject, and it can be determined in turn whether the video frame contains a preset subject until a preset subject can be detected, and then the preset subject is used as the focus subject.

[0131] For example, animal faces, human faces, animal torsos, and human torsos are used as preset subjects, and are prioritized in the order of human faces, animal faces, human torsos, and animal torsos. When detecting a video frame to be detected, the first step is to detect whether the frame contains a human face. If no human face is detected, the frame is then detected to determine whether it contains an animal face. If an animal face is detected, the animal face is used as the focus subject. If no animal face is detected, the frame is then detected to determine whether it contains a human torso, and so on, until a preset subject is detected.

[0132] In a possible implementation, when multiple subjects with the same priority are detected in the video frame to be detected, the subject with the largest area may be selected as the focus subject.

[0133] If it is detected that the video frame to be detected does not contain a focused subject, the target focus depth of the video frame to be detected can be determined, and the image position corresponding to the target focus depth is determined as the focus area in the initial focus information of the video frame to be detected, and the next video frame of the video frame to be detected is re-used as the video frame to be detected to determine the focused subject in the updated video frame to be detected.

[0134] It should be noted that focusing on a certain position / area refers to focusing on the depth corresponding to that position / area. When determining the target focus depth of the video frame to be detected, the center area of ​​the depth map corresponding to the video frame to be detected can be first determined, and then the average depth of each pixel in the center area is used as the target focus depth.

[0135] The central area of ​​the depth map may be, for example, an area of ​​a target size centered at the center of the depth map.

[0136] In order to improve the focusing speed, focus subject detection may not be performed on every frame of the video frame to be detected, but only on the video frames to be detected of the video frames to be detected. For the video frames other than the video frames to be detected, the focus may be determined by the subject tracking method.

[0137] For example, if the focused subject determined by the first frame of video to be detected (set as the m-th frame of video) is the face of user A, the face of user A can be located and the focus area can be determined through subject tracking from the m-th frame of video to the m+n-th frame of video. In the m+n+1-th frame of video, the subject tracking is interrupted and the face of user A cannot be located. Therefore, the m+n+1-th frame of video can be used as the video frame to be detected, and the focused subject in the m+n+1-th frame of video can be re-determined.

[0138] The tracking results of each focused subject in the video to be processed may refer to the position information of each focused subject in each video frame of the video to be processed. When determining the initial focus information of the video to be processed based on the tracking results of each focused subject in the video to be processed, the center of the position information of the focused subject in each video frame may be used as the focus area in the initial focus information, and the initial focus information may be obtained in combination with the blur pattern and / or blur parameters determined based on the above method.

[0139] Regarding steps 103 and 104:

[0140] In response to an editing operation on the focus area of ​​the video frame, re-determining the focus area of ​​the video frame includes the following two situations:

[0141] Case 1: In response to a deletion operation on the focus area of ​​the video frame, the focus area of ​​the video frame is re-determined based on the preset focus mode.

[0142] Specifically, since errors in detecting the focused subject may occur when automatically detecting the focus of the video frame of the video to be processed based on the above method, the video frame can be used as the video frame to be detected in response to a deletion operation, and the focused subject can be re-detected, thereby quickly correcting the focusing error and improving the focusing effect.

[0143] Case 2: In response to a setting operation for the focus area of ​​the video frame, a target position corresponding to the setting operation is used as the focus area of ​​the video frame.

[0144] For example, when a subject that the user wants to highlight appears in the video, but the subject is not the focus subject after the above detection, after detecting the user's setting operation for the focus area, the subject corresponding to the setting operation can be set as the focus subject, thereby realizing the user's personalized setting of the focus.

[0145] When the focus area of ​​any video frame changes (here it means that the focus area of ​​the video frame is re-determined based on the above method), the focus areas of the remaining video frames should also change accordingly. Therefore, when the focus area of ​​any video frame changes, the focus subject can be determined based on the focus area of ​​the video frame, and the subject can be tracked for the video frames after the video frame, and the focus areas of the video frames after the video frame can be updated based on the tracking results.

[0146] Alternatively, when the focus area of ​​any video frame changes, subject tracking can be performed on the video frames before and after the video frame, and the focus areas of the video frames before and after the video frame are updated based on the tracking results.

[0147] Regarding step 105:

[0148] When focusing the video to be processed based on the target focus information, for example, for any frame of video, the focus area and the blurred area can be determined based on the focus area in the target focus information of the frame of video, and then the blurred area can be divided into multiple area blocks, and the blurred area can be blurred according to the depth difference between each area block and the focus area.

[0149] The focus area may be an area whose corresponding depth is the depth corresponding to the focus area. For example, if the focus area is a certain point on a face, the depth of the entire face is the same. Therefore, the focus area may be a face area.

[0150] In the video frame, all areas except the focused area can be blurred. In practical applications, if all areas are blurred with the same blurring intensity, the focused subject may be overly prominent, affecting the final display effect. Therefore, the blurred areas can be blurred based on the depth difference between each part of the blurred area and the focused area.

[0151] Exemplarily, dividing the blurred area into a plurality of blocks may involve dividing the blurred area into a plurality of blocks according to a preset size, and the areas of the divided blocks may be the same. When blurring the blurred area based on the depth difference between each block and the focused area, a blur intensity corresponding to each block may be determined based on an average depth value of the focused area and an average depth value of each block, and then based on the difference between the average depth values ​​and a preset scaling factor, and blurring is performed according to the blur intensity.

[0152] Here, the greater the difference between the average depth values, the greater the blur strength corresponding to the area block can be.

[0153] In a possible implementation, after focusing processing is performed on the video to be processed based on the target blur information, the target blur information may be saved as a template so that other videos can be directly processed based on the target blur information.

[0154] Here, when saving the target blur information as a template, since the focus area of ​​other videos must be different from the focus area of ​​the video to be processed, the template may not include the focus area in the target blur information. For example, it may include the priority order of candidate subjects, or the priority order of preset subjects, blur style, blur parameters, switching method between focus points, etc., one or more of them.

[0155] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0156] Based on the same inventive concept, the embodiment of the present disclosure also provides a video processing device corresponding to the video processing method. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned video method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0157] 3 , which is a schematic diagram of the architecture of a video processing device provided by an embodiment of the present disclosure, the device includes: an acquisition module 301, a determination module 302, an adjustment module 303, an update module 304, and a focus module 305; wherein,

[0158] An acquisition module 301 is used to acquire a video to be processed;

[0159] The determination module 302 is configured to determine initial focus information of the video to be processed based on a preset focus mode; wherein the initial focus information includes a focus area in each video frame;

[0160] An adjustment module 303 is configured to modify the focus area of ​​the video frame in response to an editing operation on the focus area of ​​the video frame;

[0161] An updating module 304 is configured to update the focus areas of the remaining video frames in the to-be-processed video based on the revised focus area to obtain target focus information;

[0162] The focus module 305 is configured to perform focus processing on the video to be processed based on the target focus information.

[0163] In a possible implementation, the determining module 302, when determining the initial focus information of the video to be processed based on a preset focus method, is configured to:

[0164] Determining a video frame to be detected in the video to be processed, and determining a focus subject in the video frame to be detected based on the preset focusing method;

[0165] Tracking the position information of the focused subject in the video to be processed, resetting the video frame where tracking is interrupted as the video frame to be detected, and returning to the step of determining the focused subject;

[0166] Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.

[0167] In a possible implementation, the determination module 302, when determining the focused subject in the to-be-detected video frame based on the preset focusing mode, is configured to:

[0168] Detecting candidate subjects contained in the to-be-detected video frame based on a pre-trained subject detection model;

[0169] The focused subject is determined according to the priority order corresponding to each candidate subject.

[0170] In a possible implementation, the determination module 302, when determining the focused subject in the to-be-detected video frame based on the preset focusing mode, is configured to:

[0171] According to the priority order of each preset subject, detecting in sequence whether the video frame to be detected contains the preset subject;

[0172] The detected preset subject with the highest priority is used as the focus subject.

[0173] In a possible implementation, the determining module 302 is further configured to:

[0174] When it is detected based on the preset focus mode that the video frame to be detected does not contain a focus subject, determining a target focus depth of the video frame to be detected, and determining an image position corresponding to the target focus depth as the focus area in the initial focus information of the video frame to be detected; and

[0175] The next video frame of the video frame to be detected is used again as the video frame to be detected, and the focus subject in the updated video frame to be detected is determined.

[0176] In a possible implementation, the adjustment module 303, when determining the target focus depth of the video frame to be detected, is configured to:

[0177] Determining a central focus area of ​​the video frame to be detected;

[0178] The average depth of the central focus area is used as the target focus depth.

[0179] In one possible implementation, the adjustment module 303, in response to an editing operation on the focus area of ​​the video frame and re-determining the focus area of ​​the video frame, is configured to:

[0180] In response to a deletion operation on the focus area of ​​the video frame, re-determining the focus area of ​​the video frame based on the preset focus mode; or,

[0181] In response to a setting operation for the focus area of ​​the video frame, a target position corresponding to the setting operation is used as the focus area of ​​the video frame.

[0182] In a possible implementation, the initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize blur strength;

[0183] The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of the user.

[0184] In a possible implementation, the focusing module 305, when performing focus processing on the to-be-processed video based on the target focus information, is configured to:

[0185] For any video frame, determine a focus area and a blur area based on the focus area in the target focus information of the video frame;

[0186] The blurred area is divided into a plurality of area blocks, and blurring is performed on the blurred area according to a depth difference between each area block and the focus area.

[0187] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0188] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Referring to Figure 4, there is a schematic diagram of the structure of the computer device 400 provided in the embodiment of the present disclosure. The computer device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., or various forms of servers, such as independent servers or server clusters. The computer device shown in Figure 4 is only an example and should not bring any limitations to the functions and scope of use of the embodiment of the present disclosure.

[0189] As shown in FIG4 , a computer device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory device (ROM) 402 or a program loaded from a storage device 408 into a random access memory device (RAM) 403. Various programs and data required for the operation of the computer device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0190] Typically, the following devices may be connected to I / O interface 405: input device 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output device 407 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 408 including, for example, a magnetic tape, hard disk, etc.; and communication device 409. Communication device 409 may allow computer device 400 to communicate with other devices wirelessly or by wire to exchange data. Although FIG4 illustrates computer device 400 with various devices, it should be understood that not all of the illustrated devices are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0191] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the video processing method. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0192] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes the steps of the video processing method described in the above method embodiment. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0193] The embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the video processing method described in the above method embodiment. For details, please refer to the above method embodiment and will not be repeated here.

[0194] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0195] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0196] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0197] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0198] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0199] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A video processing method, comprising: Get the video to be processed; Based on a preset focusing method, determining initial focusing information of the video to be processed; wherein the initial focusing information includes a focusing area in a video frame; In response to an editing operation on the focus area of ​​the video frame, modifying the focus area of ​​the video frame; Based on the revised focus area, the focus areas of the remaining video frames in the to-be-processed video are updated to obtain target focus information; Focus processing is performed on the video to be processed based on the target focus information.

2. The method according to claim 1, wherein: The determining the initial focus information of the video to be processed based on a preset focus mode includes: Determine a video frame to be detected in the video to be processed, and determine a focus subject in the video frame to be detected based on the preset focusing method; Tracking the position information of the focus subject in the video to be processed, reusing the video frame where the tracking is interrupted as the video frame to be detected, and returning to the step of determining the focus subject; Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.

3. The method according to claim 2, wherein: The determining the focus subject in the to-be-detected video frame based on the preset focus mode includes: Based on a pre-trained subject detection model, detecting candidate subjects contained in the video frame to be detected; The focus subject is determined according to the priority order corresponding to each candidate subject.

4. The method according to claim 2, wherein: The determining the focus subject in the to-be-detected video frame based on the preset focus mode includes: According to the priority order of each preset subject, sequentially detecting whether the preset subject is contained in the video frame to be detected; The detected preset subject with the highest priority is used as the focus subject.

5. The method according to any one of claims 2 to 4, further comprising: Based on the preset focusing method, it is detected that the video frame to be detected does not contain the focus main In the case of a body, determining the target focus depth of the video frame to be detected, and determining the image position corresponding to the target focus depth as the focus area in the initial focus information of the video frame to be detected; and The next video frame of the to-be-detected video frame is used again as the to-be-detected video frame, and the focus subject in the updated to-be-detected video frame is determined.

6. The method according to claim 5, wherein: The determining the target focus depth of the to-be-detected video frame includes: Determine the central focus area of ​​the video frame to be detected; The mean depth of the central focus area is used as the target focus depth.

7. The method according to any one of claims 1 to 6, wherein: The step of modifying the focus area of ​​the video frame in response to the editing operation on the focus area of ​​the video frame includes: In response to a deletion operation on the focus area of ​​the video frame, the focus area of ​​the video frame is modified based on the preset focus mode; or, In response to a setting operation for the focus area of ​​the video frame, a target position corresponding to the setting operation is used as the focus area of ​​the video frame.

8. The method according to any one of claims 1 to 7, wherein: The initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize the blur strength; The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of a user.

9. The method according to any one of claims 1 to 8, wherein: The performing focus processing on the to-be-processed video based on the target focus information includes: For any video frame, determining a blurred area based on a focus area in target focus information of the video frame; The blurred area is divided into a plurality of area blocks, and the blurred area is blurred according to a depth difference between each area block and the focus area.

10. A video processing device, comprising: An acquisition module, configured to acquire a video to be processed; A determination module is configured to determine initial focus information of the video to be processed based on a preset focus mode; wherein the initial focus information includes a focus area in each video frame; The adjustment module is configured to respond to the editing operation on the focus area of ​​the video frame and modify the focus area of ​​the video frame; An updating module is configured to update the focus areas of the remaining video frames in the to-be-processed video based on the revised focus area to obtain target focus information; A focusing module is configured to perform focusing processing on the video to be processed based on the target focusing information.

11. A computer device comprising: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the video processing method according to any one of claims 1 to 9 are performed.

12. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the video processing method according to any one of claims 1 to 9 are executed.

Images