Video processing method and device, computer equipment and storage medium
Through a video processing method, the preset focus method and focus area editing operation are used to solve the problem of poor focus effect in video shooting, and the video's autofocus processing and fast focus effect are realized.
Patent Information
- Application Number
- CN202310975336.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-03
- Publication Date
- 2025-02-14
AI Technical Summary
During video shooting, when using devices with poor focus performance (such as mobile phones), it is easy to lead to poor focus effect of video.
Provide a video processing method, by obtaining the video to be processed, determining the initial focus information based on the preset focus method, modifying it in response to the editing operation of the focus area, and updating the focus area of other video frames based on the revised focus area, and finally Focus processing is performed based on the target focus information.
The automatic focus processing of video is realized. Users can edit the focus area of the video frame and quickly obtain satisfactory focus effects. It is suitable for videos captured by devices with poor focus performance.
Smart Images
Figure CN119450212A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a video processing method, apparatus, computer equipment and storage medium. Background Art
[0002] When shooting a video, it is necessary to perform focus processing to improve the display effect of a specific subject in the video. Generally, video shooting requires the use of devices with high processing performance such as cameras or camcorders to achieve a better focus effect. However, in most cases, when shooting a video, you can only use devices with poor focus performance such as mobile phones, which will result in poor focus effects on the video. Summary of the invention
[0003] The embodiments of the present disclosure at least provide a video processing method, apparatus, computer equipment and storage medium.
[0004] In a first aspect, an embodiment of the present disclosure provides a video processing method, including:
[0005] Get the video to be processed;
[0006] Based on a preset focusing method, determining initial focusing information of the video to be processed; wherein the initial focusing information includes a focusing area in each video frame;
[0007] In response to an editing operation on the focus area of the video frame, modifying the focus area of the video frame;
[0008] Based on the revised focus area, the focus areas of the remaining video frames in the to-be-processed video are updated to obtain target focus information;
[0009] Focus processing is performed on the video to be processed based on the target focus information.
[0010] In a possible implementation manner, determining the initial focus information of the video to be processed based on a preset focus mode includes:
[0011] Determine a video frame to be detected in the video to be processed, and determine a focus subject in the video frame to be detected based on the preset focusing method;
[0012] Tracking the position information of the focus subject in the video to be processed, reusing the video frame where the tracking is interrupted as the video frame to be detected, and returning to the step of determining the focus subject;
[0013] Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.
[0014] In a possible implementation manner, determining the focus subject in the to-be-detected video frame based on the preset focus mode includes:
[0015] Based on a pre-trained subject detection model, detecting candidate subjects contained in the video frame to be detected;
[0016] The focus subject is determined according to the priority order corresponding to each candidate subject.
[0017] In a possible implementation manner, determining the focus subject in the to-be-detected video frame based on the preset focus mode includes:
[0018] According to the priority order of each preset subject, sequentially detecting whether the preset subject is contained in the video frame to be detected;
[0019] The detected preset subject with the highest priority is used as the focus subject.
[0020] In a possible implementation manner, the method further includes:
[0021] When it is detected that the video frame to be detected does not contain a focus subject based on the preset focus mode, a target focus depth of the video frame to be detected is determined, and an image position corresponding to the target focus depth is determined as a focus area in the initial focus information of the video frame to be detected; and
[0022] The next video frame of the to-be-detected video frame is used again as the to-be-detected video frame, and the focus subject in the updated to-be-detected video frame is determined.
[0023] In a possible implementation manner, determining the target focus depth of the to-be-detected video frame includes:
[0024] Determine the central focus area of the video frame to be detected;
[0025] The mean depth of the central focus area is used as the target focus depth.
[0026] In a possible implementation manner, in response to the editing operation on the focus area of the video frame, re-determining the focus area of the video frame includes:
[0027] In response to a deletion operation on the focus area of the video frame, re-determining the focus area of the video frame based on the preset focus mode; or,
[0028] In response to a setting operation for the focus area of the video frame, a target position corresponding to the setting operation is used as the focus area of the video frame.
[0029] In a possible implementation manner, the initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize blur strength;
[0030] The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of a user.
[0031] In a possible implementation manner, the performing focus processing on the to-be-processed video based on the target focus information includes:
[0032] For any video frame, determine a focus area and a blur area based on the focus area in the target focus information of the video frame;
[0033] The blurred area is divided into a plurality of area blocks, and the blurred area is blurred according to a depth difference between each area block and the focus area.
[0034] In a second aspect, the present disclosure also provides a video processing device, including:
[0035] An acquisition module is used to acquire the video to be processed;
[0036] A determination module, configured to determine initial focus information of the video to be processed based on a preset focus mode; wherein the initial focus information includes a focus area in each video frame;
[0037] An adjustment module, configured to modify the focus area of the video frame in response to an editing operation on the focus area of the video frame;
[0038] An updating module, configured to update the focus areas of the remaining video frames in the to-be-processed video based on the revised focus area, to obtain target focus information;
[0039] A focusing module is used to perform focusing processing on the video to be processed based on the target focusing information.
[0040] In a possible implementation manner, when determining the initial focus information of the video to be processed based on a preset focus mode, the determination module is used to:
[0041] Determine a video frame to be detected in the video to be processed, and determine a focus subject in the video frame to be detected based on the preset focusing method;
[0042] Tracking the position information of the focus subject in the video to be processed, reusing the video frame where the tracking is interrupted as the video frame to be detected, and returning to the step of determining the focus subject;
[0043] Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.
[0044] In a possible implementation manner, the determination module, when determining the focus subject in the to-be-detected video frame based on the preset focus mode, is configured to:
[0045] Based on a pre-trained subject detection model, detecting candidate subjects contained in the video frame to be detected;
[0046] The focus subject is determined according to the priority order corresponding to each candidate subject.
[0047] In a possible implementation manner, the determination module, when determining the focus subject in the to-be-detected video frame based on the preset focus mode, is configured to:
[0048] According to the priority order of each preset subject, sequentially detecting whether the preset subject is contained in the video frame to be detected;
[0049] The detected preset subject with the highest priority is used as the focus subject.
[0050] In a possible implementation manner, the determining module is further configured to:
[0051] When it is detected that the video frame to be detected does not contain a focus subject based on the preset focus mode, a target focus depth of the video frame to be detected is determined, and an image position corresponding to the target focus depth is determined as a focus area in the initial focus information of the video frame to be detected; and
[0052] The next video frame of the to-be-detected video frame is used again as the to-be-detected video frame, and the focused subject in the updated to-be-detected video frame is determined.
[0053] In a possible implementation manner, the adjustment module, when determining the target focus depth of the video frame to be detected, is used to:
[0054] Determine the central focus area of the video frame to be detected;
[0055] The mean depth of the central focus area is used as the target focus depth.
[0056] In a possible implementation manner, the adjustment module, in response to an editing operation on the focus area of the video frame, when revising the focus area of the video frame, is configured to:
[0057] In response to a deletion operation on the focus area of the video frame, re-determining the focus area of the video frame based on the preset focus mode; or,
[0058] In response to a setting operation for the focus area of the video frame, a target position corresponding to the setting operation is used as the focus area of the video frame.
[0059] In a possible implementation manner, the initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize blur strength;
[0060] The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of a user.
[0061] In a possible implementation manner, the focus module, when performing focus processing on the to-be-processed video based on the target focus information, is used to:
[0062] For any video frame, determine a focus area and a blur area based on the focus area in the target focus information of the video frame;
[0063] The blurred area is divided into a plurality of area blocks, and the blurred area is blurred according to a depth difference between each area block and the focus area.
[0064] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.
[0065] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.
[0066] The video processing method, apparatus, computer equipment and storage medium provided by the embodiments of the present disclosure can automatically determine the initial focus information of the video to be processed based on a preset focus method after the video shooting is completed, so as to realize automatic focus processing of the video; then the user can edit the focus area of the video frame, and the target focus information determined in this way can realize fast focus processing of the video to be processed, and the focus processing result can meet the user's focus requirements.
[0067] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0069] Figure 1 A flow chart of a video processing method provided by an embodiment of the present disclosure is shown;
[0070] Figure 2 A flowchart showing a specific method for determining initial focus information in the video processing method provided by an embodiment of the present disclosure is shown;
[0071] Figure 3 A schematic diagram of a video processing device provided by an embodiment of the present disclosure is shown;
[0072] Figure 4 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0073] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0074] Research has found that when shooting a video, it is necessary to perform focus processing to improve the display effect of a specific subject in the video. Generally, video shooting requires the use of devices with high processing performance such as cameras or camcorders to achieve a better focus effect. However, in most cases, when shooting a video, you can only use devices with poor focus performance such as mobile phones, which will result in poor focus effects on the video.
[0075] Based on the above research, the present disclosure provides a video processing method, apparatus, computer equipment and storage medium, which can automatically determine the initial focus information of the video to be processed based on a preset focus method after the video shooting is completed, so as to realize automatic focus processing of the video; then the user can edit the focus area of the video frame, and the target focus information determined in this way can realize fast focus processing of the video to be processed, and the focus processing result can meet the user's focus requirements.
[0076] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0077] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.
[0078] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0079] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0080] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0081] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0082] To facilitate understanding of the present embodiment, a video processing method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the video processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the video processing method can be implemented by a processor calling a computer-readable instruction stored in a memory.
[0083] The video processing method provided by the embodiment of the present disclosure is described below by taking the execution subject as a terminal device as an example.
[0084] See also Figure 1 FIG. 1 is a flowchart of a video processing method provided by an embodiment of the present disclosure, wherein the method includes steps 101 to 105, wherein:
[0085] Step 101: Obtain the video to be processed.
[0086] Step 102: Determine initial focus information of the video to be processed based on a preset focus method.
[0087] Step 103: In response to the editing operation on the focus area of the video frame, modify the focus area of the video frame.
[0088] Step 104: Based on the revised focus area, the focus areas of the remaining video frames in the to-be-processed video are updated to obtain target focus information.
[0089] Step 105: Perform focus processing on the video to be processed based on the target focus information.
[0090] The following is a detailed description of the above steps 101 to 105.
[0091] For step 101:
[0092] The video to be processed may refer to a video to be focused, and none of the videos in the video to be processed may have been focused; or, the video to be processed may include some videos that have been focused during shooting, and may also include some videos that have not been focused during shooting.
[0093] Exemplarily, the length of the video to be processed is 10 minutes, and the 0th to 5th minutes may be the video that has been manually focused during shooting, so the 0th to 5th minutes may be the video that has been focused; the 6th to 10th minutes were not manually focused during shooting, so the 6th to 10th minutes may be the video that has not been focused.
[0094] Optionally, the method embodied in the present disclosure may be applied to a user end. In this case, when acquiring a video to be processed, a shooting instruction may be responded to, and an image acquisition device may be called to shoot a video to obtain the video to be processed.
[0095] Alternatively, when obtaining the video to be processed, a video uploaded by a user may be received and used as the video to be processed. Here, the video uploaded by the user may be a video uploaded locally by the user, or may be a video on the network selected by the user.
[0096] Regarding step 102:
[0097] Here, determining the initial focus information of the video to be processed based on a preset focus mode may refer to automatically determining the initial focus information of each video frame of the video to be processed based on a preset focus mode.
[0098] The initial focus information may include a focus area in each video frame. In addition, the initial focus information may also include a blur style and / or blur parameters.
[0099] The blur style is used to represent the style of blurring the video to be processed, for example, it may include a heart shape, an ellipse, etc.; the blur styles of different video frames in the same video to be processed may be different.
[0100] The blur parameter is used to characterize the blur strength. For example, the blur strength of complete blur is 100%, and the blur strength of partial blur can be 10%, 20%, 50%, etc.
[0101] The blur style and / or blur parameters in the initial focus information may adopt default settings, may be manually set based on user instructions, or may be automatically set based on video display content.
[0102] When the blurring style and / or blurring parameters are automatically set based on the video display content, the blurring style and / or blurring parameters that match the characteristics of the video display content may be set. For example, when the video display content includes animals, an elliptical blurring style may be used and the blurring strength may be set to 70%; when the video display content includes a little girl, a heart-shaped blurring style may be used and the blurring strength may be set to 20%.
[0103] In a possible implementation, different blurring styles and blurring parameters may be set for different video frames, and the same video frame may include a plurality of blurring styles and blurring parameters.
[0104] Specifically, when manually setting based on user instructions, the blur style and blur parameters can be set by any of the following two methods:
[0105] Method 1: Set the blur style and blur parameters according to time.
[0106] The user instructions include blur styles and blur parameters set for different time periods of the video to be processed. For example, if the length of the video to be processed is 20 minutes, the blur style from the 0th to the 15th minute can be set to a circle and the blur parameter can be 80%, and the blur style from the 16th to the 20th minute can be set to a diamond and the blur parameter can be 50%.
[0107] Method 2: Set the blur style and blur parameters according to the focused subject.
[0108] The user instruction may include blur styles and blur parameters set for different focus subjects in the video to be processed. For example, when the focus subject of the video to be processed is A, the blur style may be set to a square and the blur strength may be set to 10%; when the focus subject of the video to be processed is B, the blur style may be set to a heart shape and the blur strength may be set to 30%.
[0109] In another possible implementation, the initial focus information may further include a focus switching method. Since the focus subject in the video to be processed may change, when the focus subject changes, the focus between adjacent video frames also changes accordingly. The focus switching method refers to the switching method of the focus between adjacent video frames when the focus subject changes.
[0110] Optionally, the focus switching mode in the initial focus information may be a default preset value, or may be set in response to a setting instruction of the user.
[0111] Exemplarily, the focus switching method may be switching at a certain speed and / or in a certain direction.
[0112] In another possible implementation manner, when determining the initial focus information of the video to be processed based on a preset focus method, the initial focus information of the video to be processed may be determined by: Figure 2 Steps shown:
[0113] Step 201: for a to-be-detected video frame of the to-be-processed video, based on the preset focusing method, determine a focus subject in the to-be-detected video frame.
[0114] Step 202 , tracking the position information of the focused subject in the video to be processed, and re-using the video frame where the tracking is interrupted as the video frame to be detected, and returning to step 201 .
[0115] Step 203: Determine initial focus information of the video to be processed based on the tracking results of each focus subject in the video to be processed.
[0116] In step 201, the focus subject in the to-be-detected video frame may be, for example, an entity contained in the to-be-detected video frame, such as a person, an animal, or some objects.
[0117] Optionally, when determining the focus subject in the video frame to be detected, any one of the following methods may be used:
[0118] Method 1: Based on a pre-trained subject detection model, the candidate subjects contained in the video frame to be detected are detected, and then the focus subject is determined according to the priority order corresponding to each candidate subject.
[0119] When detecting the candidate subjects contained in the video frame to be detected based on a pre-trained subject detection model, exemplarily, the video frame to be detected can be input into the subject detection model, and the subject detection model can output all the candidate subjects contained in the video frame to be detected.
[0120] The priority order corresponding to each candidate subject may be preset, or may be set based on a user instruction.
[0121] Specifically, after detecting the candidate subjects contained in the video frame to be detected based on the subject detection model, the candidate subject identifiers can be displayed to the user, and then the priority order corresponding to each candidate subject can be determined in response to the user's sorting operation on the candidate subject identifiers.
[0122] In another possible implementation, the priority order corresponding to each candidate subject may also be automatically determined based on the video content.
[0123] Specifically, after detecting the candidate subjects contained in the video frame to be detected, the appearance time of each candidate subject in the video frame to be detected can be determined respectively, and then based on the appearance time of each candidate subject in the video frame to be detected, the priority order corresponding to each candidate subject can be determined.
[0124] Exemplarily, if the candidate subject appears for a longer time in the video frame to be detected, it means that the candidate subject accounts for a larger proportion of the video frame to be detected, and therefore, the candidate subject has a higher priority.
[0125] In an optional implementation, when determining the focus subject according to the priority order corresponding to each candidate subject, the candidate subject with the highest priority may be used as the focus subject.
[0126] Method 2: According to the priority order of each preset subject, the to-be-detected video frame is sequentially detected to see whether the preset subject is included, and the detected preset subject with the highest priority is taken as the focus subject.
[0127] Here, the preset subject is distinguished from the candidate subject in the above-mentioned method one. The candidate subject in method one refers to all the subjects detected from the video frame to be detected and contained in the video frame to be detected, while here, the preset subject refers to all the possible subjects pre-set before the video frame to be detected is detected.
[0128] The priority order of the preset subjects may be determined in the same manner as the priority order of the candidate subjects, which will not be described in detail herein.
[0129] In an optional embodiment, when determining the focus subject, the video frames to be detected can be detected according to the priority ranking of each preset subject, and it can be determined in turn whether the video frames contain the preset subjects until a preset subject can be detected, and then the preset subject is used as the focus subject.
[0130] Exemplarily, animal face, human face, animal torso, and human torso are used as preset subjects, and the priorities are arranged in the order of human face, animal face, human torso, and animal torso. When detecting a video frame to be detected, first detect whether the video frame contains a human face. If no human face is detected, continue to detect whether the video frame contains an animal face. If an animal face is detected at this time, the animal face is used as the focus subject. If no animal face is detected, continue to detect whether the video frame contains a human torso, and so on, until a preset subject is detected.
[0131] In a possible implementation, when multiple subjects with the same priority are detected in the video frame to be detected, the subject with the largest area may be selected as the focus subject.
[0132] If it is detected that the video frame to be detected does not contain a focused subject, the target focus depth of the video frame to be detected can be determined, and the image position corresponding to the target focus depth can be determined as the focus area in the initial focus information of the video frame to be detected, and the next video frame of the video frame to be detected is used again as the video frame to be detected, and the focused subject in the updated video frame to be detected is determined.
[0133] Here, it should be noted that focusing on a certain position / area means focusing on the depth corresponding to the position / area. When determining the target focus depth of the video frame to be detected, the central area of the depth map corresponding to the video frame to be detected can be determined first, and then the average depth of each pixel in the central area is used as the target focus depth.
[0134] The central area of the depth map may be, for example, an area of a target size centered at the center of the depth map.
[0135] In order to improve the focusing speed, focus subject detection may not be performed on every frame of the video frame to be detected, but only on the video frames to be detected of the video frames to be detected. For the video frames not to be detected of the video frames to be detected, the focus may be determined by the subject tracking method.
[0136] Exemplarily, if the focused subject determined by the first frame of video to be detected (set as the m-th frame of video) is the face of user A, from the m-th frame of video to the m+n-th frame of video, the face of user A can be located and the focus area can be determined through subject tracking. In the m+n+1-th frame of video, the subject tracking is interrupted and the face of user A cannot be located. Therefore, the m+n+1-th frame of video can be used as the video frame to be detected, and the focused subject in the m+n+1-th frame of video can be re-determined.
[0137] The tracking results of each focused subject in the video to be processed may refer to the position information of each focused subject in each video frame of the video to be processed. When determining the initial focus information of the video to be processed based on the tracking results of each focused subject in the video to be processed, the center of the position information of the focused subject in each video frame may be used as the focus area in the initial focus information, and the initial focus information may be obtained in combination with the blur style and / or blur parameters determined based on the above method.
[0138] For steps 103 and 104:
[0139] In response to the editing operation on the focus area of the video frame, the focus area of the video frame is re-determined, including the following two cases:
[0140] Case 1: in response to a deletion operation on the focus area of the video frame, the focus area of the video frame is re-determined based on the preset focus mode.
[0141] Specifically, since errors in detecting the focused subject may occur when automatically detecting the focus of the video frame of the video to be processed based on the above method, the video frame can be used as the video frame to be detected in response to a deletion operation, and the focused subject can be re-detected, thereby quickly correcting the focus error and improving the focus effect.
[0142] Case 2: in response to a setting operation for the focus area of the video frame, a target position corresponding to the setting operation is used as the focus area of the video frame.
[0143] For example, when a subject appears in the video that the user wants to highlight, but the subject is not the focus subject after the above detection, after detecting the user's setting operation on the focus area, the subject corresponding to the setting operation can be set as the focus subject, thereby realizing the user's personalized setting of the focus.
[0144] When the focus area of any video frame changes (here it means that the focus area of the video frame is re-determined based on the above method), the focus areas of the remaining video frames should also change accordingly. Therefore, when the focus area of any video frame changes, the focus subject can be determined based on the focus area of the video frame, and the subject can be tracked for the video frames after the video frame, and the focus areas of the video frames after the video frame can be updated based on the tracking results.
[0145] Alternatively, when the focus area of any video frame changes, subject tracking can be performed on the video frames before and after the video frame, and the focus areas of the video frames before and after the video frame are updated based on the tracking results.
[0146] Regarding step 105:
[0147] When the video to be processed is focused based on the target focus information, exemplarily, for any frame of the video, the focus area and the blurred area can be determined based on the focus area in the target focus information of the frame of the video, and then the blurred area can be divided into multiple area blocks, and the blurred area can be blurred according to the depth difference between each area block and the focus area.
[0148] The focus area may be an area whose corresponding depth is the depth corresponding to the focus area. For example, if the focus area is a certain point on a face, the depth of the entire face is the same. Therefore, the focus area may be a face area.
[0149] In the video frame, all areas except the focus area can be blurred areas. In practical applications, if all areas are blurred with the same blurring strength, the focus subject may be overly prominent, affecting the final display effect. Therefore, the blurred area can be blurred based on the depth difference between each part of the blurred area and the focus area.
[0150] Exemplarily, the defocused area is divided into a plurality of area blocks, and the defocused area may be divided according to a preset size, and the areas of the divided area blocks may be the same. When the defocused area is defocused according to the depth difference between each area block and the focus area, the defocusing strength corresponding to the area block may be determined based on the average depth value of the focus area and the average depth value of each area block, and then based on the difference between the average depth values and a preset proportionality coefficient, and the defocusing is performed according to the defocusing strength.
[0151] Here, the greater the difference between the average depth values, the greater the blur strength corresponding to the area block can be.
[0152] In a possible implementation, after the to-be-processed video is focused based on the target blur information, the target blur information may be saved as a template so that when other videos are processed, they can be directly processed based on the target blur information.
[0153] Here, when the target blur information is saved as a template, since the focus area of other videos must be different from the focus area of the video to be processed, the template may not include the focus area in the target blur information. For example, it may include the priority order of candidate subjects, or the priority order of preset subjects, blur style, blur parameters, switching method between focus points, etc., one or more of the above.
[0154] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0155] Based on the same inventive concept, a video processing device corresponding to the video processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned video method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0156] Reference Figure 3, which is a schematic diagram of the architecture of a video processing device provided by an embodiment of the present disclosure, the device includes: an acquisition module 301, a determination module 302, an adjustment module 303, an update module 304, and a focus module 305; wherein,
[0157] An acquisition module 301 is used to acquire a video to be processed;
[0158] A determination module 302 is used to determine the initial focus information of the video to be processed based on a preset focus mode; wherein the initial focus information includes a focus area in each video frame;
[0159] An adjustment module 303, configured to modify the focus area of the video frame in response to an editing operation on the focus area of the video frame;
[0160] An updating module 304 is used to update the focus areas of the remaining video frames in the to-be-processed video based on the revised focus area to obtain target focus information;
[0161] The focusing module 305 is used to perform focusing processing on the video to be processed based on the target focusing information.
[0162] In a possible implementation manner, the determination module 302, when determining the initial focus information of the video to be processed based on a preset focus mode, is used to:
[0163] Determine a video frame to be detected in the video to be processed, and determine a focus subject in the video frame to be detected based on the preset focusing method;
[0164] Tracking the position information of the focus subject in the video to be processed, reusing the video frame where the tracking is interrupted as the video frame to be detected, and returning to the step of determining the focus subject;
[0165] Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.
[0166] In a possible implementation manner, the determination module 302, when determining the focus subject in the to-be-detected video frame based on the preset focus mode, is configured to:
[0167] Based on a pre-trained subject detection model, detecting candidate subjects contained in the video frame to be detected;
[0168] The focus subject is determined according to the priority order corresponding to each candidate subject.
[0169] In a possible implementation manner, the determination module 302, when determining the focus subject in the to-be-detected video frame based on the preset focus mode, is configured to:
[0170] According to the priority order of each preset subject, sequentially detecting whether the preset subject is contained in the video frame to be detected;
[0171] The detected preset subject with the highest priority is used as the focus subject.
[0172] In a possible implementation manner, the determining module 302 is further configured to:
[0173] When it is detected that the video frame to be detected does not contain a focus subject based on the preset focus mode, a target focus depth of the video frame to be detected is determined, and an image position corresponding to the target focus depth is determined as a focus area in the initial focus information of the video frame to be detected; and
[0174] The next video frame of the to-be-detected video frame is used again as the to-be-detected video frame, and the focused subject in the updated to-be-detected video frame is determined.
[0175] In a possible implementation manner, the adjustment module 303, when determining the target focus depth of the video frame to be detected, is used to:
[0176] Determine the central focus area of the video frame to be detected;
[0177] The mean depth of the central focus area is used as the target focus depth.
[0178] In a possible implementation manner, the adjustment module 303, in response to an editing operation on the focus area of the video frame and re-determining the focus area of the video frame, is configured to:
[0179] In response to a deletion operation on the focus area of the video frame, re-determining the focus area of the video frame based on the preset focus mode; or,
[0180] In response to a setting operation for the focus area of the video frame, a target position corresponding to the setting operation is used as the focus area of the video frame.
[0181] In a possible implementation manner, the initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize blur strength;
[0182] The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of a user.
[0183] In a possible implementation manner, the focus module 305, when performing focus processing on the to-be-processed video based on the target focus information, is configured to:
[0184] For any video frame, determine a focus area and a blur area based on the focus area in the target focus information of the video frame;
[0185] The blurred area is divided into a plurality of area blocks, and the blurred area is blurred according to a depth difference between each area block and the focus area.
[0186] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0187] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 4 , which is a schematic diagram of the structure of a computer device 400 provided in an embodiment of the present disclosure. The computer device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., or various forms of servers, such as independent servers or server clusters. Figure 4 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0188] like Figure 4 As shown, the computer device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only storage device (ROM) 402 or a program loaded from a storage device 408 to a random access storage device (RAM) 403. Various programs and data required for the operation of the computer device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0189] Typically, the following devices may be connected to the I / O interface 405: input devices 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 409. The communication devices 409 may allow the computer device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4The computer device 400 is shown with various devices, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.
[0190] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the video processing method. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0191] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the video processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.
[0192] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the video processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0193] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0194] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0195] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0196] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0197] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0198] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A video processing method, characterized in that: include: Get the video to be processed; Based on a preset focusing method, determining initial focusing information of the video to be processed; wherein the initial focusing information includes a focusing area in a video frame; In response to an editing operation on the focus area of the video frame, modifying the focus area of the video frame; Based on the revised focus area, the focus areas of the remaining video frames in the to-be-processed video are updated to obtain target focus information; Focus processing is performed on the video to be processed based on the target focus information.
2. The method according to claim 1, characterized in that The determining the initial focus information of the video to be processed based on a preset focus mode includes: Determine a video frame to be detected in the video to be processed, and determine a focus subject in the video frame to be detected based on the preset focusing method; Tracking the position information of the focus subject in the video to be processed, reusing the video frame where the tracking is interrupted as the video frame to be detected, and returning to the step of determining the focus subject; Based on the tracking results of each focused subject in the video to be processed, initial focus information of the video to be processed is determined.
3. The method according to claim 2, characterized in that The determining the focus subject in the to-be-detected video frame based on the preset focus mode includes: Based on a pre-trained subject detection model, detecting candidate subjects contained in the video frame to be detected; The focus subject is determined according to the priority order corresponding to each candidate subject.
4. The method according to claim 2, characterized in that: The determining the focus subject in the to-be-detected video frame based on the preset focus mode includes: According to the priority order of each preset subject, sequentially detecting whether the preset subject is contained in the video frame to be detected; The detected preset subject with the highest priority is used as the focus subject.
5. The method according to claim 2, characterized in that: The method further comprises: When it is detected that the video frame to be detected does not contain a focus subject based on the preset focus mode, a target focus depth of the video frame to be detected is determined, and an image position corresponding to the target focus depth is determined as a focus area in the initial focus information of the video frame to be detected; and The next video frame of the to-be-detected video frame is used again as the to-be-detected video frame, and the focus subject in the updated to-be-detected video frame is determined.
6. The method according to claim 5, characterized in that The determining the target focus depth of the to-be-detected video frame includes: Determine the central focus area of the video frame to be detected; The mean depth of the central focus area is used as the target focus depth.
7. The method according to claim 1, characterized in that The step of modifying the focus area of the video frame in response to the editing operation on the focus area of the video frame includes: In response to a deletion operation on the focus area of the video frame, the focus area of the video frame is modified based on the preset focus mode; or, In response to a setting operation for the focus area of the video frame, a target position corresponding to the setting operation is used as the focus area of the video frame.
8. The method according to claim 1, characterized in that The initial focus information further includes a blur pattern and / or a blur parameter; wherein the blur parameter is used to characterize the blur strength; The blur pattern and / or blur parameter included in the initial focus information is a preset value, or may be set in response to a setting operation of a user.
9. The method according to claim 1, characterized in that: The performing focus processing on the to-be-processed video based on the target focus information includes: For any video frame, determining a blurred area based on a focus area in target focus information of the video frame; The blurred area is divided into a plurality of area blocks, and the blurred area is blurred according to a depth difference between each area block and the focus area.
10. A video processing device, characterized in that: include: An acquisition module is used to acquire the video to be processed; A determination module, configured to determine initial focus information of the video to be processed based on a preset focus mode; wherein the initial focus information includes a focus area in each video frame; An adjustment module, configured to modify the focus area of the video frame in response to an editing operation on the focus area of the video frame; An updating module, configured to update the focus areas of the remaining video frames in the to-be-processed video based on the revised focus area, to obtain target focus information; A focusing module is used to perform focusing processing on the video to be processed based on the target focusing information.
11. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the video processing method according to any one of claims 1 to 9 are performed.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video processing method according to any one of claims 1 to 9 are executed.