Video processing method, video processing device, computer equipment, and storage medium

Abstract

This disclosure provides a video processing method and apparatus, a computer device, and a storage medium, wherein the method includes acquiring a video to be processed; determining initial focus information, including the focus area in each video frame of the video to be processed, based on a preset focusing scheme; modifying the focus area of ​​a video frame in response to an editing operation on the focus area of ​​the video frame; updating the focus areas of the remaining video frames in the video to be processed based on the modified focus area to obtain target focus information; and performing a focus process on the video to be processed based on the target focus information. In an embodiment of this disclosure, target focus information can be determined by performing an automatic focus process on the video, and then receiving a focus area editing command transmitted by the user, thereby enabling rapid focus processing on the video to be processed, and the focus processing result can meet user requirements.

Description

Technical Field

[0001] (Reference to Related Applications) This application claims the priority of Chinese Patent Application No. 202310975336.9 filed on August 3, 2023, and incorporates herein by reference in its entirety the content disclosed in the above-mentioned Chinese Patent Application as part of this application.

[0002] (Technical Field) This disclosure relates to a video processing method and apparatus, a computer device, and a storage medium.

Background Art

[0003] When shooting a video, in order to enhance the display effect of a specific subject in the video, it is necessary to perform focusing processing. Generally, for video shooting, it is necessary to use a device with high processing performance, such as a camera or a video camera, so as to obtain a relatively good focusing effect. However, in many cases, when shooting a video, one can only use a device with poor focusing performance, such as a mobile phone, and thus the focusing effect of the shot video becomes poor.

Summary of the Invention

[0004] Embodiments of this disclosure provide at least a video processing method, a video processing apparatus, a computer device, and a storage medium.

[0005] According to a first aspect, embodiments of this disclosure provide a video processing method, the method comprising: acquiring a video to be processed; determining initial focusing information including a focusing area in each video frame of the video to be processed based on a preset focusing method; modifying the focusing area of the video frame in response to an editing operation on the focusing area of the video frame of the frame; updating the focusing areas of the remaining video frames in the video to be processed based on the modified focusing area to obtain target focusing information; This includes performing a focus process on a video that should be processed based on target focus information.

[0006] In one possible embodiment, determining the initial focus information of the video to be processed based on a preset focusing scheme is: This involves determining the video frame to be detected in the video to be processed, and determining the subject of focus in the video frame to be detected based on a pre-set focusing scheme. This involves tracking the positional information of the subject in focus within the video to be processed, reusing video frames where tracking was interrupted as video frames to be detected, and returning to the execution of the subject determination step. This includes determining the initial focus information of the video to be processed based on the tracking results in the video to be processed for each focusing unit.

[0007] In one possible embodiment, determining the subject of focus in the video frame to be detected based on a preset focusing scheme is: Based on a pre-trained subject detection model, the system detects candidate subjects included in the video frames to be detected, and This includes determining the subject to focus on according to the priority order corresponding to each candidate subject.

[0008] In one possible embodiment, determining the subject of focus in the video frame to be detected based on a preset focusing scheme is: The process involves sequentially detecting whether a pre-defined entity is present in the video frame to be detected, according to the priority order of each pre-defined entity. This includes setting the pre-configured subject with the highest detected priority as the subject of focus.

[0009] In one possible embodiment, the method is Based on a pre-set focusing method, if it is detected that the video frame to be detected does not contain the subject of focus, the target depth of focus of the video frame to be detected is determined, and the image position corresponding to the target depth of focus is determined as the focused region in the initial focusing information of the video frame to be detected. This further includes reusing the video frame following the video frame to be detected as the video frame to be detected, and determining the subject of focus in the updated video frame to be detected.

[0010] In one possible embodiment, determining the target depth of focus of the video frame to be detected is: Determining the central focus region of the video frame to be detected, This includes setting the average depth of field of the central focus region as the target depth of field.

[0011] In one possible embodiment, redetermining the focus region of a video frame in response to an editing operation on the focus region of the video frame is: In response to a deletion operation on the in-focus area of ​​a video frame, the in-focus area of ​​the video frame is re-determined based on a preset focusing method, or This includes setting the target position corresponding to a setting operation as the focus region of the video frame in response to a setting operation on the focus region of the video frame.

[0012] In one possible embodiment, the initial focus information further includes a blur style and / or blur parameters, where the blur parameters are used to express the degree of blur. The blur style and / or blur parameters included in the initial focus information may be preset values ​​or may be set in response to user settings.

[0013] In one possible embodiment, performing a focus operation on a video to be processed based on target focus information is: For any one video frame, the in-focus area and blurred area are determined based on the in-focus area in the target focus information of the video frame. This includes dividing the blurred area into multiple region blocks and performing a blurring process on the blurred area based on the depth difference value between each region block and the in-focus area.

[0014] According to a second aspect, embodiments of the present disclosure further provide a video processing apparatus, which is: An acquisition module for obtaining the video to be processed, A determination module for determining initial focus information, including the focus area in each video frame of the video to be processed, based on a pre-set focusing method, An adjustment module for correcting the focus area of ​​a video frame in response to editing operations on the focus area of ​​the video frame, An update module for obtaining target focus information by updating the focus area of ​​the remaining video frames in the video to be processed based on the corrected focus area, It includes a focusing module for performing focusing on video that should be processed based on target focusing information.

[0015] In one possible embodiment, when the decision module determines the initial focus information of the video to be processed based on a preset focusing scheme, Determine the video frame to be detected in the video to be processed, and determine the focus subject in the video frame to be detected based on a pre-set focusing method. Track the location information of the subject in focus in the video to be processed, reuse the video frame where tracking was interrupted as the video frame to be detected, and return to the execution of the subject in focus determination step. This is used to determine the initial focus information of the video to be processed, based on the tracking results in the video to be processed by each focusing unit.

[0016] In one possible embodiment, when the determination module determines the in-focus subject in the video frame to be detected based on a preset focusing method, detect candidate subjects included in the video frame to be detected based on a pre-trained subject detection model, and uses it to determine the in-focus subject according to the priority corresponding to each candidate subject.

[0017] In one possible embodiment, when the determination module determines the in-focus subject in the video frame to be detected based on a preset focusing method, sequentially detect whether a preset subject is included in the video frame to be detected according to the priority of each preset subject, and uses the preset subject with the highest detected priority as the in-focus subject.

[0018] In one possible embodiment, the determination module further when it detects that the in-focus subject is not included in the video frame to be detected based on a preset focusing method, determines the target focusing depth of the video frame to be detected, and determines the image position corresponding to the target focusing depth as the focusing area in the initial focusing information of the video frame to be detected, reuses the next video frame of the video frame to be detected as the video frame to be detected, and uses it to determine the in-focus subject in the updated video frame to be detected.

[0019] In one possible embodiment, when the adjustment module determines the target focusing depth of the video frame to be detected, determines the central focusing area of the video frame to be detected, and uses the depth average value of the central focusing area as the target focusing depth.

[0020] In one possible embodiment, when the adjustment module modifies the focusing area of the video frame in response to an editing operation on the focusing area of the video frame, In response to a deletion operation on the in-focus area of ​​a video frame, the in-focus area of ​​the video frame is re-determined based on a preset focusing scheme, or This is used to set the target position corresponding to a setting operation on the video frame's focus region as the video frame's focus region, in response to a setting operation on the video frame's focus region.

[0021] In one possible embodiment, the initial focus information further includes a blur style and / or blur parameters, where the blur parameters are used to express the degree of blur. The blur style and / or blur parameters included in the initial focus information may be preset values ​​or may be set in response to user settings.

[0022] In one possible embodiment, when the focusing module performs focusing on a video to be processed based on target focusing information, For any one video frame, the in-focus area and blurred area are determined based on the in-focus area in the target focus information of the video frame. This method is used to divide the blurred area into multiple region blocks and apply blurring to the blurred area based on the depth difference value between each region block and the in-focus area.

[0023] According to a third aspect, embodiments of the present disclosure further provide a computer device comprising a processor, memory, and a bus, wherein the memory stores machine-readable instructions that the processor can execute, and when the computer device is in operation, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect or any one possible embodiment of the first aspect are performed.

[0024] According to a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the first aspect or any one possible embodiment of the first aspect are performed.

[0025] To make the above-mentioned objectives, features, and advantages of this disclosure clearer and easier to understand, preferred embodiments are given below and described in detail together with the drawings. [Brief explanation of the drawing]

[0026] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings that may be used in the embodiments are briefly described below. These drawings are incorporated herein and constitute part of this specification, and they illustrate embodiments consistent with this disclosure and are used together with the specification to illustrate the technical solutions of this disclosure. The following drawings show only a few embodiments of this disclosure and should not be considered a limitation of scope, and it will be understood that those skilled in the art can obtain other relevant drawings based on these drawings without any creative effort. [Figure 1] A flowchart of the video processing method provided by the embodiments of this disclosure is shown. [Figure 2] A flowchart illustrating a specific method for determining initial focus information in a video processing method provided by an embodiment of this disclosure is shown. [Figure 3] A schematic diagram of a video processing device provided by an embodiment of this disclosure is shown. [Figure 4] A schematic diagram of the configuration of the computer equipment provided by the embodiments of this disclosure is shown. [Modes for carrying out the invention]

[0027] To clarify the objectives, technical solutions, and advantages of the embodiments of this disclosure, the technical solutions in the embodiments of this disclosure will be described clearly and completely below with reference to the drawings of the embodiments of this disclosure. Clearly, the embodiments described are some, but not all, embodiments of this disclosure. Generally, the components of the embodiments of this disclosure generally described and shown herein can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but rather to show only selected embodiments of this disclosure. All other embodiments obtained based on the embodiments of this disclosure without creative effort by a person skilled in the art are all within the scope of protection of this disclosure.

[0028] Research shows that when shooting video, focusing is necessary to enhance the visual effect of specific subjects in the video. Generally, to achieve relatively good focusing, video shooting requires the use of equipment with high processing power, such as a camera or video camera. However, in many cases, when shooting video, only equipment with poor focusing performance, such as mobile phones, can be used, resulting in poor focusing in the recorded video.

[0029] Based on the above research, this disclosure provides a video processing method and apparatus, computer equipment and storage medium that can automatically determine the initial focus information of a video to be processed based on a preset focusing method after video recording is completed, thereby enabling automatic video focusing processing, and allowing the user to edit the focus area of ​​the video frame. The target focus information thus determined enables rapid focusing processing of the video to be processed, and the focusing processing result can satisfy the user's focusing needs.

[0030] Note that similar symbols and letters represent similar terms in the following diagrams; therefore, once a term is defined in one diagram, it is not necessary to further define and interpret it in subsequent diagrams.

[0031] In this specification, the terms "and / or" simply describe a relationship and indicate that there may be three possible relationships. For example, A and / or B may represent three cases: A alone, A and B as a combination, and B alone. In this specification, the terms "at least one" refer to any combination of one or at least two of a group. For example, including at least one of A, B, and C may mean including one or more elements selected from the set consisting of A, B, and C.

[0032] Before using any of the technical proposals disclosed in each embodiment of this disclosure, it is understood that the user should be informed in an appropriate manner, in accordance with applicable laws and regulations, of the type of personal information related to this disclosure, its scope of use, and usage scenarios, and permission should be obtained from the user.

[0033] For example, in response to receiving a voluntary request from the user, prompt information is sent to the user to clearly inform the user that the operation requested to be performed requires the acquisition and use of the user's personal information. This allows the user to choose whether or not to provide personal information to software or hardware such as electronic devices, application programs, servers, or storage media that perform the operation of the proposed technical method of this disclosure based on the prompt information.

[0034] As a selective but non-restrictive implementation, a method for sending prompt information to a user in response to receiving a voluntary request from the user may be, for example, a pop-up window, and the prompt information may be presented in text format within the pop-up window. Furthermore, the pop-up window may include a selection control that allows the user to choose whether to "agree" or "disagree" to providing personal information to the electronic device.

[0035] The above-described notification and acquisition process for user permission is illustrative and does not limit the forms in which this disclosure can be implemented. It should be understood that other methods that comply with applicable laws and regulations are also applicable to the implementation of this disclosure.

[0036] To facilitate understanding of this embodiment, the video processing method disclosed in the embodiments of this disclosure will first be described in detail. The entity that executes the video processing method provided in the embodiments of this disclosure is generally a computer device having a certain degree of computing power, which may be, for example, a terminal device or a server, or other processing device. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, mobile phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible embodiments, the video processing method can be implemented by calling computer-readable instructions stored in memory via a processor.

[0037] In the following, the video processing method provided by the embodiments of this disclosure will be described using the example that the implementing entity is a terminal device.

[0038] As shown in Figure 1, this is a flowchart of a video processing method provided by an embodiment of the present disclosure, the method comprising steps 101 to 105, where, Step 101: Obtain the video to be processed.

[0039] Step 102: The initial focus information of the video to be processed is determined based on a preset focusing method.

[0040] Step 103: In response to an editing operation on the in-focus area of ​​the video frame, the in-focus area of ​​the video frame is modified.

[0041] Step 104: Based on the corrected focus area, the focus area of ​​the remaining video frames in the video to be processed is updated to obtain target focus information.

[0042] Step 105: Focusing is performed on the video to be processed based on the target focus information.

[0043] The following is a detailed explanation of steps 101 to 105 above.

[0044] Regarding Step 101, The video to be processed may refer to the video to be subjected to focusing, and the videos in the video to be processed do not necessarily have to be in focus, or the video to be processed may include videos that have already been partially in focus at the time of shooting, or videos that have not been partially in focus at the time of shooting.

[0045] For example, if the length of the video to be processed is 10 minutes, and minutes 0-5 may have been manually focused during shooting, minutes 0-5 may already be focused video, and minutes 6-10 may not have been manually focused during shooting, then minutes 6-10 may not be focused video.

[0046] Optionally, the method employed in this disclosure may be used at the user end, in which case, when acquiring video to be processed, the user may call an image acquisition device in response to a shooting command to shoot video and obtain video to be processed.

[0047] Alternatively, when obtaining a video to be processed, the system may receive a video uploaded by the user and use the uploaded video as the video to be processed. Here, the video uploaded by the user may be a video uploaded by the user from their local machine, or a video on a network selected by the user.

[0048] Regarding step 102, Here, determining the initial focus information of the video to be processed based on the preset focusing method may also mean automatically determining the initial focus information of each video frame of the video to be processed based on the preset focusing method.

[0049] Here, the initial focus information may include the focus region in each video frame, and the initial focus information may further include the blur style and / or blur parameters.

[0050] The blur style is used to indicate the style used when blurring the video to be processed, and may include, for example, heart shapes, ovals, etc., and the blur styles of different video frames in the same video to be processed may be different.

[0051] The aforementioned blur parameter is used to express the degree of blurring; for example, a complete blur may be 100%, while a partial blur may be 10%, 20%, 50%, etc.

[0052] The blur style and / or blur parameters in the initial focus information may be set to default settings, manually based on user commands, or automatically based on the video display content.

[0053] When automatically setting the blur style and / or blur parameters based on the video content, the blur style and / or blur parameters may be set to match the characteristics of the video content. For example, if the video content includes animals, an elliptical blur style may be adopted and the blur level set to 70%, and if the video content includes a little girl, a heart-shaped blur style may be adopted and the blur level set to 20%.

[0054] In one possible embodiment, different blur styles and blur parameters may be set for different video frames, and multiple blur styles and blur parameters may be included for the same video frame.

[0055] Specifically, when manual settings are made based on user commands, the blur style and blur parameters may be set by one of the following two methods.

[0056] Method 1: Set the blur style and blur parameters based on time.

[0057] The user instruction includes blur styles and blur parameters set for different time periods of the video to be processed. For example, if the video to be processed is 20 minutes long, the blur style for minutes 0-15 may be set to circular and the blur parameter to 80%, while the blur style for minutes 16-20 may be set to diamond and the blur parameter to 50%.

[0058] Method 2: Set the blur style and blur parameters based on the focus subject.

[0059] The user command may include blur styles and blur parameters set for different focus subjects in the video to be processed. For example, if the focus subject in the video to be processed is A, the blur style may be set to a square and the blur level to 10%, and if the focus subject in the video to be processed is B, the blur style may be set to a heart shape and the blur level to 30%.

[0060] In one possible embodiment, the initial focus information may further include a focus switching method. Because a change may occur in the focus subject in the video to be processed, when a change occurs in the focus subject, the focus between adjacent video frames also changes accordingly, and the focus switching method refers to a method for switching the focus between adjacent video frames when a change occurs in the focus subject.

[0061] Optionally, the focus switching method in the initial focus information may be a default, pre-set value, or it may be set in response to a user setting command.

[0062] For example, the focus switching method may be a switching method that follows a certain speed and / or direction.

[0063] In one possible embodiment, when determining the initial focus information of the video to be processed based on a preset focusing scheme, the steps shown in Figure 2 may be employed as an example.

[0064] Step 201: For the video frames to be processed, the in-focus subject in the video frames to be detected is determined based on the preset focusing method.

[0065] Step 202: Track the position information of the focus subject in the video to be processed, reuse the video frame where the tracking was interrupted as the video frame to be detected, and return to the execution of Step 201.

[0066] Step 203: Based on the tracking results in the video to be processed for each focusing unit, the initial focus information of the video to be processed is determined.

[0067] In step 201, the subject in focus in the video frame to be detected may be, for example, an entity included in the video frame to be detected, such as a person, an animal, or several objects.

[0068] When optionally determining the subject of focus in the video frame to be detected, one of the following methods may be employed.

[0069] Method 1: Based on a pre-trained subject detection model, candidate subjects included in the video frame to be detected are detected, and the subject in focus is determined according to the priority order corresponding to each candidate subject.

[0070] When detecting candidate subjects included in the video frame to be detected based on a pre-trained subject detection model, the video frame to be detected may be input to the subject detection model, and the subject detection model may output all candidate subjects included in the video frame to be detected.

[0071] Here, the priority order corresponding to each candidate entity may be predetermined, or it may be set based on user instructions.

[0072] Specifically, after detecting candidate subjects included in the video frames to be detected based on the subject detection model, candidate subject identifiers may be displayed to the user, and then a priority order for each candidate subject may be determined in response to the user's sorting operation of the candidate subject identifiers.

[0073] In one possible embodiment, the priority order corresponding to each candidate subject may be automatically determined based on the video content.

[0074] Specifically, after detecting candidate entities included in the video frames to be detected, the duration of each candidate entity's appearance in the video frames to be detected may be determined, and then a priority order corresponding to each candidate entity may be determined based on the duration of each candidate entity's appearance in the video frames to be detected.

[0075] To illustrate, the longer the duration of a candidate subject's appearance in the video frame to be detected, the higher the priority given to that candidate subject, as this increases the proportion of the video frame that the candidate subject occupies.

[0076] In one selectable embodiment, when determining the focused subject according to the priority corresponding to each candidate subject, the candidate subject with the highest priority may be designated as the focused subject.

[0077] Method 2: In accordance with the priority order of each pre-set entity, the system sequentially detects whether or not the video frame to be detected contains the pre-set entity, and the pre-set entity with the highest detected priority is designated as the focused entity.

[0078] Here, the pre-configured entities differ from the candidate entities in Method 1, where the candidate entities in Method 1 refer to all entities included in the video frame to be detected that are detected from the video frame to be detected, whereas the pre-configured entities refer to all entities that are pre-configured and may be included before the video frame to be detected is detected.

[0079] Here, the predetermined priority order of the entities may be the same as the method for determining the priority order of the candidate entities, and the explanation is omitted here.

[0080] In one selectable embodiment, when determining the subject in focus, video frames to be detected are detected according to the priority order of each preset subject, and it is determined sequentially whether or not a preset subject is included in the video frame until a preset subject can be detected, and that preset subject may be set as the subject in focus.

[0081] For example, animal faces, human faces, animal torsos, and human torsos are pre-defined subjects, and their priority is given in the order of human faces, animal faces, human torsos, and animal torsos. When detecting a video frame to be detected, first it is checked whether or not a human face is contained in the video frame of that frame. If a human face is not detected, it continues to check whether or not an animal face is contained in the video frame of that frame. If an animal face is detected at this point, the animal face is set as the subject of focus. If an animal face is not detected, it continues to check whether or not a human torso is contained in the video frame of that frame, and this process is continued until a pre-defined subject is detected.

[0082] In one possible embodiment, if multiple subjects of the same priority are detected in the video frame to be detected, the subject with the largest area may be selected as the subject in focus.

[0083] If it is detected that the video frame to be detected does not contain the subject of focus, the target depth of focus of the video frame to be detected may be determined, the image position corresponding to the target depth of focus may be determined as the focused region in the initial focus information of the video frame to be detected, the next video frame to be detected may be reused as the video frame to be detected, and the subject of focus in the updated video frame to be detected may be determined.

[0084] Here, focusing on a certain position / region should be explained as focusing on the depth corresponding to that position / region. When determining the target depth of focus for the video frame to be detected, the central region of the depth diagram corresponding to the video frame to be detected may be determined first, and the average depth of each pixel point in the central region may be used as the target depth of focus.

[0085] Here, the central region of the depth diagram may be, for example, a region of the target size with the center of the depth diagram as its center.

[0086] To improve the focusing speed, it is possible to perform focus subject detection only on the video frames to be detected within the video frames to be detected, without performing focus subject detection on each video frame within the video frames to be detected. Alternatively, a subject tracking method may be used to determine the focus on the video frames that are not to be detected within the video frames to be detected.

[0087] For example, if the subject of focus determined by the first video frame to be detected (let's call it the m-th video frame) is the face of user A, then subject tracking may be used to position user A's face and determine the focus area in the time from the m-th video frame to the m+n-th video frame. In the m+n+1-th video frame, subject tracking is interrupted and it is not possible to position user A's face, so the m+n+1-th video frame may be used as the video frame to be detected, and the subject of focus in the m+n+1-th video frame may be re-determined.

[0088] The tracking result in the video to be processed for each focusing unit may refer to the position information in each video frame of the video to be processed for each focusing unit. When determining the initial focus information of the video to be processed based on the tracking result in the video to be processed for each focusing unit, the center of the position information in each video frame of the focusing unit may be used as the focus region in the initial focus information, and the initial focus information may be obtained by combining this with the blur style and / or blur parameters determined based on the above method.

[0089] Regarding steps 103 and 104, When responding to an editing operation on the in-focus region of the video frame, and when re-determining the in-focus region of the video frame, the following two cases are included:

[0090] Case 1: In response to a deletion operation on the in-focus region of the video frame, the in-focus region of the video frame is re-determined based on the preset focusing method.

[0091] Specifically, when automatically detecting the focus of a video frame of the video to be processed based on the method described above, there is a possibility of misdetection of the in-focus subject. Therefore, in response to a deletion operation, the video frame may be reused as a video frame to be detected, and the in-focus subject may be redetected. This allows for rapid correction of focusing errors and improvement of the focusing effect.

[0092] Case 2: In response to a setting operation for the focus region of the video frame, the target position corresponding to the setting operation is set as the focus region of the video frame.

[0093] For example, if a subject that the user wishes to display prominently appears in the video, but that subject is not designated as the focused subject by the detection described above, the system may detect the user's setting operation for the focus area and then set the subject corresponding to that operation as the focused subject, thereby enabling the user to personalize the focus setting.

[0094] If a change occurs in the focus area of ​​any one video frame (here, this means that the focus area of ​​that video frame has been re-determined based on the method described above), the focus areas of the remaining video frames should also change accordingly. Therefore, if a change occurs in the focus area of ​​any one video frame, the focus subject may be determined based on the focus area of ​​that video frame, subject tracking may be performed on the video frames after that video frame, and the focus areas of the video frames after that video frame may be updated based on the tracking results.

[0095] Alternatively, if a change occurs in the focus area of ​​any one video frame, subject tracking may be performed on both the video frame before and after that video frame, and the focus areas of the video frames before and after that video frame may be updated based on the tracking results.

[0096] Regarding Step 105, When performing a focus process on a video to be processed based on the target focus information, for example, for any one video frame, the focus region and the blur region may be determined first based on the focus region in the target focus information of that video frame, the blur region may be divided into multiple region blocks, and the blur process may be performed on the blur region based on the depth difference value between each region block and the focus region.

[0097] The in-focus region may be a region where the corresponding depth is the same as the depth of the in-focus region. For example, if the in-focus region is a point on a person's face, the in-focus region may be the entire face region, since the depth of the entire face is the same.

[0098] In the aforementioned video frame, all areas other than the in-focus area may be blurred areas. In practical applications, if all areas are blurred to the same degree, the in-focus area may be excessively emphasized, potentially affecting the final display effect. Therefore, blurring may be performed on the blurred areas based on the depth difference value between each part of the blurred area and the in-focus area.

[0099] For example, dividing the blurred area into multiple area blocks may also mean dividing the blurred area according to a predetermined size, and the area of ​​the divided area blocks may be the same. When blurring the blurred area based on the depth difference value between each area block and the in-focus area, the degree of blurring corresponding to the area block may be determined first based on the average depth value of the in-focus area and the average depth value of each area block, and then based on the difference value before the average depth value and a predetermined proportionality coefficient, and the blurring process may be performed according to the degree of blurring.

[0100] Here, the larger the difference between the average depth values, the greater the degree of blurring corresponding to the region block may be.

[0101] In one possible embodiment, after performing a focus process on the video to be processed based on the target blur information, the target blur information may be saved as a template, and when processing other videos, the processing may be performed directly based on the target blur information.

[0102] Here, when saving the target blur information as a template, the in-focus area of ​​other videos and the in-focus area of ​​the video to be processed are always different. Therefore, the template does not need to include the in-focus area in the target blur information, but may include, for example, one or more of the following: priority of candidate subjects, priority of pre-set subjects, blur style, blur parameters, and switching method between focus.

[0103] Those skilled in the art will understand that in the above-described method of a specific embodiment, the order in which each step is created does not constitute any limitation on the implementation process in a strict execution order, but rather the specific execution order of each step should be determined by its function and possible inherent logic.

[0104] Based on the same concept of the invention, embodiments of this disclosure also provide a video processing apparatus corresponding to a video processing method, and since the principle by which the apparatus in embodiments of this disclosure solves the problem is similar to that of the video processing method in embodiments of this disclosure, the implementation of the apparatus can refer to the implementation of the method, and the overlaps will no longer be explained.

[0105] As shown in Figure 3, this is a schematic diagram of the architecture of a video processing device provided by an embodiment of the present disclosure, the device including an acquisition module 301, a determination module 302, an adjustment module 303, an update module 304, and a focusing module 305, where, The acquisition module 301 is used to acquire the video to be processed. The determination module 302 is used to determine initial focus information, including the focus region in each video frame of the video to be processed, based on a preset focusing method. The adjustment module 303 is used to correct the in-focus area of ​​the video frame in response to editing operations on the in-focus area of ​​the video frame. The update module 304 is used to obtain target focus information by updating the focus area of ​​the remaining video frames in the video to be processed based on the corrected focus area. The focusing module 305 is used to perform focusing on the video to be processed based on the target focusing information.

[0106] In one possible embodiment, when the determination module 302 determines the initial focus information of the video to be processed based on a preset focusing scheme, Determine the video frame to be detected in the video to be processed, and determine the focus subject in the video frame to be detected based on the preset focusing method. The position information of the subject to focus is tracked in the video to be processed, the video frame where the tracking was interrupted is reused as the video frame to be detected, and the execution of the subject to focus determination step is returned. This is used to determine the initial focus information of the video to be processed, based on the tracking results of each focusing unit in the video to be processed.

[0107] In one possible embodiment, when the determination module 302 determines the focus subject in the video frame to be detected based on the preset focusing scheme, Based on a pre-trained subject detection model, candidate subjects included in the video frame to be detected are detected. This is used to determine the subject that is in focus, according to the priority order corresponding to each candidate subject.

[0108] In one possible embodiment, when the determination module 302 determines the focus subject in the video frame to be detected based on the preset focusing scheme, In accordance with the priority order of each pre-configured entity, the system sequentially detects whether or not the video frame to be detected contains the pre-configured entity. This is used to select the pre-configured subject with the highest detected priority as the subject in focus.

[0109] In one possible embodiment, the decision module 302 further, Based on the pre-set focusing method, if it is detected that the video frame to be detected does not contain the subject of focus, the target depth of focus of the video frame to be detected is determined, and the image position corresponding to the target depth of focus is determined as the focused region in the initial focusing information of the video frame to be detected. The video frame following the video frame to be detected is reused as the video frame to be detected, and this is used to determine the subject of focus in the updated video frame to be detected.

[0110] In one possible embodiment, when the adjustment module 303 determines the target depth of focus of the video frame to be detected, The central focus region of the video frame to be detected is determined, The average depth value of the central focus region is used to determine the target focus depth.

[0111] In one possible embodiment, when the adjustment module 303 re-determines the focus region of the video frame in response to an editing operation on the focus region of the video frame, In response to a deletion operation on the in-focus region of the video frame, the in-focus region of the video frame is re-determined based on the preset focusing method, or In response to a setting operation on the in-focus region of the video frame, it is used to set the target position corresponding to the setting operation as the in-focus region of the video frame.

[0112] In one possible embodiment, the initial focus information further includes a blur style and / or blur parameters, where the blur parameters are used to express the degree of blur. The blur style and / or blur parameters included in the initial focus information may be preset values ​​or may be set in response to user settings.

[0113] In one possible embodiment, when the focusing module 305 performs focusing on the video to be processed based on the target focusing information, For any one video frame, the in-focus area and the blurred area are determined based on the in-focus area in the target focus information of that video frame. This method is used to divide the blurred area into multiple region blocks and to perform blurring on the blurred area based on the depth difference value between each region block and the in-focus area.

[0114] The processing flow of each module within the device, and the interaction flow between each module, can be described by referring to the relevant explanations in the embodiments of the above method, and will not be explained in detail here.

[0115] Based on the same technical concept, embodiments of the present disclosure further provide computer equipment. Figure 4 is a schematic diagram of the configuration of computer equipment 400 provided by embodiments of the present disclosure. Computer equipment in embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers, or various forms of servers, such as standalone servers or server clusters. The computer equipment shown in Figure 4 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.

[0116] As shown in Figure 4, the computer device 400 may include a processing unit (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 402 or programs loaded from storage device 408 into random access storage device (RAM) 403. The RAM 403 further stores various programs and data necessary for the operation of the computer device 400. The processing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0117] Generally, devices such as input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, and gyroscopes; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, and vibrators; storage devices 408 including, for example, magnetic tape and hard disks; and communication devices 409 can be connected to the I / O interface 405. The communication device 409 can allow the computer equipment 400 and other devices to exchange data via wireless or wired communication. Figure 4 shows computer equipment 400 with various devices, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have all of the devices shown instead.

[0118] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product which includes a computer program carried on a computer-readable medium, the computer program which includes program code for executing a video processing method. In such embodiments, the computer program may be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, the above functions, limited by the methods of embodiments of the present disclosure, are performed.

[0119] Embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the video processing method described in the embodiment of the above method are performed. Here, the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0120] Embodiments of the present disclosure further provide a computer program product on which program code is contained, and instructions contained in the program code may be used to perform the steps of the video processing method described in the embodiment of the above method, specifically refer to the embodiment of the above method, which is not described herein.

[0121] Here, the above-mentioned computer program product can be specifically implemented by hardware, software, or a combination thereof. In one selectable embodiment, the computer program product is specifically implemented as a computer storage medium, and in another selectable embodiment, the computer program product is specifically implemented as a software product such as a Software Development Kit (SDK).

[0122] As will be apparent to those skilled in the art, for the sake of clarity and conciseness, the specific operating processes of the systems and apparatus described above can be referenced to the corresponding processes in the embodiments of the methods described above and will not be repeated here. It should be understood that in some embodiments provided by this disclosure, the disclosed systems, apparatus, and methods can be implemented in other ways. The embodiments of the apparatus described above are merely illustrative, and for example, the division of the units is merely a division of logical functions, and there may be other methods of division in actual implementation, for example, multiple units or components may be combined, or integrated into another system, or some features may be ignored or not implemented. Also, the combinations, direct combinations, or communication connections between them that are shown or discussed may be indirect combinations or communication connections via some communication interfaces, devices, or units, and may be electrical, mechanical, or otherwise.

[0123] The units shown as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Depending on the actual needs, some or all of these units can be selected to achieve the objectives of the technical proposal of this embodiment.

[0124] Each functional unit in each embodiment of the present disclosure may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit.

[0125] If the aforementioned functions are implemented in the form of a software function unit and sold or used as an independent product, they can be stored in a processor-executable non-volatile computer-readable storage medium. Based on this understanding, any part of the proposed technology of this disclosure that is essential to or contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium containing several instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of this disclosure. The storage medium includes a variety of media capable of storing program code, such as USB memory, removable hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] Finally, the embodiments described above are merely specific embodiments of the Disclosure and are used to illustrate the technical ideas of the Disclosure, not to limit them, and the scope of protection of the Disclosure is not limited thereto. However, while the Disclosure has been described in detail with reference to the embodiments, those skilled in the art should understand that within the scope of the art disclosed, they can still easily modify, change, or substitute some of the technical features described in the embodiments, and such modifications, changes, or substitutions should be included within the scope of protection of the Disclosure without causing the essence of the corresponding technical idea to deviate from the spirit and scope of the technical idea of ​​the embodiments of the Disclosure. Therefore, the scope of protection of the Disclosure should be the same as the claims.

Claims

1. A video processing method, To obtain the video to be processed, Based on a pre-set focusing method, the initial focusing information, including the focused region in the video frame of the video to be processed, is determined. In response to an editing operation on the in-focus region of the video frame, the in-focus region of the video frame is modified. Based on the corrected focus area, the focus area of ​​the remaining video frames in the video to be processed is updated to obtain target focus information. A video processing method comprising performing a focus process on the video to be processed based on the target focus information.

2. Determining the initial focus information of the video to be processed based on the aforementioned preset focusing method is: The process involves determining the video frame to be detected in the video to be processed, and determining the focus subject in the video frame to be detected based on the pre-set focusing method. The process involves tracking the positional information of the subject to focus in the video to be processed, reusing the video frame where tracking was interrupted as the video frame to be detected, and returning to the execution of the subject to focus determination step. The method according to claim 1, further comprising determining initial focus information for the video to be processed based on the tracking results in the video to be processed for each focusing unit.

3. Determining the subject of focus in the video frame to be detected based on the aforementioned preset focusing method is: Based on a pre-trained subject detection model, the system detects candidate subjects included in the video frames to be detected. The method according to claim 2, comprising determining the focused subject in accordance with the priority order corresponding to each candidate subject.

4. Determining the subject of focus in the video frame to be detected based on the aforementioned preset focusing method is: In accordance with the priority order of each pre-configured entity, the system sequentially detects whether or not the video frame to be detected contains the pre-configured entity, The method according to claim 2, further comprising setting the pre-set subject with the highest detected priority as the subject in focus.

5. Based on the aforementioned pre-set focusing method, if it is detected that the video frame to be detected does not contain the subject of focus, the target depth of focus of the video frame to be detected is determined, and the image position corresponding to the target depth of focus is determined as the focused region in the initial focusing information of the video frame to be detected. The method according to any one of claims 2 to 4, further comprising reusing the next video frame to be detected as the video frame to be detected, and determining the subject of focus in the updated video frame to be detected.

6. Determining the target depth of focus of the video frame to be detected, Determining the central focus region of the video frame to be detected, The method according to claim 5, further comprising setting the average depth of the central focus region to the target focus depth.

7. Modifying the in-focus region of a video frame in response to an editing operation on the in-focus region of the video frame is: In response to a deletion operation on the in-focus region of the video frame, the in-focus region of the video frame is modified based on the preset focusing method, or The method according to any one of claims 1 to 6, comprising setting a target position corresponding to a setting operation for the in-focus region of a video frame as the in-focus region of the video frame in response to the setting operation.

8. The initial focus information further includes a blur style and / or blur parameters, where the blur parameters are used to express the degree of blur. The method according to any one of claims 1 to 7, wherein the blur style and / or blur parameters included in the initial focus information may be preset values ​​or set in response to user settings.

9. Performing a focus operation on the video to be processed based on the target focus information is: For any one video frame, the blurred area is determined based on the focus area in the target focus information of that video frame. The method according to any one of claims 1 to 8, further comprising dividing the blurred area into a plurality of area blocks and performing a blurring process on the blurred area based on the depth difference value between each area block and the in-focus area.

10. A video processing device, An acquisition module configured to acquire the video to be processed, A determination module configured to determine initial focus information, including the focus region in each video frame of the video to be processed, based on a pre-set focusing method, An adjustment module configured to modify the in-focus region of a video frame in response to an editing operation on the in-focus region of the video frame, An update module configured to obtain target focus information by updating the focus area of ​​the remaining video frames in the video to be processed based on the corrected focus area, A video processing apparatus including a focusing module configured to perform focusing on the video to be processed based on the target focusing information.

11. A computer device comprising a processor, memory, and a bus, wherein the memory stores machine-readable instructions that the processor can execute, and when the computer device is in operation, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the computer device performs the steps of the video processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the video processing method described in any one of claims 1 to 9 are performed.

Images