Image processing method, apparatus, computer device and storage medium

Abstract

Provided in the present disclosure are an image processing method, an apparatus, a computer device and a storage medium. The method comprises: acquiring an image to be processed, and determining a focusing area of said image; for pixel points in other areas of said image except the focusing area, determining a blurring window corresponding to the pixel points on the basis of depth difference information between the pixel points and the focusing area; and, on the basis of pixel values of the pixel points in the blurring window, determining pixel values after blurring processing corresponding to the pixel points.

Description

Image processing method, device, computer equipment and storage medium

[0001] This application claims priority to Chinese patent application No. 202310975334.X filed on August 3, 2023, the entire text of which is incorporated herein by reference as a part of this application. Technical Field

[0002] The present disclosure relates to an image processing method, apparatus, computer equipment, and storage medium. Background Art

[0003] To highlight the subject in an image or video, focusing is required during the capture process, placing high demands on both camera performance and the photographer. Taking image focusing as an example, focusing typically involves determining the subject in focus and then blurring the remaining areas outside of the focused area. The way this blurring is performed will impact the final image presentation, making image blurring a crucial step in image processing.

[0004] Summary of the Invention

[0005] The embodiments of the present disclosure at least provide an image processing method, apparatus, computer equipment, and storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising:

[0007] Acquire an image to be processed, and determine a focus area of ​​the image to be processed;

[0008] For a pixel point in an area other than the focused area in the image to be processed, determining a defocus window corresponding to the pixel point based on depth difference information between the pixel point and the focused area; wherein a shape corresponding to the defocus window is related to a target defocus style;

[0009] Based on the pixel value of each pixel point in the blur window, the pixel value after blurring corresponding to the pixel point is determined.

[0010] In a possible implementation, determining the focus area of ​​the image to be processed includes:

[0011] Determining a focused subject in the image to be processed according to entity priority;

[0012] The area where the focus subject is located is used as the focus area.

[0013] In a possible implementation, a method for acquiring the depth difference information between the pixel point and the focus area includes:

[0014] Determining a mean depth of pixels in the focus area;

[0015] The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.

[0016] In one possible implementation, for a pixel in an area other than the focused area in the image to be processed, determining a blur window corresponding to the pixel based on depth difference information between the pixel and the focused area includes:

[0017] Determining a target shape corresponding to the defocused window; wherein the target shape corresponding to the defocused window is related to a defocused style;

[0018] Determining a target size of the blur window based on the depth difference information;

[0019] For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.

[0020] In a possible implementation, determining the target shape corresponding to the virtual window includes:

[0021] In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or

[0022] Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or,

[0023] Determining the image style of the image to be processed, and determining a target shape corresponding to the image style; or,

[0024] Determine the regional characteristics of the focus area, and determine the target shape corresponding to the regional characteristics.

[0025] In a possible implementation, determining a target size of the blur window based on the depth difference information includes:

[0026] Determining a target size of the blur window based on the depth difference information and a preset scaling ratio;

[0027] The depth difference information is positively correlated with the target size of the blur window.

[0028] In a possible implementation, determining the pixel value after blurring corresponding to each pixel point based on the pixel value of the pixel point within the blurring window includes:

[0029] Determining the brightness value of each pixel in the blur window;

[0030] Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.

[0031] In a possible implementation, determining the pixel value after blurring corresponding to each pixel point based on the pixel value and brightness value of each pixel point in the blurring window includes:

[0032] Determining a weight of each pixel in the defocused window based on a brightness value of each pixel in the defocused window;

[0033] The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.

[0034] In a second aspect, an embodiment of the present disclosure further provides an image processing device, including:

[0035] An acquisition module, configured to acquire an image to be processed and determine a focus area of ​​the image to be processed;

[0036] a determination module configured to determine, for a pixel in an area other than the focused area in the image to be processed, a defocused window corresponding to the pixel based on depth difference information between the pixel and the focused area; wherein a shape corresponding to the defocused window is related to a target defocused style;

[0037] The blur module is used to determine the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blur window.

[0038] In a possible implementation, the acquisition module, when determining the focus area of ​​the image to be processed, is configured to:

[0039] Determining a focused subject in the image to be processed according to entity priority;

[0040] The area where the focus subject is located is used as the focus area.

[0041] In a possible implementation, the determining module is further configured to determine depth difference information between the pixel point and the focus area according to the following method:

[0042] Determining a mean depth of pixels in the focus area;

[0043] The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.

[0044] In one possible implementation, for a pixel in an area other than the focused area in the image to be processed, the determination module, when determining the blur window corresponding to the pixel based on depth difference information between the pixel and the focused area, is configured to:

[0045] Determining a target shape corresponding to the defocused window; wherein the target shape corresponding to the defocused window is related to a defocused style;

[0046] Determining a target size of the blur window based on the depth difference information;

[0047] For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.

[0048] In a possible implementation, the determining module, when determining the target shape corresponding to the virtual window, is configured to:

[0049] In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or

[0050] Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or,

[0051] Determining the image style of the image to be processed, and determining a target shape corresponding to the image style; or,

[0052] Determine the regional characteristics of the focus area and determine the target shape corresponding to the regional characteristics.

[0053] In a possible implementation, the determining module, when determining the target size of the blur window based on the depth difference information, is configured to:

[0054] Determining a target size of the blur window based on the depth difference information and a preset scaling ratio;

[0055] The depth difference information is positively correlated with the target size of the blur window.

[0056] In one possible implementation, the blur module, when determining the blurred pixel value corresponding to each pixel point based on the pixel value of the pixel point in the blur window, is configured to:

[0057] Determining the brightness value of each pixel in the blur window;

[0058] Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.

[0059] In one possible implementation, the blur module, when determining the blurred pixel value corresponding to each pixel point in the blur window based on the pixel value and brightness value of the pixel point, is configured to:

[0060] Determining a weight of each pixel in the defocused window based on a brightness value of each pixel in the defocused window;

[0061] The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.

[0062] In a third aspect, an embodiment of the present disclosure further provides a computer device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are performed.

[0063] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0064] In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, optional embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0066] FIG1 shows a flow chart of an image processing method provided by an embodiment of the present disclosure;

[0067] FIG2 shows a schematic diagram of a blurred window in the image processing method provided by an embodiment of the present disclosure;

[0068] FIG3 shows a schematic diagram of an image processing device provided by an embodiment of the present disclosure;

[0069] FIG4 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0071] Research has found that images or videos require focus processing during or after capture to highlight the subject. Taking image focus processing as an example, this process typically involves determining the focused subject within the image and then blurring the remaining areas outside of the focused area. This blurring is typically performed using fixed blur parameters, resulting in a poor blur effect.

[0072] Based on the above research, the present disclosure provides an image processing method, apparatus, computer equipment, and storage medium. When blurring an image to be processed, for any pixel in the area to be blurred, the blur window corresponding to the pixel can be determined based on the depth difference information between the pixel and the focused area. Then, based on the pixel values ​​of each pixel within the blur window, the pixel value corresponding to the pixel after blurring is determined. In the above method, when blurring is performed, the depth difference information between each pixel and the focused area is combined, thereby enabling different degrees of blurring to be performed on different depth areas, thereby improving the blurring effect.

[0073] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0074] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0075] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0076] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0077] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0078] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0079] To facilitate understanding of this embodiment, an image processing method disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the image processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities. The computer device includes, for example, a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, the image processing method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0080] 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure. The method includes steps 101 to 103, wherein:

[0081] Step 101: Acquire an image to be processed and determine a focus area of ​​the image to be processed.

[0082] Step 102: for a pixel point in an area other than the focused area in the image to be processed, determine a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focused area.

[0083] Step 103: Based on the pixel value of each pixel point in the blur window, determine the pixel value after blur processing corresponding to the pixel point.

[0084] The following are detailed instructions for the above steps.

[0085] For step 101,

[0086] The image to be processed may be an image that requires focus processing. For example, the image to be processed may not have been focused when it was captured, i.e., the image to be processed does not contain a focus point. The focus area of ​​the image to be processed does not refer to the area of ​​the image to be processed that is currently in focus, but rather the area to be focused.

[0087] In a possible implementation, when determining the focus area of ​​the image to be processed, the focus subject in the image to be processed may be determined according to entity priority, and then the area where the focus subject is located may be determined as the focus area.

[0088] Specifically, when determining the focused subject in the image to be processed, any one of the following methods may be used:

[0089] Method 1: Based on a pre-trained subject detection model, the candidate subjects contained in the image to be processed are detected, and then the focused subject is determined according to the priority order corresponding to each candidate subject.

[0090] When detecting the candidate subjects contained in the image to be processed based on a pre-trained subject detection model, exemplarily, the image to be processed can be input into the subject detection model, and the subject detection model can output all the candidate subjects contained in the image to be processed.

[0091] The priority order corresponding to each candidate entity (which can be considered as the entity priority) can be preset, or can be set based on a user instruction.

[0092] Specifically, after detecting the candidate subjects contained in the image to be processed based on the subject detection model, the candidate subject identifiers can be displayed to the user, and then the priority order corresponding to each candidate subject can be determined in response to the user's sorting operation on the candidate subject identifiers.

[0093] In another possible implementation, the priority order corresponding to the candidate subjects may also be automatically determined based on image content.

[0094] Specifically, after detecting the candidate subjects contained in the image to be processed, the area occupied by each candidate subject in the image to be processed can be determined respectively, and then the priority order corresponding to each candidate subject can be determined based on the area occupied by each candidate subject in the image to be processed.

[0095] Exemplarily, if the area occupied by the candidate subject in the image to be processed is larger, it means that the proportion of the candidate subject in the image to be processed is larger, and therefore, the priority of the candidate subject is higher.

[0096] In an optional implementation, when determining the focus subject according to the priority order corresponding to each candidate subject, the candidate subject with the highest priority may be used as the focus subject.

[0097] Method 2: According to the priority order of each preset subject, the image to be processed is sequentially detected to see whether the preset subject is included, and the detected preset subject with the highest priority is used as the focus subject.

[0098] Here, the preset subject is different from the candidate subject in the above-mentioned method one. The candidate subject in method one refers to all subjects detected from the image to be processed and contained in the image to be processed, while here, the preset subject refers to all possible subjects pre-set before detecting the image to be processed.

[0099] The priority order of the preset subjects (which can be considered as the entity priority) can be determined in the same way as the priority order of the candidate subjects, which will not be described in detail here.

[0100] In an optional embodiment, when determining the focus subject, the image to be processed can be detected according to the priority ranking of each preset subject, and it can be determined in turn whether the image to be processed contains the preset subject until a preset subject can be detected, and then the preset subject is used as the focus subject.

[0101] For example, animal faces, human faces, animal torsos, and human torsos are used as preset subjects, and are prioritized in the order of human faces, animal faces, human torsos, and animal torsos. When detecting an image to be processed, the image to be processed is first detected to see if it contains a human face. If no human face is detected, the image to be processed is then detected to see if it contains an animal face. If an animal face is detected, the animal face is used as the focus subject. If no animal face is detected, the image to be processed is then detected to see if it contains a human torso, and so on, until a preset subject is detected.

[0102] In a possible implementation, when multiple subjects with the same priority are detected in the image to be processed, the subject with the largest area may be selected as the focused subject.

[0103] For step 102,

[0104] In the image to be processed, all areas except the focus area are areas that need to be blurred, but the specific blurring strength may be different. Here, the blurring strength may refer to the size of the blurring range, that is, the size of the blurring window.

[0105] In one possible implementation, when determining the depth difference information between the pixel point and the focus area, the depth mean of the pixel points in the focus area can be determined first, and then the difference between the depth value of any pixel point and the depth mean can be used as the depth difference information between the pixel point and the focus area.

[0106] Here, the depth value of each pixel in the image to be processed can be determined from a depth image corresponding to the image to be processed. The depth image corresponding to the image to be processed can be determined after performing a depth analysis on the image to be processed. The specific method for determining the depth value of the pixel in the image to be processed is not limited in this disclosure.

[0107] The blur window corresponding to each pixel point is a window used to blur the pixel point. The shape of the blur window is related to the target blur style. The target blur style may refer to the blur style corresponding to the image to be processed. The size of the blur window is related to the depth difference information between the pixel point and the focus area. The position of the blur window is related to the position of the pixel point.

[0108] Determining the blur window corresponding to the pixel point based on the depth difference information between the pixel point and the focus area can be understood as determining the size of the blur window corresponding to the pixel point based on the depth difference information between the pixel point and the focus area.

[0109] Specifically, for any pixel in an area other than the focused area in the image to be processed, when determining the blur window corresponding to the pixel based on the depth difference information between the pixel and the focused area, the following steps may be performed:

[0110] Step a1: determining a target shape corresponding to the defocused window; wherein the shape corresponding to the defocused window is related to a target defocused style;

[0111] Step a2: determining a target size of the blur window based on the depth difference information;

[0112] Step a3: for any pixel point, determine a blur window with the target shape and target size centered on the pixel point.

[0113] The blur style is used to represent the style of blurring the image to be processed, and may include, for example, a heart shape, an ellipse, etc. When determining the target shape corresponding to the blur window, any one of the following methods may be used:

[0114] Method 1: In response to a setting operation of a target blur pattern for the image to be processed, a target shape corresponding to the target blur pattern set by the setting operation is determined.

[0115] Optionally, the operation of setting a target blur style for the image to be processed may include an operation of setting target blur styles for different to-be-blurred areas of the image to be processed, and different target blur styles may be set for different to-be-blurred areas.

[0116] Method 2: Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type.

[0117] Different entity types can be set with different target shapes to achieve personalized blur styles. When the entity where the pixel point is located is empty, that is, the pixel point to be blurred is not a pixel point on any entity, the default shape can be set as the target shape.

[0118] In this embodiment, when the area to be blurred of the image to be processed includes entities of multiple entity types, there may be multiple target blurring styles corresponding to the area to be blurred of the image to be processed.

[0119] Method 3: Determine the image style of the image to be processed, and determine a target shape corresponding to the image style.

[0120] Optionally, when determining the image style of the image to be processed, the image to be processed may be exemplarily input into a pre-trained style detection network to determine the image style of the image to be processed.

[0121] When determining the target shape corresponding to the image style, it can be determined based on the predetermined correspondence between the image style and the blur window shape; the correspondence can be pre-set or determined based on the blur processing process of multiple images to be processed that have been historically executed.

[0122] For example, the corresponding relationship can be determined by collecting statistics on the blur window shapes of the images to be processed that have been blurred in the past. Here, the blur window shapes of the images to be processed that have been blurred in the past can be set by a target group, such as professional groups such as graphic designers.

[0123] Method 4: Determine regional features of the focus area, and determine a target shape corresponding to the regional features.

[0124] Here, the regional features of the focus area can be extracted based on a feature extraction network. When determining the target shape corresponding to the regional features, for example, the shape features of each preset shape can be extracted, and then the similarity between the regional features and the shape features can be calculated, and the shape with the highest similarity can be used as the target shape.

[0125] Method 5: Determine a focused subject in the image to be processed, and determine a target shape corresponding to the focused subject.

[0126] This allows you to set different target shapes for different focus subjects. For example, if the focus subject is an animal, you can use an elliptical blur pattern; if the focus subject is a child, you can use a heart-shaped blur pattern. This way, you can enrich the blur patterns and improve the blur effect.

[0127] In step a2, when determining the target size of the defocused window based on the depth difference information, the target size of the defocused window can be determined based on the depth difference information and a preset scaling ratio; wherein the depth difference information is positively correlated with the target size of the defocused window.

[0128] Here, the reason why the depth difference information is positively correlated with the target size of the blur window is that if the depth difference information is larger, it means that the entity corresponding to the pixel point is farther away from the focus subject corresponding to the focus area, and the information contained in the entity farther away from the focus subject is generally useless information. Therefore, a large-size blur window can be used to blur the pixel point.

[0129] The larger the size of the blur window, the more pixels it contains. In this way, for any pixel to be blurred, when blurring the pixel based on the pixel value of the pixel within the blur window corresponding to the pixel, the greater the difference between the pixel value of the pixel after blurring and the pixel value of the pixel before blurring, the deeper the degree of blurring.

[0130] In step a3, the position of the blur window in the image to be processed is related to the position of the pixel to be blurred. For example, the target size of the blur window corresponding to pixel 1 is 3*3 pixels, the shape of the blur window is a square, and the position of the blur window is shown in Figure 2; the target size of the blur window corresponding to pixel 2 is 5*5 pixels, the shape of the blur window is a square, and the position of the blur window is shown in Figure 2.

[0131] For step 103,

[0132] The pixel points within the virtual window include the pixel points to be virtualized, that is, the pixel points corresponding to the center position of the virtual window.

[0133] Since the area to be blurred in the image to be processed may contain pixel positions with higher brightness, such as lights, etc.; in this scenario, in order to maximize the retention of the lighting effect while blurring the pixels, when blurring the pixels to be blurred, the brightness values ​​of each pixel in the blur window corresponding to the pixel to be blurred can be combined.

[0134] In one possible implementation, when determining the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blurring window, the brightness value of each pixel point in the blurring window can be determined first, and then the pixel value after blurring corresponding to the pixel point can be determined based on the pixel value and brightness value of each pixel point in the blurring window.

[0135] Specifically, when determining the pixel value after blurring corresponding to the pixel point based on the pixel value and brightness value of each pixel point in the blurring window, the weight of each pixel point in the blurring window can be first determined based on the brightness value of each pixel point in the blurring window; and then the pixel value of each pixel point in the blurring window is weightedly averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.

[0136] Exemplarily, when determining the weight of each pixel in the blurred window based on the brightness value of each pixel in the blurred window, the weight of the pixel whose corresponding brightness value exceeds the preset brightness value can be set to a first weight, and the weight of the pixel whose corresponding brightness value does not exceed the preset brightness value can be set to a second weight; wherein, the first weight is greater than the second weight.

[0137] Exemplarily, the second weight may be 1, and the first weight may be a preset value greater than 1.

[0138] In a real-time scenario, different images to be processed may have different brightness levels. For example, images taken at night may have lower brightness levels, while images taken during the day may have higher brightness levels. Setting weights based on fixed preset brightness values ​​may affect the image brightness, thereby impacting the blur effect.

[0139] Therefore, in one possible implementation, the brightness threshold can be first determined based on the overall brightness of the image, and then the weight of the pixel points whose corresponding brightness values ​​exceed the brightness threshold can be set to the first weight, and the weight of the pixel points whose corresponding brightness values ​​do not exceed the brightness threshold can be set to the second weight.

[0140] Exemplarily, the brightness threshold may be the average brightness of each pixel in the image to be processed.

[0141] When weighted averaging the pixel values ​​of each pixel point in the blurred window according to the corresponding weights, exemplarily, the areas of each pixel point on the R, G, and B channels can be weighted averaged according to the corresponding weights. In the image to be processed, each pixel point in other areas (i.e., the area to be blurred) except the focused area is processed according to the above method to obtain the blurred pixel value corresponding to each pixel point. After obtaining the blurred pixel value corresponding to each pixel point in the area to be blurred, the area constituted by the pixel values ​​of each pixel point after blurring and the focused area constitute the blurred image to be processed.

[0142] In another possible application scenario, the above method can also be applied to the blurring processing of the video. For each frame of the video, each frame of the video can be used as the image to be processed, and then each frame of the video can be processed according to the method provided in the above embodiment to obtain the blurred video.

[0143] It should be noted that when blurring a video, the focus area of ​​the previous video frame can be combined when determining the focus area. For example, the focus subject of the previous video frame can be tracked in the current frame, and the area where the tracked focus subject is located is used as the focus area.

[0144] In the above method, when blurring an image, for any pixel in the area to be blurred, the blur window corresponding to that pixel can be determined based on the depth difference information between that pixel and the in-focus area. Then, based on the pixel values ​​of each pixel within the blur window, the pixel value corresponding to that pixel after blurring is determined. In the above method, the depth difference information between each pixel and the in-focus area is combined when blurring the image, thereby enabling different degrees of blurring for different depth areas, thereby improving the blurring effect.

[0145] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0146] Based on the same inventive concept, an image processing device corresponding to the image processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0147] 3 , which is a schematic diagram of the architecture of an image processing device provided by an embodiment of the present disclosure, the device includes: an acquisition module 301 , a determination module 302 , and a blurring module 303 ; wherein,

[0148] An acquisition module 301 is configured to acquire an image to be processed and determine a focus area of ​​the image to be processed;

[0149] a determination module 302 configured to determine, for a pixel in an area other than the focused area in the image to be processed, a blur window corresponding to the pixel based on depth difference information between the pixel and the focused area; wherein a shape corresponding to the blur window is related to a target blur pattern;

[0150] The blur module 303 is configured to determine the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blur window.

[0151] In a possible implementation, the acquisition module 301, when determining the focus area of ​​the image to be processed, is configured to:

[0152] Determining a focused subject in the image to be processed according to entity priority;

[0153] The area where the focus subject is located is used as the focus area.

[0154] In a possible implementation, the determining module 302 is further configured to determine depth difference information between the pixel point and the focus area according to the following method:

[0155] Determining a mean depth of pixels in the focus area;

[0156] The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.

[0157] In one possible implementation, for a pixel in an area other than the focused area in the image to be processed, the determination module 302, when determining the blur window corresponding to the pixel based on depth difference information between the pixel and the focused area, is configured to:

[0158] Determining a target shape corresponding to the defocused window; wherein the target shape corresponding to the defocused window is related to a defocused style;

[0159] Determining a target size of the blur window based on the depth difference information;

[0160] For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.

[0161] In a possible implementation, the determining module 302, when determining the target shape corresponding to the virtual window, is configured to:

[0162] In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or

[0163] Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or,

[0164] Determining the image style of the image to be processed, and determining a target shape corresponding to the image style; or,

[0165] Determine the regional characteristics of the focus area and determine the target shape corresponding to the regional characteristics.

[0166] In a possible implementation, the determining module 302, when determining the target size of the defocused window based on the depth difference information, is configured to:

[0167] Determining a target size of the blur window based on the depth difference information and a preset scaling ratio;

[0168] The depth difference information is positively correlated with the target size of the blur window.

[0169] In one possible implementation, the blur module 303, when determining the blurred pixel value corresponding to each pixel point based on the pixel value of the pixel point in the blur window, is configured to:

[0170] Determining the brightness value of each pixel in the blur window;

[0171] Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.

[0172] In one possible implementation, the blur module 303, when determining the blurred pixel value corresponding to each pixel point in the blur window based on the pixel value and brightness value of the pixel point, is configured to:

[0173] Determining a weight of each pixel in the defocused window based on a brightness value of each pixel in the defocused window;

[0174] The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.

[0175] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0176] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Referring to Figure 4, there is a schematic diagram of the structure of the computer device 400 provided in the embodiment of the present disclosure. The computer device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., or various forms of servers, such as independent servers or server clusters. The computer device shown in Figure 4 is only an example and should not bring any limitations to the functions and scope of use of the embodiment of the present disclosure.

[0177] As shown in FIG4 , a computer device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory device (ROM) 402 or a program loaded from a storage device 408 into a random access memory device (RAM) 403. Various programs and data required for the operation of the computer device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0178] Typically, the following devices may be connected to I / O interface 405: input device 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output device 407 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 408 including, for example, a magnetic tape, hard disk, etc.; and communication device 409. Communication device 409 may allow computer device 400 to communicate with other devices wirelessly or by wire to exchange data. Although FIG4 illustrates computer device 400 with various devices, it should be understood that not all of the illustrated devices are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0179] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for the image processing method. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0180] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the image processing method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0181] The embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the image processing method described in the above method embodiment. For details, please refer to the above method embodiment and will not be repeated here.

[0182] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0183] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0184] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0186] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0187] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. An image processing method, comprising: Acquire an image to be processed, and determine a focus area of ​​the image to be processed; For pixel points in other areas of the image to be processed except the focus area, a blur window corresponding to the pixel point is determined based on depth difference information between the pixel point and the focus area; wherein a shape corresponding to the blur window is related to a target blur style; Based on the pixel value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.

2. The method according to claim 1, wherein: Determining the focus area of ​​the image to be processed includes: Determining a focused subject in the image to be processed according to entity priorities; The area where the focus subject is located is used as the focus area.

3. The method according to claim 1, wherein: The method for acquiring the depth difference information between the pixel point and the focus area includes: Determine a mean depth value of pixels in the focus area; The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.

4. The method according to claim 1 or 3, wherein: For a pixel point in an area other than the focus area in the image to be processed, based on depth difference information between the pixel point and the focus area, determining a blur window corresponding to the pixel point includes: Determine the target shape corresponding to the blurred window; Determining a target size of the blur window based on the depth difference information; For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.

5. The method according to claim 4, wherein: The step of determining a target shape corresponding to the blurred window includes: In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or, Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or, Determine the image style of the image to be processed, and determine the target shape corresponding to the image style or A regional feature of the focus area is determined, and a target shape corresponding to the regional feature is determined.

6. The method according to claim 4, wherein: The determining a target size of the blur window based on the depth difference information includes: Determining a target size of the blur window based on the depth difference information and a preset scaling ratio; The depth difference information is positively correlated with the target size of the blur window.

7. The method according to any one of claims 1 to 6, wherein: The step of determining the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blurring window comprises: Determine the brightness value of each pixel in the blur window; Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.

8. The method according to claim 7, wherein: The step of determining the pixel value after blurring corresponding to each pixel point based on the pixel value and the brightness value of each pixel point in the blurring window comprises: Determining the weight of each pixel in the blur window based on the brightness value of each pixel in the blur window; The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.

9. An image processing device, comprising: An acquisition module, used to acquire an image to be processed and determine a focus area of ​​the image to be processed; A determination module, configured to determine, for a pixel point in an area other than the focus area in the image to be processed, a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focus area; wherein a shape corresponding to the blur window is related to a target blur style; The blur module is used to determine the pixel value after blur processing corresponding to each pixel point based on the pixel value of each pixel point in the blur window.

10. A computer device comprising: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the image processing method according to any one of claims 1 to 8 is performed.

11. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 8 is executed.

12. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the computer program implements the image processing method according to any one of claims 1 to 8.

Images