Image processing method and device, computer equipment and storage medium
By determining the blur window based on the depth difference information and performing blur processing, the problem of poor blur effect in image focus processing is solved, and more efficient blur effect and image display effect are achieved.
Patent Information
- Application Number
- CN202310975334.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-03
- Publication Date
- 2025-02-14
AI Technical Summary
In image or video focus processing, the prior art is difficult to effectively perform blurring processing, resulting in poor image display effect.
By acquiring the image to be processed and determining its focus area, the shape and size of the blur window are determined based on the depth difference information between the pixel points and the focus area, and the blur process is performed based on the pixel values of the pixel points in the blur window.
It realizes different degrees of blurring processing for different depth areas, improves the blurring effect and enhances the outstandingness and aesthetics of image display.
Smart Images
Figure CN119450211A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing method, apparatus, computer equipment and storage medium. Background Art
[0002] In order to highlight the subject in an image or video, it is necessary to perform focus processing during the process of shooting the image or video, which places high demands on the performance of the camera and the shooter. Taking image focus processing as an example, when performing focus processing, the focus subject in the image is generally determined, and then the remaining areas outside the focus area are blurred. How to perform blur processing will affect the final display effect of the image, so image blur processing is a particularly important processing step in image processing. Summary of the invention
[0003] The embodiments of the present disclosure at least provide an image processing method, apparatus, computer equipment, and storage medium.
[0004] In a first aspect, an embodiment of the present disclosure provides an image processing method, including:
[0005] Acquire an image to be processed, and determine a focus area of the image to be processed;
[0006] For pixel points in other areas of the image to be processed except the focus area, a blur window corresponding to the pixel point is determined based on depth difference information between the pixel point and the focus area; wherein a shape corresponding to the blur window is related to a target blur style;
[0007] Based on the pixel value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.
[0008] In a possible implementation manner, determining the focus area of the image to be processed includes:
[0009] Determining a focused subject in the image to be processed according to entity priorities;
[0010] The area where the focus subject is located is used as the focus area.
[0011] In a possible implementation manner, a method for acquiring the depth difference information between the pixel point and the focus area includes:
[0012] Determine a mean depth value of pixels in the focus area;
[0013] The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.
[0014] In a possible implementation, for a pixel point in an area other than the focus area in the image to be processed, determining a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focus area includes:
[0015] Determine a target shape corresponding to the virtual window; wherein the target shape corresponding to the virtual window is related to the virtual style;
[0016] Determining a target size of the blur window based on the depth difference information;
[0017] For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.
[0018] In a possible implementation manner, determining the target shape corresponding to the blurred window includes:
[0019] In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or,
[0020] Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or,
[0021] Determine the image style of the image to be processed, and determine the target shape corresponding to the image style; or,
[0022] A regional feature of the focus area is determined, and a target shape corresponding to the regional feature is determined.
[0023] In a possible implementation, determining a target size of the blur window based on the depth difference information includes:
[0024] Determining a target size of the blur window based on the depth difference information and a preset scaling ratio;
[0025] The depth difference information is positively correlated with the target size of the blur window.
[0026] In a possible implementation, determining the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blurring window includes:
[0027] Determine the brightness value of each pixel in the blur window;
[0028] Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.
[0029] In a possible implementation, determining the pixel value after blurring corresponding to each pixel point based on the pixel value and brightness value of each pixel point in the blurring window includes:
[0030] Determining the weight of each pixel in the blur window based on the brightness value of each pixel in the blur window;
[0031] The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.
[0032] In a second aspect, the present disclosure also provides an image processing device, including:
[0033] An acquisition module, used to acquire an image to be processed and determine a focus area of the image to be processed;
[0034] A determination module, configured to determine, for a pixel point in an area other than the focus area in the image to be processed, a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focus area; wherein a shape corresponding to the blur window is related to a target blur style;
[0035] The blur module is used to determine the pixel value after blur processing corresponding to each pixel point based on the pixel value of each pixel point in the blur window.
[0036] In a possible implementation manner, the acquisition module, when determining the focus area of the image to be processed, is used to:
[0037] Determining a focused subject in the image to be processed according to entity priorities;
[0038] The area where the focus subject is located is used as the focus area.
[0039] In a possible implementation manner, the determining module is further configured to determine the depth difference information between the pixel point and the focus area according to the following method:
[0040] Determine a mean depth value of pixels in the focus area;
[0041] The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.
[0042] In a possible implementation manner, for a pixel point in an area other than the focus area in the image to be processed, the determination module, when determining a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focus area, is used to:
[0043] Determine a target shape corresponding to the virtual window; wherein the target shape corresponding to the virtual window is related to the virtual style;
[0044] Determining a target size of the blur window based on the depth difference information;
[0045] For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.
[0046] In a possible implementation manner, the determining module, when determining the target shape corresponding to the blurred window, is used to:
[0047] In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or,
[0048] Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or,
[0049] Determine the image style of the image to be processed, and determine the target shape corresponding to the image style; or,
[0050] A regional feature of the focus area is determined, and a target shape corresponding to the regional feature is determined.
[0051] In a possible implementation manner, when determining the target size of the blur window based on the depth difference information, the determination module is configured to:
[0052] Determining a target size of the blur window based on the depth difference information and a preset scaling ratio;
[0053] The depth difference information is positively correlated with the target size of the blur window.
[0054] In a possible implementation manner, the blur module, when determining the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blurring window, is configured to:
[0055] Determine the brightness value of each pixel in the blur window;
[0056] Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.
[0057] In a possible implementation manner, the blur module, when determining the pixel value after blurring corresponding to each pixel point based on the pixel value and brightness value of each pixel point in the blurring window, is configured to:
[0058] Determining the weight of each pixel in the blur window based on the brightness value of each pixel in the blur window;
[0059] The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.
[0060] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.
[0061] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.
[0062] The image processing method, apparatus, computer equipment and storage medium provided by the embodiments of the present disclosure can, when blurring an image to be processed, determine the blur window corresponding to any pixel in the area to be blurred based on the depth difference information between the pixel and the focus area, and then determine the pixel value after blurring corresponding to the pixel based on the pixel value of each pixel in the blur window. In the above method, when blurring is performed, the depth difference information between each pixel and the focus area is combined, so that different depth areas can be blurred to different degrees, thereby improving the blurring effect.
[0063] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0065] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure is shown;
[0066] Figure 2 A schematic diagram of a blurred window in the image processing method provided by an embodiment of the present disclosure is shown;
[0067] Figure 3 A schematic diagram of an image processing device provided by an embodiment of the present disclosure is shown;
[0068] Figure 4 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0069] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0070] Research has found that during / after the image or video is shot, it is necessary to perform focus processing to highlight the subject in the image or video. Taking image focus processing as an example, when performing focus processing, the focus subject in the image is generally determined, and then the remaining areas outside the focus area are blurred. When performing blur processing, it is generally performed according to fixed blur parameters, and the blur effect of the image after such processing is poor.
[0071] Based on the above research, the present disclosure provides an image processing method, device, computer equipment and storage medium. When blurring an image to be processed, for any pixel in the area to be blurred, the blur window corresponding to the pixel can be determined based on the depth difference information between the pixel and the focus area, and then the pixel value after blurring corresponding to the pixel is determined based on the pixel value of each pixel in the blur window. In the above method, when blurring is performed, the depth difference information between each pixel and the focus area is combined, so that different depth areas can be blurred to different degrees, thereby improving the blurring effect.
[0072] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0073] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.
[0074] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0075] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0076] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0077] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0078] To facilitate understanding of the present embodiment, an image processing method disclosed in the present embodiment is first introduced in detail. The execution subject of the image processing method provided in the present embodiment is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the image processing method can be implemented by a processor calling a computer-readable instruction stored in a memory.
[0079] See also Figure 1 FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, wherein the method includes steps 101 to 103, wherein:
[0080] Step 101: Acquire an image to be processed and determine a focus area of the image to be processed.
[0081] Step 102: for a pixel point in an area other than the focus area in the image to be processed, based on depth difference information between the pixel point and the focus area, determine a blur window corresponding to the pixel point.
[0082] Step 103: Based on the pixel value of each pixel point in the blur window, determine the pixel value after blur processing corresponding to the pixel point.
[0083] The following are detailed instructions for the above steps.
[0084] For step 101,
[0085] The image to be processed may be an image that needs to be focused. Exemplarily, the image to be processed may not be focused when it is captured, that is, the image to be processed does not contain a focus. The focus area of the image to be processed does not refer to the currently focused area in the image to be processed, but the area to be focused.
[0086] In a possible implementation, when determining the focus area of the image to be processed, the focus subject in the image to be processed may be determined according to entity priority, and then the area where the focus subject is located may be determined as the focus area.
[0087] Specifically, when determining the focused subject in the image to be processed, any one of the following methods may be used:
[0088] Method 1: Based on a pre-trained subject detection model, the candidate subjects contained in the image to be processed are detected, and then the focus subject is determined according to the priority order corresponding to each candidate subject.
[0089] When detecting the candidate subjects contained in the image to be processed based on a pre-trained subject detection model, exemplarily, the image to be processed can be input into the subject detection model, and the subject detection model can output all the candidate subjects contained in the image to be processed.
[0090] The priority order corresponding to each candidate subject (which can be regarded as the above-mentioned entity priority) may be preset, or may be set based on a user instruction.
[0091] Specifically, after detecting the candidate subjects contained in the image to be processed based on the subject detection model, the candidate subject identifiers can be displayed to the user, and then the priority order corresponding to each candidate subject can be determined in response to the user's sorting operation on the candidate subject identifiers.
[0092] In another possible implementation, the priority order corresponding to each candidate subject may also be automatically determined based on the image content.
[0093] Specifically, after detecting the candidate subjects contained in the image to be processed, the area occupied by each candidate subject in the image to be processed can be determined respectively, and then the priority order corresponding to each candidate subject can be determined based on the area occupied by each candidate subject in the image to be processed.
[0094] Exemplarily, if the area occupied by the candidate subject in the image to be processed is larger, it means that the proportion of the candidate subject in the image to be processed is larger, and therefore, the priority of the candidate subject is higher.
[0095] In an optional implementation, when determining the focus subject according to the priority order corresponding to each candidate subject, the candidate subject with the highest priority may be used as the focus subject.
[0096] Method 2: According to the priority order of each preset subject, the image to be processed is sequentially detected to see whether the preset subject is included, and the detected preset subject with the highest priority is used as the focus subject.
[0097] Here, the preset subject is distinguished from the candidate subject in the above-mentioned method one. The candidate subject in method one refers to all subjects detected from the image to be processed and contained in the image to be processed, while here, the preset subject refers to all possible subjects pre-set before detecting the image to be processed.
[0098] The priority order of the preset subjects (which can be regarded as the entity priority) can be determined in the same way as the priority order of the candidate subjects, which will not be described in detail here.
[0099] In an optional embodiment, when determining the focus subject, the image to be processed can be detected according to the priority ranking of each preset subject, and it can be determined in turn whether the image to be processed contains the preset subject, until a preset subject can be detected, and then the preset subject is used as the focus subject.
[0100] Exemplarily, animal face, human face, animal torso, and human torso are used as preset subjects, and the priority is arranged in the order of human face, animal face, human torso, and animal torso. When detecting the image to be processed, first detect whether the image to be processed contains a human face. If no human face is detected, then continue to detect whether the image to be processed contains an animal face. If an animal face is detected at this time, the animal face is used as the focus subject. If no animal face is detected, then continue to detect whether the image to be processed contains a human torso, and so on, until a preset subject is detected.
[0101] In a possible implementation, when multiple subjects with the same priority are detected in the image to be processed, the subject with the largest area may be selected as the focus subject.
[0102] For step 102,
[0103] In the image to be processed, all areas except the focus area are areas that need to be blurred, but the specific blurring strength may be different. Here, the blurring strength may refer to the size of the blurring range, that is, the size of the blurring window.
[0104] In one possible implementation, when determining the depth difference information between the pixel point and the focus area, the depth mean of the pixel points in the focus area can be determined first, and then the difference between the depth value of any pixel point and the depth mean can be used as the depth difference information between the pixel point and the focus area.
[0105] Here, the depth value of each pixel in the image to be processed may be based on a depth camera when the image to be processed is taken, so that each pixel of the image to be processed itself carries depth information; or, it may be determined from the depth image corresponding to the image to be processed. The depth image corresponding to the image to be processed may be determined after performing a depth analysis on the image to be processed, and the specific method for determining the depth value of the pixel in the image to be processed is not limited in this disclosure.
[0106] The blur window corresponding to each pixel point is a window used to perform blur processing on the pixel point. The shape of the blur window is related to the target blur style. The target blur style may refer to the blur style corresponding to the image to be processed. The size of the blur window is related to the depth difference information between the pixel point and the focus area. The position of the blur window is related to the position of the pixel point.
[0107] Determining the blurred window corresponding to the pixel point based on the depth difference information between the pixel point and the focus area can be understood as determining the size of the blurred window corresponding to the pixel point based on the depth difference information between the pixel point and the focus area.
[0108] Specifically, for any pixel in other areas of the image to be processed except the focus area, when determining the blur window corresponding to the pixel based on the depth difference information between the pixel and the focus area, the following steps may be performed:
[0109] Step a1, determining a target shape corresponding to the blur window; wherein the shape corresponding to the blur window is related to a target blur style;
[0110] Step a2: determining a target size of the blur window based on the depth difference information;
[0111] Step a3: for any pixel point, determine a blur window with the target shape and target size centered on the pixel point.
[0112] The blur style is used to represent the style of blurring the image to be processed, and may include, for example, a heart shape, an ellipse, etc. When determining the target shape corresponding to the blur window, any one of the following methods may be used:
[0113] Method 1: In response to a setting operation of a target blur pattern for the image to be processed, a target shape corresponding to the target blur pattern set by the setting operation is determined.
[0114] Optionally, the operation of setting a target blur style for the image to be processed may include an operation of setting target blur styles for different to-be-blurred areas of the image to be processed, and different target blur styles may be set for different to-be-blurred areas.
[0115] Method 2: Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type.
[0116] Different entity types can be set with different target shapes to achieve personalized blur styles. When the entity where the pixel point is located is empty, that is, the pixel point to be blurred is not a pixel point on any entity, the default shape can be set as the target shape.
[0117] In this implementation manner, when the area to be blurred of the image to be processed includes entities of multiple entity types, there may be multiple target blurring styles corresponding to the area to be blurred of the image to be processed.
[0118] Method 3: Determine the image style of the image to be processed, and determine the target shape corresponding to the image style.
[0119] Optionally, when determining the image style of the image to be processed, the image to be processed may be exemplarily input into a pre-trained style detection network to determine the image style of the image to be processed.
[0120] When determining the target shape corresponding to the image style, it can be determined based on the predetermined correspondence between the image style and the blur window shape; the correspondence can be pre-set or determined based on the blur processing process of multiple images to be processed that have been historically executed.
[0121] Exemplarily, the blur window shapes of the images to be processed that have been blurred in the past can be counted to determine the corresponding relationship. Here, the blur window shapes of the images to be processed that have been blurred in the past can be set by a target group, and the target group can be, for example, a professional group such as an image designer.
[0122] Method 4: Determine the regional features of the focus area, and determine the target shape corresponding to the regional features.
[0123] Here, the regional features of the focus area can be extracted based on a feature extraction network. When determining the target shape corresponding to the regional features, for example, shape features of each preset shape can be extracted, and then the similarity between the regional features and the shape features is calculated, and the shape with the highest similarity is used as the target shape.
[0124] Method 5: Determine a focus subject in the image to be processed, and determine a target shape corresponding to the focus subject.
[0125] In this way, different target shapes can be set for different focus subjects; for example, when the focus subject is an animal, an elliptical blur style can be used; when the focus subject is a child, a heart-shaped blur style can be used. In this way, the blur styles can be enriched and the blur effect can be improved.
[0126] In step a2, when determining the target size of the blurred window based on the depth difference information, the target size of the blurred window can be determined based on the depth difference information and a preset scaling ratio; wherein the depth difference information is positively correlated with the target size of the blurred window.
[0127] Here, the reason why the depth difference information is positively correlated with the target size of the blur window is that, if the depth difference information is larger, it means that the entity corresponding to the pixel point is farther away from the focus subject corresponding to the focus area, and the information contained in the entity farther away from the focus subject is generally useless information. Therefore, a large-size blur window can be used to blur the pixel point.
[0128] The larger the size of the blur window, the more pixels are included in the blur window. In this way, for any pixel to be blurred, when blurring the pixel based on the pixel value of the pixel within the blur window corresponding to the pixel, the greater the difference between the pixel value of the pixel after blurring and the pixel value of the pixel before blurring, the deeper the degree of blurring.
[0129] In step a3, the position of the blur window in the image to be processed is related to the position of the pixel to be blurred. For example, the target size of the blur window corresponding to pixel 1 is 3*3 pixels, the shape of the blur window is a square, and the position of the blur window is as follows: Figure 2 As shown; the target size of the blur window corresponding to pixel 2 is 5*5 pixels, the shape of the blur window is a square, and the position of the blur window is as follows Figure 2 shown.
[0130] For step 103,
[0131] The pixel points within the virtual window include the pixel points to be virtualized, that is, the pixel points corresponding to the center position of the virtual window.
[0132] Since the area to be blurred in the image to be processed may contain pixel positions with higher brightness, such as lights; in this scenario, in order to maximize the retention of the lighting effect while blurring the pixels, when blurring the pixels to be blurred, the brightness values of each pixel in the blur window corresponding to the pixel to be blurred can be combined.
[0133] In a possible implementation, when determining the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blurring window, the brightness value of each pixel point in the blurring window can be determined first, and then the pixel value after blurring corresponding to the pixel point can be determined based on the pixel value and brightness value of each pixel point in the blurring window.
[0134] Specifically, when determining the pixel value after blurring corresponding to each pixel point based on the pixel value and brightness value of each pixel point in the blurring window, the weight of each pixel point in the blurring window can be first determined based on the brightness value of each pixel point in the blurring window; and then the pixel value of each pixel point in the blurring window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.
[0135] Exemplarily, when determining the weight of each pixel in the blurred window based on the brightness value of each pixel in the blurred window, the weight of the pixel whose corresponding brightness value exceeds the preset brightness value can be set to a first weight, and the weight of the pixel whose corresponding brightness value does not exceed the preset brightness value can be set to a second weight; wherein the first weight is greater than the second weight.
[0136] Exemplarily, the second weight may be 1, and the first weight may be a preset value greater than 1.
[0137] In a possible real-time scenario, the image brightness of different images to be processed may be different. For example, the image brightness of images taken at night may be lower, while the image brightness of images taken during the day may be higher. If the weight is set according to a fixed preset brightness value, it may be affected by the image brightness, thereby affecting the blur effect.
[0138] Therefore, in one possible implementation, the brightness threshold can be first determined based on the overall brightness of the image, and then the weight of the pixel points whose corresponding brightness values exceed the brightness threshold can be set to the first weight, and the weight of the pixel points whose corresponding brightness values do not exceed the brightness threshold can be set to the second weight.
[0139] Exemplarily, the brightness threshold may be the average brightness of each pixel in the image to be processed.
[0140] When weighted averaging the pixel values of each pixel point in the blur window according to the corresponding weights, exemplarily the areas of each pixel point on the R, G, and B channels can be weighted averaged according to the corresponding weights. In the image to be processed, each pixel point in other areas (i.e., the area to be blurred) except the focused area is processed according to the above method to obtain the pixel value after blurring corresponding to each pixel point. After obtaining the pixel value after blurring corresponding to each pixel point in the area to be blurred, the area constituted by the pixel values of each pixel point after blurring and the focused area constitute the image to be processed after blurring.
[0141] In another possible application scenario, the above method can also be applied to the blurring processing of the video. For each frame of the video, each frame of the video can be used as the image to be processed, and then each frame of the video can be processed according to the method provided in the above embodiment to obtain the blurred video.
[0142] It should be noted that when blurring the video, the focus area of the previous video frame can be combined when determining the focus area. For example, the focus subject of the previous video frame can be tracked in the current frame, and then the area where the tracked focus subject is located is used as the focus area.
[0143] In the above method, when blurring the image to be processed, for any pixel in the area to be blurred, the blur window corresponding to the pixel can be determined based on the depth difference information between the pixel and the focus area, and then the pixel value after blurring corresponding to the pixel is determined based on the pixel value of each pixel in the blur window. In the above method, when blurring, the depth difference information between each pixel and the focus area is combined, so that different depth areas can be blurred to different degrees, thereby improving the blurring effect.
[0144] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0145] Based on the same inventive concept, an image processing device corresponding to the image processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0146] Reference Figure 3 , which is a schematic diagram of the architecture of an image processing device provided by an embodiment of the present disclosure, the device comprises: an acquisition module 301, a determination module 302, and a blurring module 303; wherein,
[0147] An acquisition module 301 is used to acquire an image to be processed and determine a focus area of the image to be processed;
[0148] A determination module 302 is used to determine, for a pixel point in an area other than the focus area in the image to be processed, a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focus area; wherein a shape corresponding to the blur window is related to a target blur style;
[0149] The blur module 303 is used to determine the pixel value after blur processing corresponding to each pixel point based on the pixel value of each pixel point in the blur window.
[0150] In a possible implementation manner, the acquisition module 301, when determining the focus area of the image to be processed, is used to:
[0151] Determining a focused subject in the image to be processed according to entity priorities;
[0152] The area where the focus subject is located is used as the focus area.
[0153] In a possible implementation manner, the determining module 302 is further configured to determine the depth difference information between the pixel point and the focus area according to the following method:
[0154] Determine a mean depth value of pixels in the focus area;
[0155] The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.
[0156] In a possible implementation manner, for a pixel point in an area other than the focus area in the image to be processed, the determination module 302, when determining a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focus area, is configured to:
[0157] Determine a target shape corresponding to the virtual window; wherein the target shape corresponding to the virtual window is related to the virtual style;
[0158] Determining a target size of the blur window based on the depth difference information;
[0159] For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.
[0160] In a possible implementation, the determining module 302, when determining the target shape corresponding to the blurred window, is configured to:
[0161] In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or,
[0162] Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or,
[0163] Determine the image style of the image to be processed, and determine the target shape corresponding to the image style; or,
[0164] A regional feature of the focus area is determined, and a target shape corresponding to the regional feature is determined.
[0165] In a possible implementation manner, the determination module 302, when determining the target size of the blur window based on the depth difference information, is configured to:
[0166] Determining a target size of the blur window based on the depth difference information and a preset scaling ratio;
[0167] The depth difference information is positively correlated with the target size of the blur window.
[0168] In a possible implementation, the blur module 303, when determining the pixel value after blur processing corresponding to each pixel point based on the pixel value of each pixel point in the blur window, is configured to:
[0169] Determine the brightness value of each pixel in the blur window;
[0170] Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.
[0171] In a possible implementation, the blur module 303, when determining the pixel value after blurring corresponding to each pixel point based on the pixel value and brightness value of each pixel point in the blur window, is configured to:
[0172] Determining the weight of each pixel in the blur window based on the brightness value of each pixel in the blur window;
[0173] The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.
[0174] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0175] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 4 , which is a schematic diagram of the structure of a computer device 400 provided in an embodiment of the present disclosure. The computer device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., or various forms of servers, such as independent servers or server clusters. Figure 4 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0176] like Figure 4As shown, the computer device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only storage device (ROM) 402 or a program loaded from a storage device 405 to a random access storage device (RAM) 403. Various programs and data required for the operation of the computer device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0177] Typically, the following devices may be connected to the I / O interface 405: input devices 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 409. The communication devices 409 may allow the computer device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The computer device 400 is shown with various devices, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.
[0178] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for an image processing method. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0179] The present disclosure also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.
[0180] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the image processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0181] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0182] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0183] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0184] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0185] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0186] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. An image processing method, characterized in that: include: Acquire an image to be processed, and determine a focus area of the image to be processed; For pixel points in other areas of the image to be processed except the focus area, a blur window corresponding to the pixel point is determined based on depth difference information between the pixel point and the focus area; wherein a shape corresponding to the blur window is related to a target blur style; Based on the pixel value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.
2. The method according to claim 1, characterized in that Determining the focus area of the image to be processed includes: Determining a focused subject in the image to be processed according to entity priorities; The area where the focus subject is located is used as the focus area.
3. The method according to claim 1, characterized in that: The method for acquiring the depth difference information between the pixel point and the focus area includes: Determine a mean depth value of pixels in the focus area; The difference between the depth value of the pixel point and the depth mean is used as the depth difference information.
4. The method according to claim 1 or 3, characterized in that: For a pixel point in an area other than the focus area in the image to be processed, based on depth difference information between the pixel point and the focus area, determining a blur window corresponding to the pixel point includes: Determine the target shape corresponding to the blurred window; Determining a target size of the blur window based on the depth difference information; For a pixel point in the other area, a blur window with the target shape and the target size centered on the pixel point is determined.
5. The method according to claim 4, characterized in that The step of determining a target shape corresponding to the blurred window includes: In response to a setting operation for a blur pattern of the image to be processed, determining a target shape corresponding to the blur pattern set by the setting operation; or, Determine the entity type of the entity where the pixel point is located, and determine the target shape corresponding to the entity type; or, Determine the image style of the image to be processed, and determine the target shape corresponding to the image style; or, A regional feature of the focus area is determined, and a target shape corresponding to the regional feature is determined.
6. The method according to claim 4, characterized in that The determining a target size of the blur window based on the depth difference information includes: Determining a target size of the blur window based on the depth difference information and a preset scaling ratio; The depth difference information is positively correlated with the target size of the blur window.
7. The method according to claim 1, characterized in that The step of determining the pixel value after blurring corresponding to each pixel point based on the pixel value of each pixel point in the blurring window comprises: Determine the brightness value of each pixel in the blur window; Based on the pixel value and brightness value of each pixel point in the blur window, the pixel value after blur processing corresponding to the pixel point is determined.
8. The method according to claim 7, characterized in that The step of determining the pixel value after blurring corresponding to each pixel point based on the pixel value and the brightness value of each pixel point in the blurring window comprises: Determining the weight of each pixel in the blur window based on the brightness value of each pixel in the blur window; The pixel value of each pixel point in the blur window is weighted averaged according to the corresponding weight to obtain the pixel value after blurring corresponding to the pixel point.
9. An image processing device, characterized in that: include: An acquisition module, used to acquire an image to be processed and determine a focus area of the image to be processed; A determination module, configured to determine, for a pixel point in an area other than the focus area in the image to be processed, a blur window corresponding to the pixel point based on depth difference information between the pixel point and the focus area; wherein a shape corresponding to the blur window is related to a target blur style; The blur module is used to determine the pixel value after blur processing corresponding to each pixel point based on the pixel value of each pixel point in the blur window.
10. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are performed.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are executed.