Image processing method and apparatus, device, computer-readable storage medium, and product

Abstract

Provided in embodiments of the present disclosure are an image processing method and apparatus, a device, a computer-readable storage medium, and a product. The method comprises: in response to a preset trigger operation of a user in a target application, acquiring an image to be processed, wherein said image is collected by an image collection apparatus in a terminal device, and said image at least comprises a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image; determining a target area in the preview image on the basis of the image to be processed; performing a background blurring operation on the preview image, and obtaining a target image on the basis of the target area and the preview image having undergone background blurring; and displaying the target image in the viewfinder.

Description

Image processing method, device, equipment, computer readable storage medium and product

[0001] Cross-reference to Related Applications

[0002] This application claims the benefit of Chinese Patent Application No. 202411338126.X, filed September 24, 2024. The entire teachings of the above application are incorporated herein by reference. TECHNICAL FIELD

[0003] Embodiments of the present disclosure relate to the technical field of computer, and particularly relate to an image processing method, device, equipment, computer readable storage medium and product. BACKGROUND

[0004] Current terminal devices are generally provided with image acquisition devices, and users can realize image acquisition operation based on the image acquisition devices. In order to improve the content quality of the images collected by the users, image processing algorithms are generally integrated in the terminal devices to realize image processing of the collected images, to realize background blurring and highlight the display effect of the content subject.

[0005] With the gradual development of image processing technology, more and more image processing applications have gradually entered the life of users. Users can process and collect images based on image processing applications. However, image processing applications cannot directly call the image processing algorithms inherited in the terminal devices to realize image processing. How to realize background blurring and highlight the image processing effect of the content subject has become a problem to be solved. SUMMARY

[0006] Embodiments of the present disclosure provide an image processing method, device, equipment, computer readable storage medium and product.

[0007] In a first aspect, embodiments of the present disclosure provide an image processing method, comprising:

[0008] In response to a trigger operation of a user in a target application, a to-be-processed image is acquired, wherein the to-be-processed image is collected by an image acquisition device in a terminal device, and the to-be-processed image at least includes a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image.

[0009] Based on the to-be-processed image, a target region in the preview image is determined, wherein the target region includes a content subject in the preview image.

[0010] The preview image is subjected to background blurring.

[0011] Based on the target region and the preview image subjected to background blurring, a target image is obtained.

[0012] displaying the target image in the viewfinder.

[0013] In a second aspect, the embodiments of the present disclosure provide an image processing apparatus, comprising:

[0014] an acquisition module configured to acquire a to-be-processed image in response to a trigger operation of a user in a target application, wherein the to-be-processed image is collected by an image collection device in a terminal device, and the to-be-processed image at least comprises a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image;

[0015] a determination module configured to determine a target region in the preview image based on the to-be-processed image, wherein the target region comprises a content subject in the preview image;

[0016] a processing module configured to perform background blurring on the preview image;

[0017] a generation module configured to obtain a target image based on the target region and the preview image after the background blurring;

[0018] a display module configured to display the target image in the viewfinder.

[0019] In a third aspect, the embodiments of the present disclosure provide an electronic device, comprising a processor and a memory.

[0020] The memory stores computer-executable instructions.

[0021] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the image processing method according to the first aspect and various possible designs of the first aspect.

[0022] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, which stores computer-executable instructions, and when a processor executes the computer-executable instructions, the image processing method according to the first aspect and various possible designs of the first aspect is implemented.

[0023] In a fifth aspect, the embodiments of the present disclosure provide a computer program product, comprising a computer program, and when a processor executes the computer program, the image processing method according to the first aspect and various possible designs of the first aspect is implemented. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those of ordinary skill in the art without creative labor under the premise of the drawings.

[0025] FIG. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0026] FIG. 2 is a flowchart of an image processing method according to another embodiment of the present disclosure;

[0027] FIG. 3 is a schematic diagram of a preview image according to an embodiment of the present disclosure;

[0028] FIG. 4 is a schematic diagram of another preview image according to an embodiment of the present disclosure;

[0029] FIG. 5 is a flowchart of an image processing method according to another embodiment of the present disclosure;

[0030] FIG. 6 is a flowchart of an image processing method according to another embodiment of the present disclosure;

[0031] FIG. 7 is a schematic diagram of an image processing apparatus according to an embodiment of the present disclosure;

[0032] FIG. 8 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the following will combine the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are some of the embodiments of the present disclosure, but not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present disclosure.

[0034] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0035] For example, when responding to the active request of the user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be executed will need to obtain and use the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the software or hardware such as electronic device, application program, server or storage medium, etc. that executes the operation of the technical solution of the present disclosure according to the prompt information.

[0036] As an optional but non-limiting implementation, in response to receiving the active request of the user, the manner of sending the prompt information to the user may be, for example, a pop-up window manner in which the prompt information may be presented in a text manner. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide the personal information to the electronic device.

[0037] It can be understood that the above notification and user authorization obtaining process is only illustrative and does not limit the implementation of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0038] In order to solve the technical problem that the existing image processing application cannot achieve the background blurring and highlight the content subject image processing effect, the present disclosure provides an image processing method, device, equipment, computer readable storage medium and product.

[0039] It should be noted that the image processing method, device, equipment, computer readable storage medium and product provided by the present disclosure can be applied in any scene of image processing for highlighting the content subject.

[0040] The IOS system camera provides a portrait mode shooting manner. In the portrait mode, a clear content subject and a blurred background image can be collected. However, the current iOS system does not provide an interface of the portrait mode to other third-party application software. Therefore, how to realize the portrait mode image shooting in the application software has become a problem to be solved.

[0041] In the process of solving the above problem, the inventors found through research that the AVFoundation module provided by the IOS system can collect the preview image, the depth map and the key point information of the content subject in the preview image. The content subject can be a human subject. Thus, the target region can be accurately identified based on the depth map and the key point information, and the target region includes the content subject. Then, image processing is performed based on the target region and the preview image to obtain a clear content subject and a blurred background image processing effect. Thus, the user can realize the collection of the portrait mode image in the third-party application software.

[0042] FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure. As shown in FIG. 1, the method comprises the following steps:

[0043] In step 101, in response to a trigger operation of a user in a target application, a to-be-processed image is acquired, wherein the to-be-processed image is collected by an image collection device in a terminal device, and the to-be-processed image at least includes a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image.

[0044] The execution subject of the embodiment is an image processing apparatus. The image processing apparatus can be coupled to a server associated with a target application, so as to be capable of acquiring a to-be-processed image collected by a terminal device based on a preset trigger operation of a user in the target application, determining a target region based on the to-be-processed image, and performing an image processing operation of background blurring. Alternatively, the image processing apparatus can also be coupled to a client of the target application, so as to be capable of acquiring the to-be-processed image based on the preset trigger operation of the user in the target application, and performing the image processing operation of background blurring based on the to-be-processed image.

[0045] In the embodiment, the user can perform image processing operations such as image collection and image editing in the target application. In the image processing process, the user can trigger a preset trigger operation in the target application according to actual needs, so as to achieve acquisition of an image with a blurred background and a clear content subject. The preset trigger operation can be a selection operation on a preset image processing control, or can also be a state switching operation on a preset processing control.

[0046] Taking an actual application as an example, when the user performs an image collection operation in the target application, the target application can provide the user with a plurality of image processing options, such as special effects, filters, stickers, portrait modes, and the like, each of which is associated with an image processing control. The user can perform a selection operation on a control of a preset “portrait mode” according to actual needs, so as to achieve acquisition of an image in the portrait mode. Alternatively, an image collection page can include a processing control associated with the “portrait mode”, and in response to a trigger operation of the user on the processing control, the state of the processing control can be switched to an open state. Thus, the user can collect an image in the portrait mode in the image collection page.

[0047] Further, in response to the preset trigger operation of the user in the target application, a to-be-processed image can be acquired. The to-be-processed image at least includes a preview image collected by a preset image processing apparatus in the terminal device and a depth map associated with the preview image.

[0048] The image processing apparatus can be a preset original camera in the terminal device. In addition, the terminal device is also preconfigured with a device having a depth capture capability.

[0049] As an implementable manner, a plurality of devices having a depth capture capability can be provided in the terminal device. When the user performs an image collection operation in the target application, an image collection parameter can be determined, and a device matching the parameter is selected from the plurality of devices having a depth capture capability based on the parameter determined by the user to process the preview image, so as to obtain a depth map.

[0050] In step 102, a target region in the preview image is determined based on the image to be processed, wherein the target region includes a content subject in the preview image.

[0051] In this embodiment, the depth map often includes depth information of each pixel in the preview image, which can represent the distance between the pixel and the lens of the image acquisition device.

[0052] It can be understood that in the preview image, the content closer to the shooting lens is more likely to be the content subject, and the content farther from the lens is more likely to be the background. For example, when taking a portrait picture, the person in the picture is often closer to the shooting lens than the background buildings, landscapes, and other content.

[0053] Therefore, in order to obtain an image with a blurred background and a clear content subject, it is necessary to first identify the target region in the preview image. The target region includes the content subject in the preview image. The content subject can be content with a prominent feature in the preview image. For example, when the preview image is a portrait, the content subject can be a person, and when the preview image is a building image, the content subject can be the most prominent building.

[0054] Therefore, in order to obtain an image with a blurred background and a clear content subject, it is necessary to first identify the target region in the preview image. The target region includes the content subject in the preview image. The content subject can be content with a prominent feature in the preview image. For example, when the preview image is a portrait, the content subject can be a person, and when the preview image is a building image, the content subject can be the most prominent building.

[0055] In step 103, the preview image is blurred.

[0056] In step 104, a target image is obtained based on the target region and the preview image after background blurring.

[0057] In this embodiment, in order to achieve a display effect with a blurred background and a clear content subject, the preview image can be blurred. Any image blurring algorithm can be used to blur the preview image, and the present disclosure does not limit this.

[0058] Further, a target image is obtained based on the target region and the preview image after background blurring. Since the target region is a clear image and the preview image is a blurred image, by superimposing the target region and the preview image, a display effect with a blurred background and a clear content subject can be obtained, and the target region can be prevented from deforming or blurring while the background is blurred, thereby improving the display effect of the target image.

[0059] In step 105, the target image is displayed in the viewfinder.

[0060] In this embodiment, after obtaining the target image, the target image can be displayed in the viewfinder so that the user can view the target image in real time. Thus, the user can capture the image more accurately based on the display effect in the viewfinder.

[0061] Further, based on any of the above embodiments, the method further includes:

[0062] In response to the image capture operation triggered by the user, the target image displayed in the viewfinder is stored in a preset storage path.

[0063] In this embodiment, after the target image is displayed in the viewfinder, the user can view the image processing effect of the blurred background and the clear content subject in real time.

[0064] Optionally, the user can trigger the image capture operation according to actual needs. The display page can be provided with an image capture control, and the user can trigger the image capture operation by triggering the image capture control.

[0065] In response to the image capture operation, the target image currently displayed in the viewfinder can be stored in a preset storage path.

[0066] The image processing method provided in this embodiment can accurately determine the target region by responding to the preset trigger operation of the user in the target application, obtaining at least the to-be-processed image including the preview image currently displayed in the viewfinder and the depth map associated with the preview image, performing a background blurring operation on the preview image, obtaining the target image based on the target region and the preview image after the background blurring operation, and displaying the target image in the viewfinder. Thus, the user can view and capture the image effect of the blurred background and the highlighted content subject in real time in the target application, and the image processing effect of the target application is improved.

[0067] FIG. 2 is a flowchart of an image processing method provided in another embodiment of the present disclosure. Based on any of the above embodiments, the to-be-processed image further includes key point information associated with a display object in the preview image. As shown in FIG. 2, step 102 includes:

[0068] Step 201: determining at least one content subject in the preview image based on the depth map and / or the key point information.

[0069] Step 202: performing an image segmentation operation on the preview image based on the display position of the content subject to obtain the target region.

[0070] In this embodiment, the image capturing device preset in the terminal device can also perform a key point information collection operation, and the key point information is used to represent coordinate information of the display object. The preview image can include a display object, which includes but is not limited to a person, an animal, a building, or a specific position of a person, such as a head, a limb, and the like.

[0071] For example, when the display object is a person, the image processing device can identify the key points of the person, and the key point information of the person includes coordinate information of each key point of the person.

[0072] Further, the to-be-processed image can include a preview image, a depth map associated with the preview image, and key point information associated with a display object in the preview image. After the to-be-processed image is obtained, at least one content subject in the preview image can be accurately determined based on the depth map and / or the key point information.

[0073] After the at least one content subject is determined, the preview image can be subjected to an image segmentation operation based on the at least one content subject, to obtain a target region.

[0074] The image processing method provided in this embodiment can accurately identify a content subject in a preview image based on a depth map and / or key point information associated with a display object, and then perform an image segmentation operation based on the content subject. The accuracy of image processing is improved.

[0075] Further, on the basis of any of the above embodiments, step 201 includes:

[0076] According to the depth map, at least one display object in the preview image that satisfies a first condition with the image capturing device is determined.

[0077] Based on the key point information, display sizes of the display objects are determined, and a target object that satisfies a preset second condition in the at least one display object is determined.

[0078] The display size of the target object is compared with display sizes of other display objects in the preview image, respectively, to obtain a comparison result.

[0079] According to the comparison result, a content subject in the preview image is determined.

[0080] In this embodiment, the identification operation of the content subject in the preview image can be realized based on the combination of the depth map and the key point information.

[0081] Optionally, the depth map can represent a distance between each pixel point and the shooting lens. Thus, at least one display object in the preview image can be determined according to the depth map, where the distance between the display object and the image acquisition device satisfies a preset first condition. The first condition can be that the display content closest to the shooting lens is determined as the at least one display object.

[0082] Further, in an image, an object closest to the lens and having a larger display size is generally a content subject in the image. Thus, when the number of display objects is multiple, a target object can be selected from the multiple display objects based on the display size. Since the key point information can represent coordinate information of multiple pixel points of the display object, the display size of each display object can be determined based on the key point information, and a target object having a display size satisfying a preset second condition is determined from the at least one display object. The second condition can be that a display object having the largest display size is selected as the target object from the at least one display object.

[0083] The preview image can be a single-person photograph, or can be a group photograph. When the preview image is a single-person photograph, the display size of other display objects in the preview image is much smaller than that of the target object, and when the preview image is a group photograph, the display size of the target object is similar to that of other display objects.

[0084] Thus, in order to accurately identify the content subject, the display size of the target object can be compared with the display size of other display objects in the preview image respectively, and a comparison result is obtained. The content subject in the preview image is determined according to the comparison result.

[0085] Further, based on any of the above embodiments, the content subject in the preview image is determined according to the comparison result, including:

[0086] For each display object, if the size ratio between the display size of the target object and the display size of the display object is within a preset first range, whether a depth difference between the display object and the target object is within a preset second range is determined based on the depth map.

[0087] If yes, the target object and the display object are determined as the content subject.

[0088] If no, the target object is determined as the content subject.

[0089] In this embodiment, the first range can be preset. The first range can be a range close to 1. The user can set the first range according to actual needs, and the present disclosure does not limit this.

[0090] Further, for each display object, if the size ratio between the display size of the target object and the display size of the display object is within a preset first range, it is indicated that the display size of the display object is close to the display size of the target object. It is determined whether the depth difference between the display object and the target object is within a preset second range based on the depth map. The depth difference can be a value set by the user according to actual needs.

[0091] If yes, it is indicated that the display size of the display object is close to the display size of the target object, and the distance from the display object to the shooting lens is close, so it is likely to be a group shot scene, and therefore the target object and the display object can be determined as the content subject. If no, it is indicated that the display size of the display object is close to the display size of the target object, but the display object is far away from the shooting lens, and it is not a group shot scene, and therefore the target object is determined as the content subject.

[0092] FIG. 3 is a schematic diagram of a preview image provided by an embodiment of the present disclosure, as shown in FIG. 3, the preview image 31 includes a plurality of display objects. After the target object 32 is determined, the display size ratio between the target object 32 and other display objects 33 can be determined. If the display size ratio is within a preset first range, and the depth difference is within a second range, it is indicated that the distance from the target object 32 to the shooting lens is close to the distance from the other display objects 33 to the shooting lens, and the display size is close, and therefore the target object 32 and the other display objects 33 are likely to be in a group shot relationship. Therefore, the target object 32 and the other display objects 33 can be determined as the content subject.

[0093] The image processing method provided by the embodiment can accurately identify the content subject close to the image acquisition device and having a large display size by determining the distance between the display object and the image acquisition device based on the depth map, determining the display size of the display object in the preview image based on the key point information, and pre-setting the first condition and the second condition, and then accurately performing image segmentation operation based on the content subject, thereby improving the display effect of the processed image.

[0094] Further, based on any of the above embodiments, the determining the content subject in the preview image according to the comparison result comprises:

[0095] For each display object, if the size ratio between the display size of the target object and the display size of the display object is not within a preset first range, the target object is determined as the content subject.

[0096] In the embodiment, if the size ratio between the display size of the target object and the display size of the display object is not in the preset first range, it is indicated that the display size difference between the target object and the display object is large, and thus the possibility of the preview image being a group photo is low, and thus the target object can be determined as the content subject.

[0097] FIG. 4 is another schematic diagram of a preview image provided by the embodiment of the present disclosure. As shown in FIG. 4, the preview image 41 includes a plurality of display objects. After the target object 42 is determined, the display size ratio between the target object 42 and other display objects 43 can be determined. If the size ratio is not in the preset first range, it is indicated that the other display objects 43 in the preview image 41 can be the shooting background of the target object 42. Taking the target object 42 as a person as an example, the other display objects can be passers-by in the shooting place.

[0098] The image processing method provided by the embodiment can accurately distinguish the content subject and the shooting background in the preview image by determining the distance between the display object and the image acquisition device based on the depth map, determining the display size of the display object in the preview image based on the key point information, and presetting the first condition, thereby improving the accuracy of the content subject recognition.

[0099] Optionally, in any of the above embodiments, step 201 includes:

[0100] The depth information corresponding to each display object in the preview image is determined based on the depth map.

[0101] At least one display object with the depth information in the preset depth range is determined as the at least one content subject.

[0102] In the embodiment, since the depth information can represent the distance between the pixel in the preview image and the shooting lens, the determination of the content subject can be realized only based on the depth information.

[0103] Optionally, a preset depth range associated with the preview image can be determined in advance. The maximum depth value and the minimum depth value associated with the preview image can be determined based on the depth map, and the preset depth range can be determined based on the maximum depth value, the minimum depth value, and a preset ratio. For example, the preset ratio can be 80%, or the preset ratio can be adjusted according to actual needs, which is not limited in the present disclosure.

[0104] Further, one display object can be determined as a subject, the depth information corresponding to each display object in the preview image is determined based on the depth map, and at least one display object with the depth information in the preset depth range is determined as at least one content subject.

[0105] The image processing method provided in the embodiment can accurately identify the content subject based on the depth range after the depth map associated with the preview image is acquired, thereby improving the accuracy and efficiency of content subject identification.

[0106] FIG. 5 is a flowchart of an image processing method provided in another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG. 5, step 202 includes:

[0107] Step 501: Determine the key point information associated with each content subject.

[0108] Step 502: Determine the target mask corresponding to the content subject according to the key point information.

[0109] Step 503: Perform image segmentation on the preview image through the target mask to obtain the target region.

[0110] In the embodiment, after at least one content subject in the preview image is determined, for each content subject, the key point information of the content subject can be determined based on the to-be-processed image. The target mask corresponding to the content subject is determined according to the key point information. In the target mask, the pixel value of the target region can be 1, and the pixel value of other regions can be 0. Therefore, after the target mask is multiplied by the preview image, the target subject region can be accurately obtained.

[0111] For example, the to-be-processed image can include face key point information. The content subject can be a human body region corresponding to the face. Therefore, the face key point information can be used as an index to further determine the key point information of the entire human body region corresponding to the face. Any key point recognition algorithm can be used to recognize the key points of the human body region, and the present disclosure does not limit this.

[0112] The image processing method provided in the embodiment can determine the mask of the content subject based on the key point information, thereby accurately performing segmentation operation on the content subject based on the target mask.

[0113] Optionally, based on any of the above embodiments, step 102 includes:

[0114] Perform image segmentation on the preview image through the preset foreground and background segmentation algorithm in the terminal device to obtain a target region in the preview image.

[0115] In the embodiment, the preview image can not include a display object. For example, the preview image can be a landscape image, which does not include a human figure.

[0116] For the preview image not including the display object, the preview image can be subjected to image segmentation operation through a preset foreground and background segmentation algorithm in the terminal device to obtain the target region in the preview image.

[0117] The image processing method provided in the embodiment can improve the accuracy and efficiency of target region segmentation by implementing segmentation operation on the content subject through the foreground and background segmentation algorithm when the preview image does not include a person.

[0118] FIG. 6 is a flowchart of an image processing method provided in another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG. 6, step 103 includes:

[0119] Step 601: performing Gaussian blur operation on the preview image to obtain a preview image after background blurring.

[0120] Step 104 includes:

[0121] Step 602: covering the segmented target region in a corresponding position in the preview image after background blurring to obtain the target image.

[0122] In the embodiment, in order to achieve the display effect of clear content subject and blurred background, Gaussian blur operation can be performed on the preview image to obtain a preview image after background blurring. Alternatively, any other image blurring algorithm can be used to perform blurring processing on the preview image, which is not limited in the present disclosure.

[0123] Further, the target region obtained through image segmentation can be covered in a corresponding position in the preview image after background blurring to obtain a target image. Thus, the target region can be prevented from deformation and blurring while ensuring background blurring, thereby improving the display effect of the target image.

[0124] The image processing method provided in the embodiment can segment the target region, perform blurring operation on the preview image, and cover the target region on the preview image after background blurring. Thus, the target region boundary can be ensured to be clear and not deformed on the basis of achieving background blurring.

[0125] Fig. 7 is a structural schematic diagram of an image processing apparatus provided by the embodiment of the present disclosure. As shown in Fig. 7, the apparatus comprises an acquisition module 71, a determination module 72, a processing module 73, and a display module 74. The acquisition module 71 acquires a to-be-processed image in response to a triggering operation of a user in a target application, wherein the to-be-processed image is collected by an image collection device in a terminal device, and the to-be-processed image at least comprises a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image. The determination module 72 determines a target region in the preview image based on the to-be-processed image, wherein the target region comprises a content subject in the preview image. The processing module 73 performs background blurring on the preview image. The generation module 74 obtains a target image based on the target region and the preview image after the background blurring. The display module 75 displays the target image in the viewfinder.

[0126] Further, on the basis of any of the above embodiments, the to-be-processed image further comprises key point information associated with a display object in the preview image. The determination module determines at least one content subject in the preview image based on the depth map and / or the key point information. The preview image is subjected to image segmentation operation based on the display position of the content subject, and the target region is obtained.

[0127] Further, on the basis of any of the above embodiments, the determination module determines at least one display object in the preview image that satisfies a first condition in terms of distance between the display object and the image collection device based on the depth map. The display size of each display object is determined based on the key point information, and a target object that satisfies a preset second condition in terms of display size is determined in the at least one display object. The display size of the target object is compared with the display size of other display objects in the preview image, and a comparison result is obtained. The content subject in the preview image is determined based on the comparison result.

[0128] Further, on the basis of any of the above embodiments, the determination module determines, for each display object, whether a depth difference between the target object and the display object is within a preset second range based on the depth map if a size ratio between the display size of the target object and the display size of the display object is within a preset first range. If yes, the target object and the display object are determined as the content subject. If no, the target object is determined as the content subject.

[0129] Further, based on any of the above embodiments, the determining module is configured to: for each display object, if a size ratio between the display size of the target object and the display size of the display object is not within a preset first range, determining the target object as the content subject.

[0130] Further, based on any of the above embodiments, the determining module is configured to: determine, according to the depth map, depth information corresponding to each display object in the preview image. Determine at least one display object with the depth information within a preset depth range as the at least one content subject.

[0131] Further, based on any of the above embodiments, the determining module is configured to: determine key point information associated with each content subject. Determine a target mask corresponding to the content subject according to the key point information. Perform image segmentation operation on the preview image through the target mask to obtain the target region.

[0132] Further, based on any of the above embodiments, the determining module is configured to: perform image segmentation operation on the preview image through a preset foreground and background segmentation algorithm in the terminal device to obtain a target region in the preview image.

[0133] Further, based on any of the above embodiments, the processing module is configured to: perform Gaussian blur operation on the preview image to obtain a preview image after background blurring. The generating module is configured to: overlay the segmented target region in a corresponding position in the preview image after background blurring to obtain the target image.

[0134] Further, based on any of the above embodiments, the method further includes a collecting module configured to: in response to an image collecting operation triggered by a user, store the target image displayed in the viewfinder to a preset storage path.

[0135] The device provided in the embodiment can be used to execute the technical solutions of the above method embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.

[0136] In order to implement the above embodiments, the disclosure embodiments further provide a computer readable storage medium, the computer readable storage medium stores computer execution instructions, when the processor executes the computer execution instructions, the image processing method is realized as any of the above embodiments.

[0137] In order to implement the above embodiments, the disclosure embodiments further provide a computer program product, including a computer program, the computer program is executed by the processor to realize the image processing method as any of the above embodiments.

[0138] To achieve the above-mentioned embodiments, the embodiments of the present disclosure further provide an electronic device, comprising: a processor and a memory;

[0139] The memory stores computer-executable instructions;

[0140] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the image processing method according to any one of the above-mentioned embodiments.

[0141] FIG. 8 is a structural schematic diagram of an electronic device provided by the embodiments of the present disclosure. The electronic device 800 can be a terminal device or a server. The terminal device can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a Personal Digital Assistant (PDA), a Portable Android Device (PAD), a Portable Media Player (PMP), a vehicle-mounted terminal (for example, a vehicle-mounted navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in FIG. 8 is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0142] As shown in FIG. 8, the electronic device 800 can include a processing device (for example, a central processor, a graphics processor, and the like) 801, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage device 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An Input / Output (I / O) interface 805 is also connected to the bus 804.

[0143] In general, the following devices can be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 807 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 808 including, for example, a tape, a hard disk, and the like; and communication devices 809. The communication devices 809 can allow the electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. While FIG. 8 shows the electronic device 800 with various devices, it is understood that all of the shown devices are not required to be implemented or possessed. More or less devices can be alternatively implemented or possessed.

[0144] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 809, or installed from the storage devices 808, or installed from the ROM 802. When the computer program is executed by the processing devices 801, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0145] It should be noted that the computer-readable medium in the above disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0146] The computer-readable medium described above can be contained in the electronic device described above; or can exist separately and not be assembled into the electronic device.

[0147] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0148] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0149] The flow diagrams and the block diagrams in the drawings are meant as possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0150] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0151] The functions described above in the specification of the present disclosure can be performed by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0152] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] In a first aspect, according to one or more embodiments of the present disclosure, an image processing method is provided, comprising:

[0154] In response to a triggering operation of a user in a target application, obtaining a to-be-processed image, wherein the to-be-processed image is collected by an image collection device in a terminal device, and the to-be-processed image at least includes a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image;

[0155] Based on the to-be-processed image, determining a target region in the preview image, wherein the target region includes a content subject in the preview image;

[0156] Performing background blurring on the preview image;

[0157] Based on the target region and the preview image after background blurring, obtaining a target image;

[0158] Displaying the target image in the viewfinder.

[0159] According to one or more embodiments of the present disclosure, the to-be-processed image further includes key point information associated with a display object in the preview image;

[0160] The determining the target region in the preview image based on the to-be-processed image comprises:

[0161] Based on the depth map and / or the key point information, determining at least one content subject in the preview image;

[0162] Based on a display position of the content subject, performing image segmentation operation on the preview image to obtain the target region.

[0163] According to one or more embodiments of the present disclosure, the determining at least one content subject in the preview image based on the depth map and / or the key point information comprises:

[0164] determining, according to the depth map, at least one display object in the preview image that satisfies a first condition with the image capturing device;

[0165] determining, based on the key point information, display sizes of the display objects, and determining, in the at least one display object, a target object that satisfies a preset second condition in display size;

[0166] performing a comparison operation on the display size of the target object and display sizes of other display objects in the preview image respectively, to obtain a comparison result;

[0167] determining a content subject in the preview image according to the comparison result.

[0168] According to one or more embodiments of the present disclosure, the determining a content subject in the preview image according to the comparison result comprises:

[0169] for each display object, if a size ratio between the display size of the target object and the display size of the display object is within a preset first range, determining, based on the depth map, whether a depth difference between the display object and the target object is within a preset second range;

[0170] if yes, determining the target object and the display object as the content subject;

[0171] if no, determining the target object as the content subject.

[0172] According to one or more embodiments of the present disclosure, the determining a content subject in the preview image according to the comparison result comprises:

[0173] for each display object, if a size ratio between the display size of the target object and the display size of the display object is not within a preset first range, determining the target object as the content subject.

[0174] According to one or more embodiments of the present disclosure, the determining at least one content subject in the preview image based on the depth map and / or the key point information comprises:

[0175] determining, according to the depth map, depth information corresponding to each display object in the preview image respectively;

[0176] determining, as the at least one content subject, at least one display object whose depth information is within a preset depth range.

[0177] According to one or more embodiments of the present disclosure, the image segmentation operation is performed on the preview image based on the display position of the content subject to obtain the target region, including:

[0178] Determine the key point information associated with each content subject;

[0179] Determine the target mask corresponding to the content subject according to the key point information;

[0180] Perform an image segmentation operation on the preview image through the target mask to obtain the target region.

[0181] According to one or more embodiments of the present disclosure, the target region in the preview image is determined according to the image to be processed, including:

[0182] Perform an image segmentation operation on the preview image through a preset foreground and background segmentation algorithm in the terminal device to obtain the target region in the preview image.

[0183] According to one or more embodiments of the present disclosure, the background blurring operation is performed on the preview image, and a target image is obtained based on the target region and the preview image after background blurring, including:

[0184] Perform a Gaussian blur operation on the preview image to obtain the preview image after background blurring;

[0185] The target image is obtained based on the target region and the preview image after background blurring, including:

[0186] Cover the segmented target region in the corresponding position in the preview image after background blurring to obtain the target image.

[0187] According to one or more embodiments of the present disclosure, the method further includes:

[0188] In response to a user triggering an image acquisition operation, store the target image displayed in the viewfinder to a preset storage path.

[0189] In a second aspect, according to one or more embodiments of the present disclosure, an image processing apparatus is provided, including:

[0190] An acquisition module, in response to a triggering operation of a user in a target application, acquires an image to be processed, wherein the image to be processed is acquired by an image acquisition device in a terminal device, and the image to be processed at least includes a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image.

[0191] determining, based on the image to be processed, a target region in the preview image, wherein the target region comprises a content subject in the preview image;

[0192] processing the preview image to perform background blurring;

[0193] generating a target image based on the target region and the preview image after the background blurring;

[0194] displaying the target image in the viewfinder.

[0195] According to one or more embodiments of the present disclosure, the image to be processed further comprises key point information associated with display objects in the preview image;

[0196] The determining module is configured to:

[0197] determine at least one content subject in the preview image based on the depth map and / or the key point information;

[0198] perform image segmentation on the preview image based on a display position of the content subject to obtain the target region.

[0199] According to one or more embodiments of the present disclosure, the determining module is configured to:

[0200] determine at least one display object in the preview image that satisfies a first condition with the image capturing device according to the depth map;

[0201] determine a target object that satisfies a preset second condition in the at least one display object based on the key point information and a display size of each display object;

[0202] perform a comparison operation on the display size of the target object and display sizes of other display objects in the preview image to obtain a comparison result;

[0203] determine a content subject in the preview image according to the comparison result.

[0204] According to one or more embodiments of the present disclosure, the determining module is configured to:

[0205] for each display object, if a size ratio between the display size of the target object and the display size of the display object is within a preset first range, determine whether a depth difference between the display object and the target object is within a preset second range based on the depth map;

[0206] if yes, determine the target object and the display object as the content subject.

[0207] If not, the target object is determined as the content subject.

[0208] According to one or more embodiments of the present disclosure, the determining module is configured to:

[0209] For each display object, if a size ratio between a display size of the target object and a display size of the display object is not within a preset first range, the target object is determined as the content subject.

[0210] According to one or more embodiments of the present disclosure, the determining module is configured to:

[0211] According to the depth map, depth information corresponding to each display object in the preview image is determined respectively;

[0212] At least one display object with the depth information within a preset depth range is determined as the at least one content subject.

[0213] According to one or more embodiments of the present disclosure, the determining module is configured to:

[0214] Key point information associated with each content subject is determined;

[0215] According to the key point information, a target mask corresponding to the content subject is determined;

[0216] The preview image is subjected to an image segmentation operation through the target mask, and a target region is obtained.

[0217] According to one or more embodiments of the present disclosure, the determining module is configured to:

[0218] The preview image is subjected to an image segmentation operation through a preset foreground and background segmentation algorithm in the terminal device, and a target region in the preview image is obtained.

[0219] According to one or more embodiments of the present disclosure, the processing module is configured to:

[0220] The preview image is subjected to a Gaussian blur operation, and a preview image after background blurring is obtained.

[0221] The generating module is configured to:

[0222] The segmented target region is overlaid on a corresponding position in the preview image after background blurring, and the target image is obtained.

[0223] According to one or more embodiments of the present disclosure, the method further includes:

[0224] The acquisition module is used to respond to the image acquisition operation triggered by the user and store the target image displayed in the viewfinder to a preset storage path.

[0225] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0226] The memory stores computer-executed instructions;

[0227] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the image processing method as described in the first aspect and various possible designs of the first aspect.

[0228] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, which, when executed by a processor, implement the image processing method described in the first aspect and various possible designs of the first aspect.

[0229] Fifthly, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the image processing method as described in the first aspect and various possible designs of the first aspect.

[0230] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0231] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0232] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. An image processing method, comprising: in response to a triggering operation of a user in a target application, obtaining a to-be-processed image, wherein the to-be-processed image is captured by an image capturing device in a terminal device, and the to-be-processed image at least includes a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image; determining a target region in the preview image based on the to-be-processed image, wherein the target region includes a content subject in the preview image; performing background blurring on the preview image; obtaining a target image based on the target region and the preview image after the background blurring; and displaying the target image in the viewfinder.

2. The method of claim 1, wherein the to-be-processed image further includes key point information associated with a display object in the preview image; and the determining the target region in the preview image based on the to-be-processed image comprises: determining at least one content subject in the preview image based on the depth map and / or the key point information; and performing image segmentation on the preview image based on a display position of the content subject to obtain the target region.

3. The method of claim 2, wherein the determining the at least one content subject in the preview image based on the depth map and / or the key point information comprises: determining at least one display object in the preview image that satisfies a first condition with the image capturing device according to the depth map; determining a target object that satisfies a preset second condition in the at least one display object based on display sizes of the display objects according to the key point information; performing a comparison operation on the display size of the target object and display sizes of other display objects in the preview image to obtain a comparison result; and determining the content subject in the preview image according to the comparison result.

4. The method of claim 3, wherein the determining the content subject in the preview image according to the comparison result comprises: for each display object, if a size ratio between the display size of the target object and the display size of the display object is within a preset first range, determining whether a depth difference between the target object and the display object is within a preset second range based on the depth map; if yes, determining the target object and the display object as the content subject; and if no, determining the target object as the content subject.

5. The method of claim 3, wherein the determining the content subject in the preview image according to the comparison result comprises: for each display object, if a size ratio between the display size of the target object and the display size of the display object is not within a preset first range, determining the target object as the content subject.

6. The method of claim 2, wherein the determining the at least one content subject in the preview image based on the depth map and / or the key point information comprises: determining depth information corresponding to each display object in the preview image based on the depth map, respectively. The at least one display object in the preset depth range is determined as the at least one content subject based on the depth information.

7. The method of any one of claims 2-6, wherein the image segmentation operation is performed on the preview image based on the display position of the content subject to obtain the target region, including: determining key point information associated with each content subject; determining a target mask corresponding to the content subject based on the key point information; performing an image segmentation operation on the preview image based on the target mask to obtain the target region.

8. The method of claim 1, wherein the target region in the preview image is determined based on the to-be-processed image, including: performing an image segmentation operation on the preview image based on a preset foreground and background segmentation algorithm in the terminal device to obtain the target region in the preview image.

9. The method of any one of claims 1-6 or 8, wherein the background blurring of the preview image includes: performing a Gaussian blur operation on the preview image to obtain a background-blurred preview image; the target image is obtained based on the target region and the background-blurred preview image, including: overlaying the segmented target region on a corresponding position in the background-blurred preview image to obtain the target image.

10. The method of any one of claims 1-6 or 8, further comprising: in response to a user triggering an image capturing operation, storing the target image displayed in the viewfinder to a preset storage path.

11. An image processing apparatus, comprising: an acquisition module configured to acquire a to-be-processed image in response to a user triggering operation in a target application, wherein the to-be-processed image is captured by an image capturing device in a terminal device, and the to-be-processed image at least includes a preview image currently displayed in a viewfinder of the terminal device and a depth map associated with the preview image; a determination module configured to determine a target region in the preview image based on the to-be-processed image, wherein the target region includes a content subject in the preview image; a processing module configured to perform background blurring on the preview image; a generation module configured to obtain a target image based on the target region and the background-blurred preview image; a display module configured to display the target image in the viewfinder.

12. An electronic device comprising: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method of any one of claims 1-10.

13. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the image processing method of any one of claims 1-10 is implemented.

14. A computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the image processing method of any one of claims 1-10.

Images