Image processing method, computer program product and electronic device

Abstract

This application provides an image processing method, a computer program product, and an electronic device. One specific embodiment of the method includes: acquiring a clear image and a blurred image of the same scene; wherein the clear image and the blurred image are continuously captured by the same camera of the same shooting device within a target time period; determining a partial image of a subject corresponding to the clear image and a partial image of a background corresponding to the blurred image; and fusing the clear image and the blurred image based on the partial image of the subject and the partial image of the background to obtain a target image with a blurred background. The resulting target image exhibits a relatively realistic background blur effect.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image processing method, a computer program product, and an electronic device. Background Technology

[0002] Background blurring is achieved by reducing the depth of field, thus focusing the attention on the subject. It can generally be achieved using a professional camera with a large aperture lens. This results in a blurred background in the image, where areas other than the main subject are blurred.

[0003] In background blurring, an important concept is "depth of field (DOF)." It refers to the range of distances in front of and behind the subject that allow for a sharp image to be captured by the camera lens or other imaging device. Depth of field can be calculated using formulas... The calculation yields the result. Here, f represents the lens focal length, N represents the aperture value, U represents the distance between the object and the lens, and C represents the diameter of the circle of confusion. From the formula, we know that when the size of the circle of confusion is constant (i.e., the C value remains unchanged), the following conditions must be met to achieve a background blur effect: 1) The distance between the subject and the lens is relatively short, i.e., the U value is sufficiently small; 2) The focal length is relatively long, i.e., the f value is sufficiently large; 3) The aperture is large, i.e., the N value is sufficiently large. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method, computer program product, and electronic device for obtaining a target image with a more realistic background blurring effect.

[0005] In a first aspect, embodiments of this application provide an image processing method, which includes: acquiring a clear image and a blurred image in the same scene; wherein the clear image and the blurred image are continuously captured by the same camera of the same shooting device within a target time period; determining a subject partial image corresponding to the clear image and a background partial image corresponding to the blurred image; and fusing the clear image and the blurred image based on the subject partial image and the background partial image to obtain a target image with background blur. Thus, the blurring effect of the target image is relatively realistic.

[0006] Optionally, before determining the subject partial image corresponding to the clear image and the background partial image corresponding to the blurred image, the image processing method further includes: acquiring difference information between the clear image and the blurred image; and adjusting the clear image or the blurred image according to the difference information. In this way, by acquiring the difference information between the two images and specifically reducing the differences corresponding to the difference information, the two images can be made highly similar, facilitating the obtaining of a more realistic and complete target image.

[0007] Optionally, the difference information includes scale difference information corresponding to the same object in the scene; and the acquisition of difference information between the clear image and the blurred image includes: acquiring the clear image distance and blurred image distance corresponding to the shooting device when shooting the clear image and the blurred image, respectively; the adjustment of the clear image or the blurred image according to the difference information includes: adjusting the size of the clear image or the blurred image according to the clear image distance and the blurred image distance to reduce the scale difference between the clear image and the blurred image. In this way, the scale difference between images can be reduced by the clear image distance and the blurred image distance, so that the clear image and the blurred image can be highly similar.

[0008] Optionally, adjusting the size of the sharp image or the blurred image based on the sharp image distance and the blurred image distance includes:

[0009] Based on the clear image distance and the blurred image distance, the size of the blurred image is adjusted to obtain an intermediate blurred image; and after fusing the clear image and the blurred image based on the subject local image and the background local image to obtain a background-blurred target image, the image processing method further includes: cropping the target image based on the intermediate blurred image to obtain an image completely covered by the intermediate blurred image. In this way, the target image obtained after fusion can be cropped based on the intermediate blurred image to remove non-overlapping image areas, resulting in an image completely covered by the intermediate blurred image, thereby making the final image imaging effect more complete.

[0010] Optionally, the difference information includes pose difference information corresponding to the same object in the scene; and the acquisition of difference information between the clear image and the blurred image includes: acquiring clear pose information and blurred pose information corresponding to the shooting device when capturing the clear image and the blurred image respectively; the adjustment of the clear image or the blurred image according to the difference information includes: adjusting the pose of the same object in the clear image or the blurred image according to the clear pose information and the blurred pose information to reduce the pose difference between the clear image and the blurred image. In this way, the pose difference between images can be reduced by using clear pose information and blurred pose information, so that the clear image and the blurred image can be highly similar.

[0011] Optionally, the clear image and the blurred image are captured by the same camera of the same shooting device within a target duration based on the following steps: capturing the clear image while in focus; adjusting the image plane of the shooting device away from the lens group of the camera, and capturing the blurred image when the target duration is reached, with the moment the clear image is captured as the starting point. This allows for the rapid and accurate capture of highly similar clear and blurred images.

[0012] Optionally, determining the subject local image corresponding to the clear image and the background local image corresponding to the blurred image includes: performing semantic segmentation processing on the clear image to mark the subject image region corresponding to the subject and the background image region corresponding to the background in the clear image; determining the subject image region as the subject local image, and determining the image in the blurred image corresponding to the background image region as the background local image; or, determining the subject local image corresponding to the clear image and the background local image corresponding to the blurred image includes: obtaining a depth image corresponding to the clear image and determining the pixel value corresponding to each pixel in the depth image; determining the pixel region corresponding to the subject in the clear image as the subject local image, and determining the image in the blurred image corresponding to other pixel regions in the clear image as the background local image. Here, two methods for determining the subject local image and the background local image are provided.

[0013] Secondly, embodiments of this application provide a computer program product, including computer program instructions, which are read and executed by a processor to perform the steps in the method provided in the first aspect above.

[0014] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method provided in the first aspect above are performed.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0016] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram showing the depth of field range corresponding to different aperture imaging in the embodiments of this application;

[0019] Figure 2 This is a schematic diagram of the depth of field formation corresponding to the shooting subject involved in the embodiments of this application;

[0020] Figure 3 This is a schematic diagram illustrating the formation principle of the large aperture and shallow depth of field effect involved in the embodiments of this application;

[0021] Figure 4 This is a schematic diagram illustrating the formation principle of the small aperture and large depth of field effect in the embodiments of this application;

[0022] Figure 5 A flowchart illustrating an image processing method provided in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of an application scenario of the imaging process under accurate focusing conditions, as described in the embodiments of this application.

[0024] Figure 7 This is a schematic diagram of an application scenario of the imaging process under defocusing conditions involved in the embodiments of this application;

[0025] Figure 8 This is a schematic diagram of an application scenario involving the use of two cameras to capture images, as described in an embodiment of this application.

[0026] Figure 9 This is a schematic diagram of an application scenario involving the use of a single camera to capture images, as described in an embodiment of this application.

[0027] Figure 10 This is a schematic diagram illustrating the imaging differences of the same object on different image planes using the same camera in an embodiment of this application.

[0028] Figure 11 A structural block diagram of an image processing apparatus provided in an embodiment of this application;

[0029] Figure 12 This is a schematic diagram of the structure of an electronic device for performing an image processing method, provided as an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0031] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] It should be noted that, unless otherwise specified, the embodiments or technical features in the embodiments of this application may be combined.

[0033] In related technologies, portable shooting devices (such as mobile phones) are limited by their size, resulting in small physical focal lengths and apertures. Furthermore, in order to capture the full view of the subject, portable shooting devices cannot take close-up shots of the subject. Therefore, images taken with portable shooting devices generally do not have a background blur effect.

[0034] Please see Figure 1 It shows a schematic diagram of the depth of field range corresponding to imaging at different apertures. For example... Figure 1 As shown, when the allowable circle of confusion diameter is the same, the depth of field range corresponding to the aperture being fully open (i.e., a large aperture) is smaller than the depth of field range corresponding to the aperture being stopped down (i.e., a small aperture).

[0035] Please see Figure 2 It shows a diagram illustrating the depth of field formation corresponding to the subject being photographed; such as Figure 2 As shown, clear imaging results can be obtained when the image sensor of the shooting device is within the depth of focus range.

[0036] Please see Figure 3 It illustrates the principle behind the formation of a shallow depth of field effect (background blur effect) with a large aperture; such as Figure 3As shown, when the subject is object A, object B, which is outside the depth of field, is imaged in front of the image plane of the shooting device, and its image point diffuses into a light spot in front of the image plane. Thus, the resulting image is a background-blurred image where the subject A is clear, but the background B is blurred.

[0037] Please see Figure 4 It illustrates the principle behind the formation of a large depth of field effect with a small aperture; such as Figure 4 As shown, portable shooting devices such as mobile phones and cameras generally have a large depth of field. Therefore, both the subject A and the background B can be captured with relatively clear images. That is, in this case, a blurred background image cannot be obtained.

[0038] In related technologies, portable shooting devices often need to utilize depth information of the shooting scene in order to capture images with blurred backgrounds. For example, in Figure 4 In this method, the distances of object A and background B relative to the lens can be obtained through the sensor. Then, based on the distances of the two objects relative to the lens, the image B2 of background B on the image plane is blurred to achieve a background blur image where the subject A is clear and the background B is blurred. This method of obtaining a background blur image has the following drawbacks: (1) After imaging with a large aperture lens, point light sources in the scene can present circular light spots on the image plane. However, there are no light spots in the image after blurring, so it is necessary to apply a texture to the point light source before blurring to simulate the light spot effect. In this way, if the light spot is applied incorrectly during the texture application, the correct light spot effect cannot be simulated. (2) For scenes with a solid color background, the background blur image obtained after blurring is not realistic, resulting in poor visual quality. (3) The degree of blur is determined by the depth information of the shooting scene, which has the problem of unrealistic image layering and is prone to errors.

[0039] Therefore, the background blurring images obtained using related technologies suffer from unrealistic blurring effects. To address this issue, this application provides an image processing method, a computer program product, and an electronic device. Furthermore, by capturing a clear image and a blurred image consecutively within a short period using the same camera, the two images are then fused to obtain the corresponding background blurring image. In this way, the blurred image is obtained by altering the image plane of the shooting device, more closely resembling the imaging process of a camera with a true large aperture. Therefore, point light sources in the original scene will diffuse into circular light spots, and the background hierarchy in the original scene will still be blurred to varying degrees on the image plane, resulting in a more realistic blurring effect. Moreover, the blurred image originates entirely from the real scene, thus avoiding the post-processing of applying light spots and image blurring, and eliminating the phenomenon of incorrectly applying light spots. This solves the aforementioned problems.

[0040] In some application scenarios, the image processing method of this application can be applied to a shooting device, allowing the shooting device to quickly obtain a background-blurred image after shooting. In other application scenarios, the image processing method of this application can also be applied to an image processing device, allowing a clear image and a blurred image to be input into the image processing device and fused to obtain the corresponding background-blurred image. Exemplarily, this application is described in the context of application to a shooting device.

[0041] The defects in the solutions in the above-mentioned related technologies are all the result of the inventors' practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present invention in the following text should be the inventors' contributions to the present invention.

[0042] Please refer to Figure 5 This illustrates a flowchart of a first image processing method provided in an embodiment of this application. For example... Figure 5 As shown, the image processing method includes steps 501 to 503.

[0043] Step 501: Acquire a clear image and a blurry image of the same scene; wherein the clear image and the blurry image are continuously captured by the same camera of the same shooting device within the target time period;

[0044] In some alternative implementations, the clear image and the blurred image can be obtained by the same camera of the same shooting device within a target duration based on the following steps: first, in a focused state, the clear image is captured; then, the image plane of the shooting device is adjusted in a direction away from the lens group of the camera, and the blurred image is captured when the target duration is reached, with the time when the clear image is captured as the starting point.

[0045] In some applications, the shooting equipment can be pre-configured to quickly and accurately capture highly similar sharp and blurred images. Specifically, a sharp image can be captured in focus, and then the shooting equipment can be quickly switched to out-of-focus mode to capture a blurred image within a target time period, so that the two images are highly similar in content except for sharpness.

[0046] In some applications, the camera can capture clear images when the focus is accurate. For example... Figure 6 As shown, for objects A and B in the scene, when the focus is accurate, since both are imaged on the image plane of the shooting device, the captured images A2 and B2 are relatively clear.

[0047] Furthermore, if the focus is incorrect, that is, if the shot is taken out of focus, a blurry image will be obtained. For example... Figure 7 As shown, for objects A and B in the scene, when out of focus, since both are imaged in front of the image plane of the shooting device, the captured images A2 and B2 are relatively blurry.

[0048] In some applications, to ensure that the content of clear and blurry images is highly similar, clear and blurry images can be captured continuously using the same camera on the same shooting device within a target duration. Here, the target duration can include short durations such as 0.5 seconds, 0.3 seconds, or 0.2 seconds.

[0049] Step 502: Determine the subject local image corresponding to the clear image and the background local image corresponding to the blurred image;

[0050] After acquiring both a sharp image and a blurred image of the same scene, the local image of the subject corresponding to the sharp image can be determined. In some applications, for example, the object in focus can be identified as the subject of the image. Therefore, the local image corresponding to that object can be considered the subject's local image.

[0051] Furthermore, the background local image corresponding to the blurred image can be determined. In some applications, the background local image can be determined using the determined subject local image. That is, the other areas of the blurred image excluding the subject local image can be regarded as the background local image.

[0052] In some optional implementations, step 502 above may include: performing semantic segmentation processing on the clear image to mark the subject image region corresponding to the subject in the clear image and the background image region corresponding to the background; determining the subject image region as the subject local image, and determining the image in the blurred image corresponding to the background image region as the background local image.

[0053] In other words, semantic segmentation can be performed on sharp images to classify and label each pixel, and the subject and background local images can be determined based on the labeling results. For example, when photographing a person, semantic segmentation can be performed on the resulting sharp image to label the pixel regions corresponding to the person and the pixel regions corresponding to the background. Then, the pixel regions corresponding to the person can be considered the subject local image, and the pixel regions in the blurred image corresponding to the background in the sharp image can be considered the background local image.

[0054] In some alternative implementations, determining the subject local image corresponding to the clear image and the background local image corresponding to the blurred image may also include: obtaining a depth image corresponding to the clear image and determining the pixel value corresponding to each pixel in the depth image; determining the pixel area corresponding to the subject in the clear image as the subject local image, and determining the image in the blurred image corresponding to other pixel areas in the clear image as the background local image.

[0055] In other words, it is possible to obtain the depth image corresponding to the clear image, and obtain the pixel value corresponding to each pixel point on the depth image, and then obtain the distance of each pixel in the clear image relative to the lens.

[0056] In some applications, the subject corresponding to the clear image can be identified first. Then, based on the pixel value range corresponding to the subject (i.e., the distance from each point in the scene where the clear image was captured to the lens), the pixel regions within that range can be defined as the subject's partial image, while the pixel regions in the blurred image corresponding to other pixels in the clear image can be defined as the background's partial image. For example, if the distance from each point on the subject to the lens is between 1500mm and 3000mm, then the pixel regions within this range can be defined as the subject's partial image, and the pixel regions in the blurred image corresponding to pixels outside this range in the clear image can be defined as the background's partial image.

[0057] Step 503: Based on the subject partial image and the background partial image, fuse the clear image and the blurred image to obtain a target image with a blurred background.

[0058] After determining the subject and background partial images, the sharp and blurred images can be merged. During merging, the subject partial image can be used in the subject area of ​​the target image, and the background partial image can be used in the background area. This results in a blurred background image with a sharp subject.

[0059] In this embodiment, by going through steps 501 to 503 above, a target image with a more realistic background blurring effect can be obtained.

[0060] In some application scenarios, to achieve a smoother transition between partial images of the subject and the background, the image region at the transition point can be blurred. Furthermore, since this embodiment achieves a blurred image without requiring blurring, it eliminates the need for blurring steps and simplifies the processing flow.

[0061] In addition, to obtain a blurred background image, some related technologies sometimes use two cameras from the same shooting device to capture the same scene. For example... Figure 8 As shown, when cameras 1 and 2, which are not at the same shooting angle, capture images of objects A and B in the same scene, the image captured by camera 1 includes both objects A and B. However, the image captured by camera 2 only includes object B because object A is obscured by object B. Therefore, when using two cameras to capture the same scene, there is a situation where the image content is obscured, resulting in an incomplete image of the merged target image.

[0062] like Figure 9 As shown, this embodiment adjusts the image plane corresponding to the same camera, so that there is no problem of image content being occluded in the clear image and the blurred image of the same scene (that is, both images include object A and object B), so that the image content of the fused target image is complete and the background blur effect is realistic.

[0063] In some alternative implementations, prior to step 502 above, the image processing method further includes the following steps:

[0064] Step 1: Obtain the difference information between the clear image and the blurred image;

[0065] In some applications, due to dynamic changes in the scene, sharp and blurred images captured at different times cannot be completely identical. Therefore, it is possible to minimize the difference between the two to obtain a more realistic background blur image.

[0066] Therefore, we can first obtain the difference information between the clear image and the blurry image, and then reduce the corresponding difference in a targeted manner based on this difference information.

[0067] Step 2: Adjust the clear image or the blurry image according to the difference information.

[0068] In some applications, after obtaining the difference information between a clear image and a blurry image, the clear image or the blurry image can be adjusted based on this difference information. That is, if the clear image is used as the reference, the blurry image can be adjusted; if the blurry image is used as the reference, the clear image can be adjusted.

[0069] In this way, by acquiring the difference information between two images and selectively reducing the difference corresponding to the difference information, the two images can be made highly similar, making it easier to obtain a more realistic and complete target image.

[0070] In some optional implementations, the difference information includes scale difference information corresponding to the same object in the scene; and step 1 above may include: obtaining the clear image distance and blur image distance corresponding to the shooting device when shooting the clear image and the blur image, respectively;

[0071] Please see Figure 10 This illustrates the differences in the imaging of the same object on different image planes using the same camera. For example... Figure 10 As shown, since sharp and blurry images are imaged on image planes at different distances, there are scale differences in the corresponding image content (e.g., Figure 10 As shown, the image on image plane 1 corresponding to the sharp image is smaller than the image on image plane 2 corresponding to the blurred image. To reduce scale differences, the sharp image distance F1 and the blurred image distance F2 corresponding to the sharp image and the blurred image can be obtained respectively.

[0072] Thus, step 2 above may include: adjusting the size of the sharp image or the blurry image based on the sharp image distance and the blurry image distance, so as to reduce the scale difference between the sharp image and the blurry image.

[0073] After obtaining the sharp image distance and the blurred image distance, the size of the sharp image or the blurred image can be adjusted to reduce the scale difference of the same object in the sharp image and the blurred image. For example, the average image distance corresponding to the two image distances can be taken, and the blurred image can be reduced or the sharp image can be enlarged based on the average image distance, so that the sharp image and the blurred image can be highly similar.

[0074] In some alternative implementations, adjusting the size of the sharp image or the blurry image based on the sharp image distance and the blurry image distance may include: determining the image distance ratio between the sharp image distance and the blurry image distance; and reducing the size of the blurry image or enlarging the sharp image based on the image distance ratio to reduce the scale difference between the sharp image and the blurry image.

[0075] In some applications, the image distance ratio between the sharp image distance and the blurry image distance can be determined to identify the scale difference of the same object in the sharp and blurry images. Subsequently, based on this image distance ratio, the blurry image can be scaled up or the sharp image can be enlarged. This ensures that the scaled blurry image matches the scale of the same object in the original sharp image, or the enlarged sharp image matches the scale of the same object in the original blurry image.

[0076] In some optional implementations, adjusting the size of the sharp image or the blurred image based on the sharp image distance and the blurred image distance includes: adjusting the size of the blurred image based on the sharp image distance and the blurred image distance to obtain an intermediate blurred image.

[0077] After adjusting the size of the blurred image, an intermediate blurred image with a smaller scale difference from the clear image can be obtained.

[0078] Thus, after step 503, the image processing method may further include the following step 504:

[0079] Based on the intermediate blurred image, the target image is cropped to obtain an image completely covered by the intermediate blurred image.

[0080] In some applications, after reducing the scale difference between a sharp image and a blurry image, the two images still differ in size. Therefore, the two images cannot be perfectly superimposed.

[0081] Next, the clear and blurred images can be fused to obtain a fused target image. This target image is then processed to obtain a more complete final image. Specifically, after obtaining the intermediate blurred image, the fused target image can be cropped based on this image to remove non-overlapping image areas, resulting in a final image completely covered by the intermediate blurred image, thus making the final image more complete.

[0082] In some optional implementations, the difference information includes pose difference information corresponding to the same object in the scene; and step 1 above may include: obtaining clear pose information and blur pose information corresponding to the shooting device when shooting the clear image and the blurry image respectively;

[0083] In some applications, camera shake can cause pose differences between the same object in sharp and blurry images. Therefore, to reduce this pose difference, it's possible to acquire the pose information of the camera when capturing a sharp image, and the pose information of the camera when capturing a blurry image. For example, a gyroscope can be used to collect the pose information of the camera.

[0084] After obtaining the pose information corresponding to the clear image and the blurry image captured by the shooting device, the pose information corresponding to the clear image can be regarded as the aforementioned clear pose information, and the pose information corresponding to the blurry image can be regarded as the aforementioned blurry pose information.

[0085] Thus, step 2 above may include: adjusting the pose of the same object in the clear image or the blurry image based on the clear pose information and the blurry pose information, so as to reduce the pose difference between the clear image and the blurry image.

[0086] After obtaining both sharp and blurred pose information, the pose differences corresponding to the same object in the two images can be adjusted accordingly. For example, the average pose information between the sharp and blurred pose information can be determined, and the two images can be transformed towards this average pose information to make the two images highly similar.

[0087] In some optional implementations, adjusting the pose of the same object in the clear image or the blurry image based on the clear pose information and the blurry pose information to reduce the pose difference between the clear image and the blurry image may include: obtaining the rotation angle corresponding to the same object in the scene based on the clear pose information and the blurry pose information; and performing an angle transformation based on the rotation angle to reduce the pose difference between the clear image and the blurry image.

[0088] In some application scenarios, the pose difference between sharp pose information and fuzzy pose information can be determined. For example, when the sharp pose information is valued at G1 and the fuzzy pose information is valued at G2, the pose difference between the two can be (G2-G1); this pose difference can be regarded as the aforementioned rotation angle.

[0089] After obtaining the rotation angle, a transformation can be performed, for example, according to the Rodrigues rotation formula (referred to as Rodrigues transformation), which can reduce the pose difference between sharp and blurry images.

[0090] Please refer to Figure 11 This diagram illustrates a structural block diagram of an image processing apparatus according to an embodiment of this application. The image processing apparatus may be a module, program segment, or code on an electronic device. It should be understood that this apparatus is similar to the one described above. Figure 5 The method implementation corresponds to this and can be executed. Figure 5 The specific functions of the device involved in the method embodiments can be found in the description above. To avoid repetition, detailed descriptions are omitted here.

[0091] Optionally, the image processing device includes an acquisition module 1101, a determination module 1102, and a fusion module 1103. The acquisition module 1101 acquires a clear image and a blurred image in the same scene; wherein the clear image and the blurred image are continuously captured by the same camera of the same shooting device within a target time period; the determination module 1102 determines the subject partial image corresponding to the clear image and the background partial image corresponding to the blurred image; the fusion module 1103 fuses the clear image and the blurred image based on the subject partial image and the background partial image to obtain a target image with a blurred background.

[0092] Optionally, the image processing device further includes a difference reduction module, which is used to: obtain difference information between the clear image and the blurry image before determining the subject local image corresponding to the clear image and the background local image corresponding to the blurry image; and adjust the clear image or the blurry image according to the difference information.

[0093] Optionally, the difference information includes scale difference information corresponding to the same object in the scene; and the difference reduction module is further configured to: obtain the clear image distance and blurry image distance corresponding to the shooting device when shooting the clear image and the blurry image respectively; and adjust the size of the clear image or the blurry image according to the clear image distance and the blurry image distance to reduce the scale difference between the clear image and the blurry image.

[0094] Optionally, the difference reduction module is further configured to: adjust the size of the blurred image according to the clear image distance and the blurred image distance to obtain an intermediate blurred image; and the image processing device further includes a cropping module, which is configured to: after fusing the clear image and the blurred image according to the subject local image and the background local image to obtain a background-blurred target image, crop the target image according to the intermediate blurred image to obtain an image completely covered by the intermediate blurred image.

[0095] Optionally, the difference information includes pose difference information corresponding to the same object in the scene; and the difference reduction module is further configured to: acquire clear pose information and blur pose information corresponding to the shooting device when shooting the clear image and the blur image respectively; adjust the pose of the same object in the clear image or the blur image according to the clear pose information and the blur pose information, so as to reduce the pose difference between the clear image and the blur image.

[0096] Optionally, the clear image and the blurry image are obtained by the same camera of the same shooting device within a target duration based on the following steps: in a focused state, the clear image is captured; the image plane of the shooting device is adjusted in a direction away from the lens group of the camera, and the blurry image is captured when the target duration is reached, with the time when the clear image is captured as the starting point.

[0097] Optionally, the determining module 1102 is further configured to: perform semantic segmentation processing on the clear image, marking the subject image region corresponding to the subject in the clear image and the background image region corresponding to the background; determine the subject image region as the subject local image, and determine the image in the blurred image corresponding to the background image region as the background local image. Alternatively, the determining module 1102 is further configured to: obtain the depth image corresponding to the clear image, and determine the pixel value corresponding to each pixel in the depth image; determine the pixel region corresponding to the subject in the clear image as the subject local image, and determine the image in the blurred image corresponding to other pixel regions in the clear image as the background local image.

[0098] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0099] Please refer to Figure 12 , Figure 12 This is a schematic diagram of an electronic device for executing an image processing method, provided in an embodiment of this application. The electronic device may include: at least one processor 1201, such as a CPU, at least one communication interface 1202, at least one memory 1203, and at least one communication bus 1204. The communication bus 1204 is used to establish direct communication between these components. In this embodiment, the communication interface 1202 is used for signaling or data communication with other node devices. The memory 1203 may be a high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1203 may also be at least one storage device located remotely from the aforementioned processor. The memory 1203 stores computer-readable instructions. When these computer-readable instructions are executed by the processor 1201, the electronic device can perform the aforementioned... Figure 5 The method and process are shown.

[0100] Understandable. Figure 12 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 12 The more or fewer components shown, or having the same Figure 12 The different configurations shown. Figure 12 The components shown can be implemented using hardware, software, or a combination thereof.

[0101] This application provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it can perform actions such as... Figure 5 The method process executed by the electronic device in the illustrated method embodiment.

[0102] This application provides a computer program product, which includes computer program instructions. When the computer program instructions are read and executed by a processor, they can perform the methods provided in the above-described method embodiments. For example, the method may include: acquiring a clear image and a blurred image in the same scene; wherein the clear image and the blurred image are continuously captured by the same camera of the same shooting device within a target duration; determining a subject partial image corresponding to the clear image and a background partial image corresponding to the blurred image; and fusing the clear image and the blurred image based on the subject partial image and the background partial image to obtain a target image with a blurred background.

[0103] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0104] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0106] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0107] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Acquire clear and blurry images of the same scene; wherein the clear image and the blurry image are captured continuously by the same camera of the same shooting device within a target time period; Determine the subject local image corresponding to the clear image and the background local image corresponding to the blurred image; Based on the subject partial image and the background partial image, the clear image and the blurred image are fused to obtain a target image with a blurred background. The method further includes, before determining the subject local image corresponding to the clear image and the background local image corresponding to the blurred image: Obtain the difference information between the clear image and the blurred image; wherein, the difference information includes the pose difference information corresponding to the same object in the scene; Adjust the clear image or the blurry image based on the difference information; The step of obtaining the difference information between the clear image and the blurred image includes: Acquire the clear pose information and blur pose information of the shooting device when capturing the clear image and the blurry image, respectively; The step of adjusting the clear image or the blurry image based on the difference information includes: Based on the clear pose information and the blurred pose information, adjust the pose of the same object in the clear image or the blurred image to reduce the pose difference between the clear image and the blurred image; The method of adjusting the pose of the same object in the clear image or the blurry image based on the clear pose information and the blurry pose information to reduce the pose difference between the clear image and the blurry image includes: obtaining the rotation angle corresponding to the same object in the scene based on the clear pose information and the blurry pose information; and performing angle transformation based on the rotation angle to reduce the pose difference between the clear image and the blurry image.

2. The method according to claim 1, characterized in that, The difference information includes scale difference information corresponding to the same object in the scene; as well as The step of obtaining the difference information between the clear image and the blurred image includes: The sharp image distance and blurry image distance of the shooting device are obtained respectively when the sharp image and the blurry image are captured. The step of adjusting the clear image or the blurry image based on the difference information includes: Based on the sharp image distance and the blurry image distance, adjust the size of the sharp image or the blurry image to reduce the scale difference between the sharp image and the blurry image.

3. The method according to claim 2, characterized in that, The step of adjusting the size of the sharp image or the blurry image based on the sharp image distance and the blurry image distance includes: Based on the clear image distance and the blurred image distance, the size of the blurred image is adjusted to obtain an intermediate blurred image; and After fusing the sharp image and the blurred image based on the subject partial image and the background partial image to obtain a target image with a blurred background, the method further includes: Based on the intermediate blurred image, the target image is cropped to obtain an image completely covered by the intermediate blurred image.

4. The method according to claim 1, characterized in that, The clear image and the blurry image were captured by the same camera of the same shooting device within the target time period based on the following steps: In focus mode, the clear image is captured; The image plane of the shooting device is adjusted in a direction away from the lens group of the camera, and the blurred image is captured when the time when the clear image is captured is taken as the starting point of the time.

5. The method according to claim 1, characterized in that, Determining the subject local image corresponding to the clear image and the background local image corresponding to the blurred image includes: The clear image is semantically segmented to mark the subject image region corresponding to the subject being photographed and the background image region corresponding to the background being photographed in the clear image. The main image region is determined as the main local image, and the image in the blurred image corresponding to the background image region is determined as the background local image; or... Determining the subject local image corresponding to the clear image and the background local image corresponding to the blurred image includes: Obtain the depth image corresponding to the clear image, and determine the pixel value corresponding to each pixel in the depth image; The pixel region corresponding to the subject in the clear image is determined as the subject local image, and the image in the blurred image corresponding to other pixel regions in the clear image is determined as the background local image.

6. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-5.

7. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the method as described in any one of claims 1-5.

Images