Background blurring method and apparatus, computer storage medium and terminal

Abstract

A background blurring method and apparatus, a computer storage medium, and a terminal. The background blurring method comprises: acquiring a foreground region and a background region of an original image frame; generating a blur intensity mapping table corresponding to the background region; correcting the blur intensity mapping table on the basis of a focus distance of the original image frame; blurring the background region on the basis of the corrected blur intensity mapping table, to obtain a blurred background image; and obtaining a blur result image on the basis of the blurred background image and the original image frame.

Description

Background blur method, device, computer storage medium and terminal

[0001] This application claims priority to Chinese patent application number 2023109065123, filed on July 21, 2023, entitled “A background blurring method, device, computer storage medium and terminal”, the contents of which should be understood as incorporated into this application by reference. Technical Field

[0002] The present application relates to, but is not limited to, image processing technology, and in particular to a background blur method, device, computer storage medium, and terminal. Background Art

[0003] By processing images or videos using single-camera semantic segmentation or depth prediction technology, a gradual background blur effect similar to that achieved by a large aperture SLR camera can be achieved. In recent years, single-camera background blur technology has become increasingly sophisticated, boasting advantages such as simple hardware and assembly, low cost, fast computational speed, and low power consumption. However, single-camera semantic segmentation can only distinguish between foreground and background, and single-camera depth prediction can only predict the relative depth between objects, i.e., the relative distance between them, but not absolute depth. When using a single camera for background blur processing, it is difficult to adjust the background blur intensity to the appropriate level based on the actual distance of the focused object from the camera. That is, the closer the focused object is to the camera, the stronger the background blur intensity should be, and vice versa.

[0004] In some technologies, methods for background blur processing generally include the following: using dual cameras to obtain a depth map, and using the parallax of the dual cameras to calculate a depth map with an absolute scale; using structured light (Structured Light) or ToF (Time of Flight) cameras to obtain absolute depth; estimating the absolute distance of the object under conditions such as a fixed and known height between the camera and the ground or a known actual height of the object; in portrait scenes, using a face frame, head frame, body frame, etc. to correct the background blur intensity; correcting the background blur intensity by adjusting the physical parameters of the camera lens, such as aperture, focal length, and the distance from the lens to the object.

[0005] In summary, how to achieve background blur processing that is applicable to different scenarios has become a problem to be solved.

[0006] SUMMARY OF THE INVENTION

[0007] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0008] In the first aspect, an embodiment of the present application provides a background blur method, including: obtaining the foreground area and background area of ​​the original image frame; generating a blur intensity mapping table corresponding to the background area; correcting the blur intensity mapping table based on the focus distance of the original image frame; blurring the background area according to the corrected blur intensity mapping table to obtain a blurred background image; obtaining a blurred result image according to the blurred background image and the original image frame.

[0009] In some embodiments, obtaining the foreground area and the background area of ​​the original image frame includes: performing semantic segmentation on the original image frame to obtain the foreground area and the background area; or obtaining a depth map of the original image frame and obtaining the foreground area and the background area based on the depth map.

[0010] In some embodiments, before generating the blur intensity mapping table corresponding to the background area, the background blurring method further includes: acquiring a target segmentation mask of the foreground area.

[0011] In some embodiments, obtaining a blurred result image based on the blurred background image and the original image frame includes: fusing the blurred background image and the original image frame according to the target segmentation mask to obtain the blurred result image.

[0012] In some embodiments, before generating the blur intensity mapping table corresponding to the background area, the background blurring method further includes: determining a blur baseline of the background area, wherein the blur intensity corresponding to the pixels on the blur baseline is a first blur intensity, and the first blur intensity is not higher than a preset blur intensity; obtaining the distance from the pixels of the background area to the blur baseline; dividing the pixels of the background area into a plurality of pixel groups according to the distance; and generating the blur intensity mapping table based on the plurality of pixel groups.

[0013] In some embodiments, dividing the pixels of the background area into multiple pixel groups according to the distance includes: dividing the pixels on the same side of the blur baseline and at the same distance from the blur baseline into the same pixel group, wherein each pixel group corresponds to a blur intensity value.

[0014] In some embodiments, generating the blur intensity mapping table based on the multiple pixel groups includes: determining the blur intensity mapping table according to the blur intensity value corresponding to each of the pixel groups; wherein the blur intensity mapping table corresponds to one or more blur intensity intervals, and each blur intensity interval is 0 to the first blur intensity or the first blur intensity to the preset blur intensity.

[0015] In some embodiments, the correction of the blur intensity mapping table based on the focus distance of the original image frame includes: correcting the focus distance of the original image frame to obtain a corrected focus distance; and correcting the blur intensity mapping table according to the corrected focus distance.

[0016] In some embodiments, the focus distance of the original image frame is corrected to obtain a corrected focus distance, including: when there is no historical image frame before the original image frame, determining the corrected focus distance based on the focus distance of the original image frame; when there is a historical image frame before the original image frame, determining the corrected focus distance based on the focus distance of the original image frame and the focus distance of the historical image frame.

[0017] In some embodiments, determining the corrected focus distance based on the focus distance of the original image frame and the focus distance of the historical image frame includes: judging whether the focus distance of the original image frame is abnormal; if the focus distance of the original image frame is abnormal, using the focus distance of the historical image frame as the corrected focus distance; if the focus distance of the original image frame is normal, smoothing the focus distance of the original image frame and the focus distance of the historical image frame to obtain the corrected focus distance.

[0018] In the second aspect, an embodiment of the present application also provides a background blur device, including: an image processing unit, configured to obtain the foreground area and background area of ​​the original image frame; a blur intensity generation unit, configured to generate a blur intensity mapping table corresponding to the background area; a correction unit, configured to correct the blur intensity mapping table based on the focus distance of the original image frame; a blur processing unit, configured to blur the background area according to the corrected blur intensity mapping table to obtain a blurred background image; a result output unit, configured to obtain a blurred result image based on the blurred background image and the original image frame.

[0019] In a third aspect, an embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by a processor, the background blurring method in any of the above embodiments is implemented.

[0020] In a fourth aspect, an embodiment of the present application further provides a terminal comprising: a memory and a processor, wherein the memory stores a computer program; wherein the processor is configured to execute the computer program in the memory; and when the computer program is executed by the processor, the background blurring method in any of the above embodiments is implemented.

[0021] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0022] Summary of the Figures

[0023] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0024] FIG1 is a flow chart of a background blurring method according to an embodiment of the present application;

[0025] FIG2 a is an original image frame according to an embodiment of the present application;

[0026] FIG2 b is a schematic diagram of a foreground area and a background area of ​​an original image frame according to an embodiment of the present application;

[0027] FIG3 is a structural block diagram of a background blurring device according to an embodiment of the present application.

[0028] Details

[0029] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0030] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0031] This application describes multiple embodiments, but this description is exemplary rather than restrictive, and it will be apparent to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described herein. Although many possible feature combinations are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0032] This application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features, and elements disclosed in this application can be combined with any conventional features or elements to form a unique inventive solution defined by the claims. Any feature or element of any embodiment can be combined with features or elements from other inventive solutions to form another unique inventive solution defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application can be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the appended claims and their equivalents, the embodiments are not subject to other limitations. In addition, various modifications and changes can be made within the scope of protection of the appended claims.

[0033] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other orders of steps are also possible. Therefore, the specific order of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to performing their steps in the order written, and those skilled in the art can readily understand that these orders can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0034] FIG1 is a flow chart of a background blurring method according to an embodiment of the present application. The background blurring method can be implemented by a computer or a hardware unit with an image processing function. As shown in FIG1 , the method includes the following steps:

[0035] Step 101: Acquire the foreground area and background area of ​​the original image frame.

[0036] Step 102: Generate a blur intensity mapping table corresponding to the background area.

[0037] Step 103: Correct the blur intensity mapping table based on the focus distance of the original image frame.

[0038] Step 104: blur the background area according to the corrected blur intensity mapping table to obtain a blurred background image.

[0039] Step 105: Obtain a blurred result image according to the blurred background image and the original image frame.

[0040] The embodiments of the present application can reduce the hardware requirements and processing difficulty of background blur processing, realize adaptive background blur processing, and can be applied to a variety of image processing scenarios. In addition, the background blur intensity is corrected by focusing distance, which more accurately simulates the blur intensity performance of the lens at multiple distances.

[0041] In some embodiments, the above step 101 can be performed in the following manner: semantic segmentation is performed on the original image frame to obtain the foreground area and the background area; or a depth map of the original image frame is obtained, and the foreground area and the background area are obtained based on the depth map.

[0042] As an example, the depth map can be obtained by referring to related technologies.

[0043] In this embodiment, the foreground area and background area of ​​the original image frame can be obtained directly through the semantic segmentation method without using depth information; or, the foreground area and background area of ​​the original image frame can be distinguished through the depth map of the original image frame using the depth prediction method, where the foreground area is the target area of ​​the original image frame.

[0044] For example, the target area can be a portrait area, and the target is a portrait image. Using a portrait as a target is only one implementation method provided in the embodiments of this application and does not limit the technical solution of this application. Other types of targets are also applicable to the technical solution of this application, such as animals, plants, vehicles and other means of transportation, license plates, etc. As an example, the target can meet at least one of the following conditions: having a specific shape; having a recognizable outline; and being able to use a corresponding detection algorithm to determine the area to which it belongs in the image.

[0045] As an example, a semantic segmentation method may include predicting each pixel in the original image frame as being in the foreground or background region. For each pixel in the original image frame, the semantic segmentation output is a probability of being in the foreground region and a probability of being in the background region. Generally, if the probability of a pixel being in the foreground region is greater than the probability of being in the background region, the pixel is considered to be in the foreground region; otherwise, the pixel is considered to be in the background region. The foreground region may be the region where the object is located.

[0046] In an optional example, deep neural networks (DNNs) can be used to implement semantic segmentation. For example, a DNN model can be trained using manually labeled samples based on the target scene, and semantic segmentation can be performed based on the trained DNN model. The embodiments of the present application can also use other image processing methods to perform semantic segmentation on the original image frame to obtain the foreground and background regions described above, and the embodiments of the present application are not limited to this.

[0047] In some embodiments, before generating a blur intensity mapping table corresponding to the background area of ​​the original image frame, the background blurring method of the embodiment of the present application may further include: obtaining a target segmentation mask of the foreground area.

[0048] A mask is a single-channel image that can be used to block all or part of an image during image processing. In this embodiment, the target segmentation mask can be a mask corresponding to a foreground area, which can be used to block the foreground area during image processing.

[0049] As an example, the mask is a binary image consisting of 0 and 1, or the mask may also be a multi-valued image.

[0050] In some embodiments, the above step 105 may include: fusing the blurred background image with the original image frame according to the target segmentation mask to obtain a blurred result image.

[0051] In this embodiment, the target segmentation mask is used to block the foreground area of ​​the original image frame, so that the background area in the original image frame is replaced by the background area in the blurred background image, thereby completing the fusion and obtaining a blurred result image.

[0052] In some embodiments, the above-mentioned step 102 of the present application may include: setting a preset blur intensity; determining a blur baseline of the background area, wherein the blur intensity corresponding to the pixels on the blur baseline is a first blur intensity, and the first blur intensity is not higher than the preset blur intensity; obtaining the distance from the pixels of the background area to the blur baseline; dividing the pixels of the background area into multiple pixel groups according to the distance; and generating a blur intensity mapping table based on the multiple pixel groups.

[0053] In some optional implementations of this embodiment, the blur intensity mapping table may be a one-dimensional array.

[0054] If the bokeh intensity is too high, the image's depth of field will be too shallow, resulting in only the focused plane being sharp while other areas lose clarity. Therefore, the maximum bokeh intensity can be limited. In this embodiment, a preset bokeh intensity can be set based on the physical large aperture lens. The bokeh intensity in the bokeh intensity mapping table is set no higher than the preset bokeh intensity, thus preventing the adverse effects of excessive bokeh intensity on overall image quality.

[0055] In this embodiment, pixels in the same pixel group correspond to the same blurring intensity, and the blurring intensities corresponding to different pixel groups may be the same or different.

[0056] In some optional implementations of this embodiment, the blur baseline of the background area may be determined in the following manner: the blur baseline is determined based on a focus plane of the original image frame, wherein the first blur intensity may be 0.

[0057] Taking the foreground area as a full-body portrait and the full-body portrait being in the focus plane as an example, the pixel row where the soles of the person's feet are located can be selected as the blur baseline. At this time, the blur baseline is on the focus plane, and the first blur intensity can be 0.

[0058] In some embodiments, dividing the pixels of the background area into multiple pixel groups based on distance may include: dividing the pixels on the same side of the blur baseline and at the same distance from the blur baseline into the same pixel group, wherein each pixel group corresponds to a blur intensity value.

[0059] For example, pixel A in the background area is above the blur baseline, pixel B is below the blur baseline, and pixel A and pixel B are at the same distance from the blur baseline, but pixel A and pixel B are not on the same side of the blur baseline. Then, pixel A and pixel B do not belong to the same pixel group, and their corresponding blur intensities are not necessarily the same.

[0060] For another example, if pixel C and pixel D in the background area are both located above the blur baseline and are at the same distance from the blur baseline, then pixel C and pixel D belong to the same pixel group and their blur intensities may be the same.

[0061] In some embodiments, generating a blur intensity mapping table based on multiple pixel groups may include: determining the blur intensity mapping table according to the blur intensity value corresponding to each pixel group; wherein the blur intensity mapping table corresponds to one or more blur intensity intervals, and each blur intensity interval is 0 to the first blur intensity or the first blur intensity to the preset blur intensity.

[0062] In this embodiment, the blur intensity mapping table may correspond to one or more blur intensity intervals based on the position of the blur reference line in the original image frame. When the blur reference line divides the original image frame into multiple regions, each region may correspond to a blur intensity interval, and each blur intensity interval may range from 0 to a first blur intensity or from the first blur intensity to a preset blur intensity.

[0063] Taking Figure 2 as an example, Figure 2a is an original image frame with a half-length portrait as the target area; Figure 2b is a schematic diagram of the foreground area and background area of ​​the original image frame, where the white part is the foreground area and the black part is the background area.

[0064] As an example, the blur baseline can be the lower boundary kk'. As can be seen from Figure 2a, the lower boundary kk' belongs to the focus plane of the original image frame. From the lower boundary kk' to the upper boundary JJ', the distance from the background to the focus plane gradually increases, and the blur intensity corresponding to the pixel also gradually increases. The first blur intensity can be 0, and the blur intensity interval is from the first blur intensity to the preset blur intensity. According to the distance from the pixels of the background area to the blur baseline kk', n pixel groups can be obtained (n is a positive integer greater than 2). In the interval not lower than the first blur intensity and not higher than the preset blur intensity, n blur intensity values ​​are determined. The first blur intensity can be used as the minimum blur intensity value, and gradually increasing n-1 blur intensity values ​​are obtained, and the maximum blur intensity value does not exceed the preset blur intensity. The n blur intensity values ​​thus obtained can form a blur intensity mapping table.

[0065] In another example, the blur baseline can be the upper boundary JJ'. As shown in Figure 2a, the upper boundary JJ' belongs to the distant view area. From the upper boundary JJ' to the lower boundary kk', the distance from the background to the focus plane gradually decreases, and the blur intensity corresponding to the pixel also gradually decreases. The blur intensity interval can be 0 to the first blur intensity. According to the distance from the pixel of the background area to the blur baseline JJ', n pixel groups can be obtained (n is a positive integer greater than 2); within the interval not lower than 0 and not higher than the first blur intensity, n blur intensity values ​​can be determined, the first blur intensity can be used as the maximum blur intensity value, and n-1 blur intensity values ​​that gradually decrease can be obtained. The n blur intensity values ​​thus obtained can form a blur intensity mapping table.

[0066] In another example, the blur reference line can be an arbitrary straight line II' parallel to the upper boundary or the lower boundary, and II' divides the background area into a first background area in the upper part and a second background area in the lower part. As can be seen from Figure 2a, the first background area belongs to the distant view area, and the second background area belongs to the focus plane to the near view area; in the first background area, from the blur reference line II' to the upper boundary JJ', the distance from the background to the focus plane gradually increases, and the blur intensity corresponding to the pixel also gradually increases. The first blur intensity interval can be the first blur intensity to the preset blur intensity. According to the distance from the pixel to the blur reference line II', m pixel groups can be obtained (m is a positive integer greater than 2). In the interval not lower than the first blur intensity and not higher than the preset blur intensity, m blur intensity values ​​are determined, wherein the first blur intensity can be used as the minimum blur intensity value to obtain gradually increasing m-1 first background area blur intensity values, and the maximum blur intensity value does not exceed the preset blur intensity. In the second background area, the distance from the background to the focus plane gradually decreases from the blur reference line II' to the lower boundary kk', and the blur intensity corresponding to the pixel also gradually decreases. The second blur intensity range can be 0 to the first blur intensity. Based on the distance from the pixel to the blur reference line II', h pixel groups can be obtained (h is a positive integer greater than 2). Within the range of not less than 0 and not greater than the first blur intensity, h blur intensity values ​​are determined, wherein the first blur intensity can be used as the maximum blur intensity value, and h-1 gradually decreasing blur intensity values ​​of the second background area can be obtained.

[0067] Alternatively, since the second background area is within the range from the focal plane to the foreground area, the pixels in the second background area may not be blurred. That is, the h-1 blur intensity values ​​for the second background area are all the first blur intensity. The m+h-1 blur intensity values ​​may form a blur intensity mapping table.

[0068] In the above example, the gradual increase or decrease of the blur intensity value may be linear or nonlinear, and the amount of increase or decrease may be determined according to an empirical formula.

[0069] In some examples, the blur intensity values ​​in the blur intensity mapping table may be the same, for example, may all be the first blur intensity.

[0070] In this embodiment, pixels at the same distance from the blur reference line have the same blur intensity, which can achieve an orderly gradient effect on the background blur image, thereby helping to improve the quality of the background blur image.

[0071] In some embodiments, the above step 103 may include: correcting the focus distance of the original image frame to obtain a corrected focus distance; and correcting the blur intensity mapping table according to the corrected focus distance.

[0072] The embodiment of the present application can improve the quality of background blur processing by correcting the blur intensity mapping table.

[0073] In some optional implementations of this embodiment, the focus distance of the original image frame is corrected to obtain a corrected focus distance, which may include: when there is no historical image frame before the original image frame, determining the corrected focus distance based on the focus distance of the original image frame; when there is a historical image frame before the original image frame, determining the corrected focus distance based on the focus distance of the original image frame and the focus distance of the historical image frame.

[0074] In this embodiment, the focus distance of the original image frame can be obtained through the input focus distance parameter. When there is no historical image frame before the original image frame, the focus distance of the original image frame can be corrected.

[0075] In an example of this embodiment, determining the corrected focus distance based on the focus distance of the original image frame and the focus distance of the historical image frame can include: judging whether the focus distance of the original image frame is abnormal; if the focus distance of the original image frame is abnormal, using the focus distance of the historical image frame as the corrected focus distance; if the focus distance of the original image frame is normal, smoothing the focus distance of the original image frame and the focus distance of the historical image frame to obtain the corrected focus distance.

[0076] As an example, due to the initial setting of the lens, when the focus distance is abnormal, abnormal values ​​such as -1, 0 or 1000 may appear in the original image frame, which can be used to determine whether the focus distance of the original image frame is abnormal.

[0077] As an example, the above-mentioned smoothing process can be achieved by using a one-dimensional low-pass filter (e.g., a one-dimensional Gaussian filter). The one-dimensional low-pass filter can be used to perform a weighted average of the focus distance of the original image frame and the focus distance of the historical image frame to obtain a corrected focus distance. Here, the focus distance of the original image frame can be weighted the most, and the weight of the historical image frame that is further away from the original image frame in time is reduced.

[0078] In one example, the blur intensity mapping table may be corrected according to the corrected focus distance, so that the closer the focus distance is, the stronger the blur intensity is.

[0079] For example, it is known that the shallower the depth of field, the stronger the blur intensity. The relationship between depth of field and focus distance can be expressed as:

[0080] DOF≈2*u*u*N*c / f / f.

[0081] Among them, DOF stands for depth of field; u stands for the corrected focus distance; N stands for the aperture size; c stands for the maximum allowable circular focus (CoC), that is, the maximum tolerable blur during imaging; f stands for the contrast focal length.

[0082] It can be seen that the closer the focusing distance, the shallower the depth of field and the stronger the blur intensity.

[0083] This embodiment uses the corrected focus distance to correct the blur intensity mapping table, which can achieve fast and efficient blur intensity setting and improve the efficiency of background blur processing.

[0084] Exemplarily, the above-mentioned step 104 can obtain a blurred background image in the following manner: according to the one-to-one correspondence between the blur intensity values ​​in the corrected blur intensity mapping table and the multiple pixel groups in the background area, a corrected blur intensity value is assigned to each pixel group, and then each pixel in the background area is blurred according to the blur intensity value to obtain a blurred background image.

[0085] An embodiment of the present application also provides a background blurring device, including: an image processing unit 100, configured to obtain a foreground area and a background area of ​​an original image frame; a blurring intensity generating unit 200, configured to generate a blurring intensity mapping table corresponding to the background area; a correction unit 300, configured to correct the blurring intensity mapping table based on the focus distance of the original image frame; a blurring processing unit 400, configured to blur the background area according to the corrected blurring intensity mapping table to obtain a blurred background image; and a result output unit 500, configured to obtain a blurred result image based on the blurred background image and the original image frame.

[0086] In some embodiments, the image processing unit 100 is configured to: perform semantic segmentation on the original image frame to obtain a foreground area and a background area; or obtain a depth map of the original image frame and obtain a foreground area and a background area based on the depth map.

[0087] In this embodiment, the depth map can be obtained by referring to related technologies.

[0088] In this embodiment, the foreground area and background area of ​​the original image frame can be obtained directly through the semantic segmentation method without using depth information; or, the foreground area and background area of ​​the original image frame can be distinguished through the depth map of the original image frame using the depth prediction method, where the foreground area is the target area of ​​the original image frame.

[0089] For example, the target area can be a portrait area, and the target is a portrait. Taking a portrait as a target is only a specific implementation method provided in the embodiments of this application and does not limit the technical solution of this application. Other types of targets are also applicable to the technical solution of this application, such as animals, plants, vehicles and other means of transportation, license plates, etc. As an example, the target under the corresponding target type must meet at least one of the following conditions: have a specific shape; have a relatively clear outline; and be able to use the corresponding detection algorithm to determine the area position to which it belongs in the image.

[0090] In some embodiments, the background blurring device further includes a mask acquiring unit configured to acquire a target segmentation mask of the foreground area.

[0091] A mask is a single-channel image that can be used to block all or part of an image to be processed during image processing. In this embodiment, the target segmentation mask can be a mask corresponding to a foreground area, which can be used to block the foreground area during image processing.

[0092] As an example, the mask is a binary image consisting of 0 and 1, or the mask may also be a multi-valued image.

[0093] In some embodiments, the result output unit 500 is configured to: perform fusion processing on the blurred background image and the original image frame according to the target segmentation mask to obtain a blurred result image.

[0094] In this embodiment, the target segmentation mask is used to block the foreground area of ​​the original image frame, so that the background area in the original image frame is replaced by the background area in the blurred background image, thereby completing the fusion and obtaining a blurred result image.

[0095] In some embodiments, the blur intensity generation unit 200 is configured to: determine a blur baseline of the background area, wherein the blur intensity corresponding to the pixels on the blur baseline is a first blur intensity, and the first blur intensity is not higher than a preset blur intensity; obtain the distance from the pixels of the background area to the blur baseline; divide the pixels of the background area into multiple pixel groups according to the distance; and generate a blur intensity mapping table based on the multiple pixel groups, wherein the multiple pixel groups correspond one-to-one to the blur intensities in the blur intensity mapping table.

[0096] In some optional implementations of this embodiment, the blur intensity mapping table may be a one-dimensional array.

[0097] If the bokeh intensity is too high, the depth of field will be too shallow, resulting in only the focused plane being sharp while other areas become less clear. Therefore, the maximum bokeh intensity can be limited. In this embodiment, a preset bokeh intensity can be set based on the physical large aperture lens. The bokeh intensity in the bokeh intensity mapping table is set no higher than the preset bokeh intensity, thus preventing the adverse effects of excessive bokeh intensity on overall image quality.

[0098] In some embodiments, determining the blur reference line of the background area may include: determining the blur reference line based on a focus plane of the original image frame, wherein the first blur intensity is zero.

[0099] Taking the foreground area as a full-body portrait and the full-body portrait in the focus plane as an example, the pixel row where the soles of the person's feet are located can be selected as the blur baseline. At this time, the blur baseline is on the focus plane and the first blur intensity is 0.

[0100] In some embodiments, dividing the pixels of the background area into a plurality of pixel groups according to distance may include: pixels on the same side of the virtual reference line and at the same distance from the virtual reference line belong to the same pixel group.

[0101] In some embodiments, generating a blur intensity mapping table based on multiple pixel groups may include: determining a corresponding number of multiple blur intensity values ​​in one or more blur intervals according to the number of multiple pixel groups; the multiple blur intensity values ​​constitute a blur intensity mapping table; wherein each blur intensity interval is 0 to a first blur intensity or from the first blur intensity to a preset blur intensity.

[0102] In this embodiment, the original image frame may correspond to one or more blur intensity intervals based on the position of the blur reference line in the original image frame. When the blur reference line divides the original image frame into multiple regions, each region may correspond to a blur intensity interval, and each blur intensity interval may range from 0 to a first blur intensity or from the first blur intensity to a preset blur intensity.

[0103] In an exemplary embodiment, the correction unit 300 is configured to: correct the focus distance of the original image frame to obtain a corrected focus distance; and correct the blur intensity mapping table according to the corrected focus distance.

[0104] This embodiment improves the quality of background blur processing by correcting the blur intensity mapping table.

[0105] In some embodiments, the focus distance of the original image frame is corrected to obtain a corrected focus distance, which may include: when there is no historical image frame before the original image frame, determining the corrected focus distance based on the focus distance of the original image frame; when there is a historical image frame before the original image frame, determining the corrected focus distance based on the focus distance of the original image frame and the focus distance of the historical image frame.

[0106] In this embodiment, when there is no historical image frame before the original image frame, the focus distance of the original image frame can be corrected, and the focus distance of the original image frame can be obtained through the input focus distance parameter.

[0107] In some embodiments, determining the corrected focus distance based on the focus distance of the original image frame and the focus distance of the historical image frame can include: judging whether the focus distance of the original image frame is abnormal; if the focus distance of the original image frame is abnormal, using the focus distance of the historical image frame as the corrected focus distance; if the focus distance of the original image frame is normal, smoothing the focus distance of the original image frame and the focus distance of the historical image frame to obtain the corrected focus distance.

[0108] For example, when the focus distance is abnormal, values ​​such as -1, 0, or 1000 may appear in the original image frame. This is related to the initial setting of the lens, and can be used to determine whether the focus distance of the original image frame is abnormal.

[0109] In some embodiments, the above-described smoothing process can be implemented using a one-dimensional low-pass filter (e.g., a Gaussian filter). The one-dimensional low-pass filter can be used to perform a weighted average of the focus distance of the original image frame and the focus distance of the historical image frames to obtain a corrected focus distance. For example, the focus distance of the original image frame can be weighted the most, while the weight of the historical image frames that are further away from the original image frame in time is reduced.

[0110] In some embodiments, the blur intensity mapping table may be corrected according to the corrected focus distance, so that the closer the focus distance is, the stronger the blur intensity is.

[0111] For example, it is known that the shallower the depth of field, the stronger the blur intensity, and the relationship between depth of field and focus distance can be expressed as:

[0112] DOF≈2*u*u*N*c / f / f.

[0113] Among them, DOF stands for depth of field; u stands for the corrected focus distance; N stands for the aperture size; c stands for the maximum allowable circular focus (CoC), that is, the maximum tolerable blur during imaging; f stands for the contrast focal length.

[0114] It can be seen that the closer the focusing distance, the shallower the depth of field, and the stronger the blur intensity.

[0115] This embodiment uses the corrected focus distance to correct the blur intensity mapping table, which can achieve fast and efficient blur intensity setting and improve the efficiency of background blur processing.

[0116] In some embodiments, the blur processing unit 400 is configured to: assign a corrected blur intensity value to each pixel group according to a one-to-one correspondence between the blur intensity values ​​in the corrected blur intensity mapping table and multiple pixel groups in the background area, and then blur each pixel in the background area according to the blur intensity value to obtain a blurred background image.

[0117] An embodiment of the present application further provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the background blurring method in any of the above embodiments is implemented.

[0118] An embodiment of the present application also provides a terminal, including: a memory and a processor, wherein a computer program is stored in the memory; wherein the processor is configured to execute the computer program in the memory; when the computer program is executed by the processor, the background blur method as in any of the above embodiments is implemented.

[0119] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A background blur method, comprising: Obtaining the foreground area and background area of ​​the original image frame; Generate a blur intensity mapping table corresponding to the background area; Correcting the blur intensity mapping table based on the focus distance of the original image frame; According to the corrected blur intensity mapping table, blurring the background area to obtain a blurred background image; A blurred result image is obtained according to the blurred background image and the original image frame.

2. The background blurring method according to claim 1, wherein: The step of obtaining the foreground area and the background area of ​​the original image frame includes: Performing semantic segmentation on the original image frame to obtain the foreground area and the background area; or A depth map of the original image frame is acquired, and the foreground area and the background area are obtained based on the depth map.

3. The background blurring method according to claim 1, before generating the blurring intensity mapping table corresponding to the background area, the background blurring method further comprises: Obtain a target segmentation mask for the foreground area.

4. The background blurring method according to claim 3, wherein: The step of obtaining a blurred result image according to the blurred background image and the original image frame comprises: According to the target segmentation mask, the blurred background image and the original image frame are fused to obtain the blurred result image.

5. The background blurring method according to claim 1, wherein: The generating the blur intensity mapping table corresponding to the background area includes: Determine a blur reference line of the background area, wherein a blur intensity corresponding to pixels on the blur reference line is a first blur intensity, and the first blur intensity is not higher than a preset blur intensity; Obtaining the distance from the pixel of the background area to the blur reference line; Dividing the pixels of the background area into a plurality of pixel groups according to the distance; The blur intensity mapping table is generated based on the plurality of pixel groups.

6. The background blurring method according to claim 5, wherein: The dividing the pixels of the background area into a plurality of pixel groups according to the distance comprises: Pixels on the same side of the blur reference line and at the same distance from the blur reference line are divided into the same pixel group, wherein each pixel group corresponds to a blur intensity value.

7. The background blurring method according to claim 6, wherein: The generating the blur intensity mapping table based on the plurality of pixel groups comprises: Determine the blur intensity mapping table according to the blur intensity value corresponding to each pixel group; The blur intensity mapping table corresponds to one or more blur intensity intervals, and each blur intensity interval is from 0 to the first blur intensity or from the first blur intensity to the preset blur intensity.

8. The background blurring method according to any one of claims 1 to 7, wherein: The correcting the blur intensity mapping table based on the focus distance of the original image frame includes: Correcting the focus distance of the original image frame to obtain a corrected focus distance; The blur intensity mapping table is corrected according to the corrected focus distance.

9. The background blurring method according to claim 8, wherein: The step of correcting the focus distance of the original image frame to obtain a corrected focus distance includes: In the case where there is no historical image frame before the original image frame, determining the corrected focus distance according to the focus distance of the original image frame; In the case that the historical image frame exists before the original image frame, the corrected focus distance is determined according to the focus distance of the original image frame and the focus distance of the historical image frame.

10. The background blurring method according to claim 9, wherein: The step of determining the corrected focus distance according to the focus distance of the original image frame and the focus distance of the historical image frame includes: Determining whether the focus distance of the original image frame is abnormal; In the case where the focus distance of the original image frame is abnormal, using the focus distance of the historical image frame as the corrected focus distance; When there is no abnormality in the focus distance of the original image frame, the focus distance of the original image frame and the focus distance of the historical image frame are smoothed to obtain the corrected focus distance.

11. A background blurring device, comprising: An image processing unit configured to obtain a foreground region and a background region of an original image frame; A blur intensity generating unit, configured to generate a blur intensity mapping table corresponding to the background area; a correction unit, configured to correct the blur intensity mapping table based on the focus distance of the original image frame; A blur processing unit, configured to blur the background area according to the corrected blur intensity mapping table to obtain a blurred background image; The result output unit is configured to obtain a blurred result image according to the blurred background image and the original image frame.

12. A computer storage medium, wherein a computer program is stored in the computer storage medium, and when the computer program is executed by a processor, the background blurring method according to any one of claims 1 to 10 is implemented.

13. A terminal, comprising: A memory and a processor, wherein the memory stores a computer program; wherein, The processor is configured to execute the computer program in the memory; When the computer program is executed by the processor, the background blurring method according to any one of claims 1 to 10 is implemented.

Images