Image processing method and device, electronic equipment and medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a medium. The image processing method comprises the steps that the outer contour of a target body in a to-be-processed first image is expanded outwards and contracted inwards to obtain a second image, and the first image is composed of the target body and a background image; according to the first image, pixels are filled in a first area between external expansion and internal contraction of the outer contour in the second image and fused in the first image, a third image is obtained, and the pixels filled in the first area are related to the background image; performing fuzzy processing on the third image to obtain a fourth image; and fusing the target main body in the first image with the fourth image to obtain a target image. According to the image processing method and device, the first area possibly generating smear is determined, the pixels related to the background image are filled, smear cannot occur between the target body and the background image after the background image is subjected to fuzzy processing, and therefore the image processing effect is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and medium. Background Technology

[0002] With advancements in photography technology, users can now pan to capture moving subjects, achieving a dynamic effect. To simplify panning, the subject and background images are separated after image acquisition, and the background image is blurred to achieve the same effect. However, blurring the background image can sometimes result in motion blur of the subject, leading to poor image processing quality. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides an image processing method, apparatus, electronic device, and medium.

[0004] According to a first aspect of the present disclosure, an image processing method is provided, the image processing method comprising:

[0005] The outer contour of the target subject in the first image to be processed is expanded outward and shrunk inward to obtain the second image. The first image is composed of the target subject and the background image.

[0006] Based on the first image, pixels are filled in the first region between the outward expansion and inward contraction of the outer contour in the second image and fused into the first image to obtain a third image, wherein the pixels filled in the first region are related to the background image;

[0007] The third image is blurred to obtain the fourth image;

[0008] The target subject in the first image is fused with the fourth image to obtain the target image.

[0009] In some embodiments of this disclosure, the step of expanding and shrinking the outer contour of the target subject in the first image to be processed to obtain the second image includes:

[0010] The first image is transformed to obtain a fifth image that only contains the outer contour;

[0011] The outer contour in the fifth image is expanded to obtain the sixth image;

[0012] The outer contour in the fifth image is shrunk inward to obtain the seventh image;

[0013] The second image is generated based on the sixth image and the seventh image.

[0014] In some embodiments of this disclosure, the conversion processing of the first image to obtain a fifth image containing only the outer contour includes:

[0015] The first image is binarized to obtain the eighth image;

[0016] Based on the edge texture of the target subject in the eighth image, morphological processing is performed on the eighth image to obtain the fifth image;

[0017] In the eighth image, the pixel value of the target subject is a first preset value, and the pixel value of the background image is a second preset value.

[0018] In some embodiments of this disclosure, expanding the outer contour in the fifth image to obtain the sixth image includes:

[0019] The outer contour in the fifth image is expanded outward multiple times to obtain the sixth image; and / or,

[0020] The step of shrinking the outer contour in the fifth image to obtain the seventh image includes:

[0021] The outer contour in the fifth image is shrunk inward multiple times to obtain the seventh image.

[0022] In some embodiments of this disclosure, the second image includes the largest outer contour in the sixth image and the smallest outer contour in the seventh image, the first region is the region between the largest and the smallest outer contour, and the width of the first region is greater than or equal to the blur degree of the blur kernel used for blur processing.

[0023] In some embodiments of this disclosure, the step of filling pixels in a first region between the outward expansion and inward contraction of the outer contour in the second image and fusing it into the first image to obtain a third image includes:

[0024] Determine the target pixel value of the first image in the second region extending beyond the outer contour;

[0025] Based on the target pixel value, pixels are filled into the first region of the second image to obtain the ninth image;

[0026] The image within the outer contour of the ninth image is fused with the first image to obtain the third image.

[0027] In some embodiments of this disclosure, the outer contour expands multiple times, and the second region is the area between the largest and smallest expanded outer contours; determining the target pixel value of the first image in the second region of the outer contour expansion includes:

[0028] Using the center point of the target entity as the center, the second region is divided under the circumference to obtain multiple first sub-regions;

[0029] The average pixel value of each pixel in each first sub-region of the first image is taken as the target pixel value of the corresponding first sub-region.

[0030] In some embodiments of this disclosure, filling the first region of the second image with pixels according to the target pixel value to obtain the ninth image includes:

[0031] Using the center point as the center, the first region is divided into multiple second sub-regions along the circumference;

[0032] The pixel value of each pixel in each second sub-region of the second image is set to the target pixel value of the included first sub-region to fill the pixel, thus obtaining the ninth image.

[0033] In some embodiments of this disclosure, before blurring the third image to obtain the fourth image, the image processing method further includes:

[0034] Obtain the location information and optical flow information of the target entity;

[0035] Based on the position information and the optical flow information, a first motion vector of the optical flow in a first preset direction and a second motion vector in a second preset direction are determined;

[0036] Based on the first motion vector and the second motion vector, determine the blur degree and blur angle of the blur kernel;

[0037] The process of blurring the third image to obtain the fourth image includes:

[0038] The third image is blurred based on the degree of blur and the blur angle to obtain the fourth image.

[0039] In some embodiments of this disclosure, the step of blurring the third image according to the degree of blur and the blur angle to obtain the fourth image includes:

[0040] Establish an identity matrix with the same number of rows and columns as the desired level of fuzziness;

[0041] The identity matrix is ​​rotated according to the fuzzy angle to obtain the target matrix;

[0042] The third image is blurred based on the target matrix to obtain the fourth image.

[0043] In some embodiments of this disclosure, fusing the target subject in the first image with the fourth image to obtain a target image includes:

[0044] Based on the first image and the fifth image which only contains the outer contour, a tenth image containing only the target subject is generated;

[0045] The edge texture of the target subject in the tenth image is blurred to obtain the eleventh image;

[0046] The target image is obtained by fusing the eleventh image as the foreground and the fourth image as the background.

[0047] According to a second aspect of the present disclosure, an image processing apparatus is provided, the image processing apparatus comprising:

[0048] A first processing module is configured to expand and shrink the outer contour of the target subject in the first image to be processed to obtain a second image, wherein the first image is composed of the target subject and a background image.

[0049] A filling module is configured to fill pixels in a first region between the outward expansion and inward contraction of the outer contour in a second image based on the first image and fuse it into the first image to obtain a third image, wherein the pixels filled in the first region are related to the background image.

[0050] A second processing module is configured to blur the third image to obtain a fourth image.

[0051] A generation module is configured to fuse the target subject in the first image with the fourth image to obtain a target image.

[0052] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0053] processor;

[0054] Memory used to store the processor's executable instructions;

[0055] The processor is configured to perform the image processing method described above.

[0056] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to perform the image processing method as described above.

[0057] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0058] The outer contour of the target subject in the first image to be processed is expanded and contracted inward to identify areas that may cause motion blur, resulting in a second image. Based on the first image, pixels are filled into a first region between the expanded and contracted outer contours in the second image and merged into the first image to obtain a third image. This third image is then used to fill the areas that may cause motion blur with pixels related to the background image to eliminate motion blur. The third image is then blurred to blur both the background image and the filled pixels, achieving the same effect as panning, resulting in a fourth image. The target subject in the first image is merged with the fourth image to obtain the target image, which contains a sharp target subject and a blurred background image. By identifying the first region that may cause motion blur and filling it with pixels related to the background image, no motion blur appears between the target subject and the background image after the background image is blurred, thus improving the image processing effect.

[0059] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0061] Figure 1 This is a schematic flowchart illustrating an image processing method according to an exemplary embodiment;

[0062] Figure 2 This is a flowchart illustrating an image processing method according to another exemplary embodiment;

[0063] Figure 3 This is a schematic diagram of the sixth image according to an exemplary embodiment;

[0064] Figure 4 This is a schematic diagram of a second image according to an exemplary embodiment;

[0065] Figure 5 This is a flowchart illustrating an image processing method according to another exemplary embodiment;

[0066] Figure 6 This is a flowchart illustrating an image processing method according to another exemplary embodiment;

[0067] Figure 7 This is a flowchart illustrating an image processing method according to another exemplary embodiment;

[0068] Figure 8 This is a schematic diagram of a third image shown according to an exemplary embodiment;

[0069] Figure 9 This is a schematic diagram of a fourth image shown according to an exemplary embodiment;

[0070] Figure 10 This is a flowchart illustrating an image processing method according to another exemplary embodiment;

[0071] Figure 11 This is a schematic diagram of a target image according to an exemplary embodiment;

[0072] Figure 12 This is a flowchart illustrating an image processing method according to another exemplary embodiment;

[0073] Figure 13 This is a block diagram of an image processing apparatus according to an exemplary embodiment;

[0074] Figure 14 This is a block diagram of an electronic device according to an exemplary embodiment.

[0075] In the picture:

[0076] 100 - First processing module; 150 - Filling module; 200 - Second processing module; 250 - Generation module; 400 - Electronic device; 402 - Processing component; 404 - Memory; 406 - Power supply component; 408 - Multimedia component; 410 - Audio component; 412 - Input / output interface; 414 - Sensor component; 416 - Communication component; 420 - Processor. Detailed Implementation

[0077] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. It should also be understood that the term “and / or” as used in this disclosure refers to and includes any or all possible combinations of one or more of the associated listed items.

[0078] With the development of photography technology, users can now pan to capture moving subjects, achieving a dynamic effect. Panning is a technique that uses a slow shutter speed to track the subject. That is, the shutter speed is set to a slow speed, and the camera is panned along with the subject to achieve a dynamic effect where the moving subject is sharp and the background image is blurred. However, when the subject (such as a vehicle) is moving quickly, it is difficult for users to keep the electronic device and the subject moving in sync, resulting in a blurred subject. To reduce the difficulty of panning, the subject and background images are separated after image capture, and the background image is blurred to achieve the same effect as panning. However, blurring the background image may produce motion blur of the subject, resulting in poor image processing quality.

[0079] To address the aforementioned technical issues, this disclosure provides an image processing method that fills the region at the boundary between the target subject and the background image with pixels related to the background image. This prevents motion blur when blurring the background image, thereby improving the image processing effect.

[0080] This disclosure provides an image processing method, such as... Figure 1 As shown, the method includes:

[0081] S100. Expand and shrink the outer contour of the target subject in the first image to be processed to obtain the second image. The first image consists of the target subject and the background image.

[0082] S200. Based on the first image, pixels are filled in the first region between the outward expansion and inward contraction of the outer contour in the second image and fused into the first image to obtain the third image. The pixels filled in the first region are related to the background image.

[0083] S300. Blur the third image to obtain the fourth image.

[0084] S400: Merge the target subject in the first image with the fourth image to obtain the target image.

[0085] In this embodiment, the outer contour of the target subject in the first image to be processed is expanded outward and shrunk inward to determine the region that may produce motion blur, resulting in a second image. Based on the first image, pixels are filled into the first region between the expanded and shrunk outer contours in the second image and fused into the first image to obtain a third image. This third image is then used to fill the region that may produce motion blur with pixels related to the background image to eliminate motion blur. The third image is then blurred to blur both the background image and the filled pixels, achieving the same effect as panning, resulting in a fourth image. The target subject in the first image is fused with the fourth image to obtain a target image, which contains a clear target subject and a blurred background image. By determining the first region that may produce motion blur and filling it with pixels related to the background image, no motion blur appears between the target subject and the background image after the background image is blurred, thus improving the image processing effect.

[0086] For example, the target subject can be a person, vehicle, animal, etc. The background image is the image in the first image excluding the target subject. The target subject can be identified by an algorithm or selected by the user. The first image can be a pre-stored image or an image acquired in real time.

[0087] In one embodiment, such as Figure 2 As shown, in step S100, the outer contour of the target subject in the first image to be processed is expanded outward and shrunk inward to obtain the second image, which is determined in the following way:

[0088] S110. The first image is transformed to obtain a fifth image that only contains the outer contour.

[0089] S120. Expand the outer contour of the fifth image to obtain the sixth image.

[0090] S130. Shrink the outer contour of the fifth image to obtain the seventh image.

[0091] S140. Generate a second image based on the sixth and seventh images.

[0092] In this embodiment, since the first image contains both the target subject and the background image, directly expanding and shrinking the outer contour of the target subject in the first image would make it difficult to identify the target subject and the background image. The first image is transformed to remove the image inside the target subject and the background image, resulting in a fifth image containing only the outer contour. The outer contour in the fifth image is expanded to obtain a sixth image with a proportionally enlarged outer contour. The outer contour in the fifth image is shrunk to obtain a seventh image with a proportionally reduced outer contour. Based on the sixth and seventh images, a second image containing both the enlarged and reduced outer contours is generated. By expanding and shrinking the outer contour of the transformed fifth image, interference caused by changes in the outer contour of the target subject is avoided, thereby improving the reliability of image processing.

[0093] In one embodiment, the conversion process performed on the first image in step S110 to obtain a fifth image containing only the outer contour is determined as follows:

[0094] The first image is binarized to obtain the eighth image.

[0095] Based on the edge texture of the target subject in the eighth image, morphological processing is performed on the eighth image to obtain the fifth image.

[0096] In the eighth image, the pixel value of the target subject is a first preset value, and the pixel value of the background image is a second preset value.

[0097] In this embodiment, since only a fifth image containing the outer contour is needed, the first image is binarized to distinguish the target subject from the background image to facilitate image recognition and processing. Because the outer contour needs to be expanded and contracted, if the pixel values ​​at different positions of the target subject in the image are all the same first preset value, it is difficult to distinguish the outer contour after each expansion and contraction. Based on the edge texture of the target subject in the eighth image, morphological processing is performed on the eighth image to obtain a fifth image containing only the largest connected component as the outer contour. By converting the first image into a fifth image containing only two pixel values, it is easier to distinguish the outer contour in the fifth image for expansion and contraction, thereby improving the reliability of image processing.

[0098] For example, the first preset value can be 255, and the second preset value can be 0. That is, in the fifth image, the outer contour of the target subject is white, and the rest is black.

[0099] For example, in step S120, expanding the outer contour of the fifth image to obtain the sixth image can be done by dilating the fifth image. In step S130, shrinking the outer contour of the fifth image to obtain the seventh image can be done by eroding the fifth image.

[0100] In one embodiment, the step S120 of expanding the outer contour of the fifth image to obtain the sixth image is determined in the following way:

[0101] The outer contour in the fifth image is expanded outward multiple times to obtain the sixth image.

[0102] In this embodiment, by expanding the outer contour in the fifth image multiple times in sequence, the filled pixels can be related to the background image to naturally transition between the target subject and the background image, thereby improving the image processing effect.

[0103] For example, such as Figure 3 As shown, the outer contour can expand outwards twice, for example. In the sixth image, the area between the two outward expansions is the second region. In the second image, the pixel value of each pixel in the first region can be a first preset value, and the pixel value of each pixel outside the first region can be a second preset value. In the sixth image, the pixel value of each pixel in the second region can be the first preset value, and the pixel value of each pixel outside the second region can be the second preset value.

[0104] In one embodiment, the step S130 of shrinking the outer contour of the fifth image to obtain the seventh image is determined in the following way:

[0105] The outer contour in the fifth image is shrunk inwards multiple times to obtain the seventh image.

[0106] In this embodiment, by repeatedly shrinking the outer contour in the fifth image, pixels can be filled at the boundary between the target subject and the background image to eliminate ghosting, thereby improving the image processing effect.

[0107] For example, the number of times the outer contour is shrunken and the degree of each shrunkenness can be determined based on the degree of blurring of the blur kernel used for blurring.

[0108] For example, in step S140, generating a second image based on the sixth and seventh images can be achieved by fusing the largest outer contour in the sixth image with the smallest outer contour in the seventh image into a single image to obtain the second image.

[0109] In one embodiment, such as Figure 4 As shown, the second image includes the largest outer contour in the sixth image and the smallest outer contour in the seventh image. The first region is the area between the largest and smallest outer contours, and the width of the first region is greater than or equal to the blur degree of the blur kernel used for blur processing.

[0110] In this embodiment, by making the width of the first region of the filling pixel greater than or equal to the blur degree of the blur kernel used for blur processing, the first region can cover the area where the ghosting is located to avoid generating ghosting, thereby improving the image processing effect.

[0111] In one embodiment, such as Figure 5 As shown, in step S200, based on the first image, pixels are filled in the first region between the outward expansion and inward contraction of the outer contour in the second image and fused into the first image to obtain the third image, which is determined in the following manner:

[0112] S210. Determine the target pixel value of the first image in the second region outside the outer contour.

[0113] S220. Based on the target pixel value, fill the first region of the second image with pixels to obtain the ninth image.

[0114] S230. Merge the image within the outer contour of the ninth image with the first image to obtain the third image.

[0115] In this embodiment, a target pixel value is determined in the second region extending beyond the outer contour of the first image to determine the pixel value associated with the background image for filling pixels. Based on the target pixel value, pixels are filled in the first region of the second image to obtain a ninth image filled with pixels associated with the background image at the boundary. Since the ninth image only contains the image at the boundary, the image within the extended outer contour of the ninth image is merged with the first image to obtain a third image where the image at the boundary and part of the background image are connected. By filling the first region with pixels using the target pixel value of the second region, the image at the boundary can seamlessly transition with the background image, thus improving the image processing effect.

[0116] In one embodiment, such as Figure 6 As shown, the outer contour expands multiple times, and the second region is the area between the largest and smallest outer contours. The target pixel value of the first image in the second region of the outer contour expansion in step S210 is determined as follows:

[0117] S211. Using the center point of the target as the center, divide the second region under the circumference to obtain multiple first sub-regions.

[0118] S212. Take the average pixel value of each pixel in each first sub-region of the first image as the target pixel value of the corresponding first sub-region.

[0119] In this embodiment, since filling pixels one-to-one with each pixel value would be too complex, a simplified pixel filling method is needed. The second region is divided into multiple first sub-regions around the center point of the target object. The average pixel value of each pixel within each first sub-region is used as the target pixel value for that sub-region, and the first region is filled using the average pixel value from each sub-region. By filling pixels on a region-by-region basis, one-to-one pixel filling is avoided, thus reducing the complexity of image processing.

[0120] For example, the center point of the target subject differs depending on the target subject. For instance, for a vehicle, the center point of the target subject is the vehicle's center of mass.

[0121] For example, in step S211, the second region is divided into multiple first sub-regions by using the center point of the target object as the center. This can be achieved by dividing the 360° circumference into multiple parts (e.g., 3600 parts) using the center point of the target object as the center, resulting in multiple dividing lines. The second region is then divided using these multiple dividing lines, with the area between two adjacent dividing lines forming a first sub-region.

[0122] For example, steps S211 and S212 are determined by the following formula:

[0123]

[0124] Among them, Dict sector The target pixel value is represented by θ, the angle formed by connecting each pixel in the second region to the center point, n, the number of pixels in the first sub-region, sector, and sector+1. Input (y,x,i) represents the pixel value of the pixel at the corresponding position in the first image, where x and y represent the position of the pixel and i represents the channel of the color space, which can be RGB color space.

[0125] In one embodiment, such as Figure 7 As shown, in step S220, the ninth image is obtained by filling the first region of the second image with pixels based on the target pixel value, determined in the following way:

[0126] S221. Using the center point as the center, divide the first region under the circumference to obtain multiple second sub-regions.

[0127] S222. Set the pixel value of each pixel in each second sub-region of the second image to the target pixel value of the included first sub-region to fill the pixel, and obtain the ninth image.

[0128] In this embodiment, since filling pixels one-to-one with each pixel value would be too complex, the pixel filling method needs to be simplified. The first region is divided into multiple second sub-regions around the center point. The pixel value of each pixel within each second sub-region in the second image is set to the target pixel value of the included first sub-region to fill the pixels, resulting in a ninth image for filling the first region. By filling pixels on a region-by-region basis, the one-to-one pixel filling is avoided, thus reducing the complexity of image processing.

[0129] For example, in step S221, dividing the first region into multiple second sub-regions with the center point as the center can be achieved by dividing the 360° circumference into multiple parts (e.g., 3600 parts) with the center point of the target object as the center, resulting in multiple dividing lines. The first region is divided using multiple dividing lines, and the area between two adjacent dividing lines is a second sub-region.

[0130] For example, such as Figure 8 As shown, the third image includes the pixels filled in the ninth image and a portion of the background image in the first image. The pixels filled in the first region are related to the background image and are located between the background image and the target subject, enabling a natural transition between the background image and the target subject.

[0131] In one embodiment, before blurring the third image in step S300 to obtain the fourth image, the image processing method further includes:

[0132] Obtain the location and optical flow information of the target entity.

[0133] Based on the location information and optical flow information, determine the first motion vector of the optical flow in the first preset direction and the second motion vector in the second preset direction.

[0134] The degree of blur and the blur angle of the blur kernel are determined based on the first motion vector and the second motion vector.

[0135] The blurring of the third image in step S300 to obtain the fourth image is determined as follows:

[0136] The third image is blurred based on the degree and angle of blur to obtain the fourth image.

[0137] In this embodiment, since the target subject moves in a certain direction, the blurring direction of the background image needs to correspond to the direction of the target subject's movement to achieve a dynamic effect. The position information and optical flow information of the target subject are acquired, and based on these information, a first motion vector in a first preset direction and a second motion vector in a second preset direction are determined to obtain the direction of the target subject's movement. Based on the first and second motion vectors, the blurring degree and blurring angle of the blur kernel are determined to ensure that the blurring degree reaches the expected level and the blurring direction corresponds to the direction of the target subject's movement. Based on the blurring degree and blurring angle, the third image is blurred to obtain the fourth image. By determining the blurring degree and blurring angle of the blur kernel to blur the third image, the blurring direction of the background image can correspond to the direction of the target subject's movement, thereby improving the image processing effect.

[0138] For example, the steps of obtaining the location information and optical flow information of the target subject can be either obtaining the location information and optical flow information of a pre-stored first image, or obtaining a preview stream collected by an electronic device and determining the location information and optical flow information based on the preview stream.

[0139] For example, in the above steps, determining the first motion vector of the optical flow in a first preset direction and the second motion vector in a second preset direction based on the position information and optical flow information can be achieved by converting the position information and optical flow information through a preset conversion relationship to obtain the first motion vector and the second motion vector. The first preset direction can be a horizontal direction, and the second preset direction can be a vertical direction.

[0140] For example, in the above steps, the blur degree and blur angle of the blur kernel are determined based on the first motion vector and the second motion vector using the following formula:

[0141] kernel degree =sqrt(dx 2 +dy 2 )*gain

[0142]

[0143] Among them, kernel degree Indicates the degree of ambiguity, kernel angle dx represents the blur angle, dy represents the first motion vector, and gain represents the gain used to control the degree of blur.

[0144] In one embodiment, the process of blurring the third image based on the degree and angle of blur in the above steps to obtain the fourth image is determined as follows:

[0145] Create an identity matrix with both the number of rows and columns having a certain degree of fuzziness.

[0146] The target matrix is ​​obtained by rotating the identity matrix according to the fuzzy angle.

[0147] Based on the target matrix, the third image is blurred to obtain the fourth image.

[0148] In this embodiment, an identity matrix is ​​established where both the number of rows and columns represent the desired level of blur. The identity matrix is ​​then rotated according to the blur angle to obtain a target matrix, which reflects the blur direction. Based on the target matrix, the third image is blurred to obtain a fourth image whose blur level and direction meet the requirements. By blurring the third image with the target matrix to obtain the fourth image, the background image in the fourth image achieves the expected blur effect, thereby improving the image processing efficiency.

[0149] For example, such as Figure 9 As shown, the background image and the pixels filled in the first region in the fourth image are both blurred. The blurred pixels can avoid the generation of ghosting, thereby improving the image processing effect.

[0150] In one embodiment, such as Figure 10 As shown, the target image obtained by fusing the target subject in the first image with the fourth image in step S400 is determined in the following way:

[0151] S410. Based on the first image and the fifth image which only contains the outer contour, generate a tenth image which only contains the target subject.

[0152] S420. Blur the edge texture of the target subject in the tenth image to obtain the eleventh image.

[0153] S430. Using the eleventh image as the foreground and the fourth image as the background, perform image fusion to obtain the target image.

[0154] In this embodiment, since the third image does not contain the target subject and contains a blurred background image, it is necessary to merge the unblurred target subject into the blurred background image. Based on the first image and the fifth image (containing only the outer contour), a tenth image containing only the target subject is generated, and the target subject is selected from the outer contour. The edge texture of the target subject in the tenth image is blurred, allowing a natural transition between the target subject's edge and the filled pixels, resulting in the eleventh image. Using the eleventh image as the foreground and the fourth image as the background, image fusion is performed to obtain the target image, which contains a clear target subject and a blurred background image. By merging the target subject with blurred edge texture into the blurred background image, a natural transition occurs between the target subject and the background image without ghosting, thereby improving the image processing effect.

[0155] For example, in step S420, blurring the edge texture of the target subject in the tenth image to obtain the eleventh image can be achieved by performing Gaussian blur edge feathering on the fourth image to obtain the eleventh image.

[0156] For example, such as Figure 11 As shown, in step S430, image fusion is performed using the eleventh image as the foreground and the fourth image as the background to obtain the target image. The target subject in the target image is clear while the background image is blurred, achieving a dynamic effect without producing motion blur.

[0157] This disclosure provides an image processing method, such as... Figure 12 As shown, the method includes:

[0158] S500: Acquire the location information and optical flow information of the target subject.

[0159] S510. Based on the position information and optical flow information, determine the first motion vector of the optical flow in the first preset direction and the second motion vector in the second preset direction.

[0160] S520. Determine the blur degree and blur angle of the blur kernel based on the first motion vector and the second motion vector.

[0161] S530. The first image after fusing the acquired multi-frame images is binarized to obtain the eighth image.

[0162] S540. Based on the edge texture of the target subject in the eighth image, perform morphological processing on the eighth image to obtain the fifth image.

[0163] S550. Expand the outer contour of the fifth image multiple times to obtain the sixth image.

[0164] S560. The outer contour in the fifth image is shrunk inward multiple times to obtain the seventh image.

[0165] S570. Generate a second image based on the sixth and seventh images.

[0166] S580. Using the center point of the target as the center, divide the second region of the sixth image to obtain multiple first sub-regions.

[0167] S590. The average pixel value of each pixel in each first sub-region of the first image is taken as the target pixel value of the corresponding first sub-region.

[0168] S600. Using the center point as the center, divide the first region of the second image into multiple second sub-regions.

[0169] S610. Set the pixel value of each pixel in each second sub-region of the second image to the target pixel value of the included first sub-region to obtain the ninth image.

[0170] S620. Merge the image within the outer contour of the ninth image with the first image to obtain the third image.

[0171] S630. Create an identity matrix with both the number of rows and columns having a certain degree of fuzziness.

[0172] S640. Rotate the identity matrix according to the fuzzy angle to obtain the target matrix.

[0173] S650. Based on the target matrix, blur the third image to obtain the fourth image.

[0174] S660. Based on the first image and the fifth image which only contains the outer contour, generate a tenth image which only contains the target subject.

[0175] S670. Blur the edge texture of the target subject in the tenth image to obtain the eleventh image.

[0176] S680. Using the eleventh image as the foreground and the fourth image as the background, perform image fusion to obtain the target image.

[0177] In this embodiment, the position information and optical flow information of the target subject are acquired, and based on the position information and optical flow information, a first motion vector of the optical flow in a first preset direction and a second motion vector in a second preset direction are determined. Based on the first and second motion vectors, the blur degree and blur angle of the blur kernel are determined to perform blur processing. The first image obtained by fusing multiple acquired images is binarized to obtain an eighth image, making the target subject and background clearly distinguishable. Based on the edge texture of the target subject in the eighth image, morphological processing is performed on the eighth image to obtain a fifth image. The outer contour in the fifth image is sequentially expanded multiple times to obtain a sixth image, to determine the pixel values ​​associated with the background image for filling pixels. The outer contour in the fifth image is sequentially shrunk multiple times to obtain a seventh image, to determine the pixel filling area. Based on the sixth and seventh images, a second image is generated to display the first area to be filled. The second area of ​​the sixth image is divided with the center point of the target subject as the center, resulting in multiple first sub-regions. The average pixel value of each pixel in each first sub-region of the first image is used as the target pixel value for the corresponding first sub-region. The target pixel value is related to the background image to fill pixels using the background image. The first region of the second image is divided into multiple second sub-regions centered on the center point. The pixel value of each pixel in each second sub-region of the second image is set to the target pixel value of the included first sub-region to fill pixels, resulting in the ninth image, ensuring that the pixels at the boundary between the target subject and the background image are adapted to the background image. The image within the outer contour of the ninth image is merged with the first image to obtain the third image, which includes both the filled pixels and the background image. An identity matrix is ​​established with rows and columns equal to the degree of blur, reflecting the degree of blur. The identity matrix is ​​rotated according to the blur angle to obtain the target matrix. The third image is blurred according to the target matrix to obtain the fourth image with a blurred background image. Based on the first image and the fifth image containing only the outer contour, a tenth image containing only the target subject is generated to obtain a clear target subject. The edge texture of the target subject in the tenth image is blurred to obtain the eleventh image, allowing for a natural transition between the target subject and the blurred, filled pixels. The eleventh image is then used as the foreground and the fourth image as the background for image fusion to obtain the target image, giving it a dynamic effect. By identifying the first region that might produce motion blur and filling it with pixels related to the background image, no motion blur occurs between the target subject and the background image after blurring the background image, thus improving the image processing effect.

[0178] In one exemplary embodiment, an image processing apparatus is provided for implementing the method described above. (Reference) Figure 13As shown, the image processing apparatus may include a first processing module 100, a filling module 150, a second processing module 200, and a generation module 250. During the implementation of the above method,

[0179] The first processing module 100 is configured to expand and shrink the outer contour of the target subject in the first image to be processed to obtain a second image. The first image consists of the target subject and the background image.

[0180] The filling module 150 is configured to fill pixels in a first region between the outward expansion and inward contraction of the outer contour in a second image based on the first image and fuse them into the first image to obtain a third image, wherein the pixels filled in the first region are related to the background image.

[0181] The second processing module 200 is configured to blur the third image to obtain the fourth image.

[0182] The generation module 250 is configured to fuse the target subject in the first image with the fourth image to obtain the target image.

[0183] In one exemplary embodiment, an image processing apparatus is provided, wherein a first processing module 100 is configured to:

[0184] The first image is transformed to obtain a fifth image that only contains the outer contour.

[0185] The outer contour of the fifth image is expanded to obtain the sixth image.

[0186] The outer contour of the fifth image is shrunk inward to obtain the seventh image.

[0187] Generate the second image based on the sixth and seventh images.

[0188] In one exemplary embodiment, an image processing apparatus is provided, wherein a first processing module 100 is configured to:

[0189] The first image is binarized to obtain the eighth image.

[0190] Based on the edge texture of the target subject in the eighth image, morphological processing is performed on the eighth image to obtain the fifth image.

[0191] In the eighth image, the pixel value of the target subject is a first preset value, and the pixel value of the background image is a second preset value.

[0192] In one exemplary embodiment, an image processing apparatus is provided, wherein a first processing module 100 is configured to:

[0193] The outer contour in the fifth image is expanded outward multiple times to obtain the sixth image.

[0194] In one exemplary embodiment, an image processing apparatus is provided, wherein a first processing module 100 is configured to:

[0195] The outer contour in the fifth image is shrunk inwards multiple times to obtain the seventh image.

[0196] In one exemplary embodiment, an image processing apparatus is provided, wherein a filling module 150 is configured to:

[0197] Determine the target pixel value of the second region outside the outer contour of the first image.

[0198] Based on the target pixel value, pixels are filled into the first region of the second image to obtain the ninth image.

[0199] The third image is obtained by fusing the image within the outer contour of the ninth image with the first image.

[0200] In one exemplary embodiment, an image processing apparatus is provided, wherein a filling module 150 is configured to:

[0201] Using the center point of the target as the center, the second region is divided under the circumference to obtain multiple first sub-regions.

[0202] The average pixel value of each pixel in each first sub-region of the first image is taken as the target pixel value of the corresponding first sub-region.

[0203] In one exemplary embodiment, an image processing apparatus is provided, wherein a filling module 150 is configured to:

[0204] Using the center point as the center, the first region is divided into multiple second sub-regions along the circumference.

[0205] The pixel values ​​of each pixel in each second sub-region of the second image are set to the target pixel values ​​of the included first sub-region to fill the pixels, thus obtaining the ninth image.

[0206] In one exemplary embodiment, an image processing apparatus is provided, the apparatus further comprising:

[0207] The acquisition module is configured to acquire the location information and optical flow information of the target subject.

[0208] The determination module is configured to determine, based on position information and optical flow information, a first motion vector of optical flow in a first preset direction and a second motion vector in a second preset direction.

[0209] The degree of blur and the blur angle of the blur kernel are determined based on the first motion vector and the second motion vector.

[0210] In one exemplary embodiment, an image processing apparatus is provided, wherein a second processing module 200 is configured to:

[0211] The third image is blurred based on the degree and angle of blur to obtain the fourth image.

[0212] In one exemplary embodiment, an image processing apparatus is provided, wherein a second processing module 200 is configured to:

[0213] Create an identity matrix with both the number of rows and columns having a certain degree of fuzziness.

[0214] The target matrix is ​​obtained by rotating the identity matrix according to the fuzzy angle.

[0215] Based on the target matrix, the third image is blurred to obtain the fourth image.

[0216] In one exemplary embodiment, an image processing apparatus is provided, wherein a generation module 250 is configured to:

[0217] Based on the first image and the fifth image which only contains the outer contour, a tenth image containing only the target subject is generated.

[0218] The edge texture of the target subject in the tenth image is blurred to obtain the eleventh image.

[0219] The target image is obtained by fusing the eleventh image as the foreground and the fourth image as the background.

[0220] In one exemplary embodiment, an electronic device is provided, such as a mobile phone, a laptop computer, a tablet computer, and a wearable device.

[0221] refer to Figure 14 As shown, the electronic device 400 may include one or more of the following components: processing component 402, memory 404, power supply component 406, multimedia component 408, audio component 410, input / output (I / O) interface 412, sensor component 414, and communication component 416.

[0222] Processing component 402 typically controls the overall operation of electronic device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0223] Memory 404 is configured to store various types of data to support the operation of electronic device 400. Examples of this data include instructions for any application or method operating on electronic device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0224] Power supply component 406 provides power to various components of electronic device 400. Power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.

[0225] Multimedia component 408 includes a screen that provides an output interface between electronic device 400 and user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera module and / or a rear-facing camera module. When electronic device 400 is in an operating mode, such as shooting mode or video mode, the front-facing camera module and / or rear-facing camera module may receive external multimedia data. Each front-facing camera module and rear-facing camera module may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0226] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0227] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0228] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 may detect the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0229] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other terminals. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0230] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing terminals (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods shown in the above embodiments or combinations thereof.

[0231] In one exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to perform the methods shown in the embodiments or combinations thereof. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage terminal, etc. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is able to perform the methods shown in the embodiments or combinations thereof.

[0232] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0233] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0234] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0235] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0236] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method, characterized in that, The image processing method includes: The outer contour of the target subject in the first image to be processed is expanded outward and shrunk inward to obtain the second image. The first image is composed of the target subject and the background image. Based on the first image, pixels are filled in the first region between the outward expansion and inward contraction of the outer contour in the second image and fused into the first image to obtain a third image, wherein the pixels filled in the first region are related to the background image; The third image is blurred to obtain the fourth image; The target subject in the first image is fused with the fourth image to obtain the target image.

2. The image processing method according to claim 1, characterized in that, The process of expanding and shrinking the outer contour of the target subject in the first image to be processed to obtain the second image includes: The first image is transformed to obtain a fifth image that only contains the outer contour; The outer contour in the fifth image is expanded to obtain the sixth image; The outer contour in the fifth image is shrunk inward to obtain the seventh image; The second image is generated based on the sixth image and the seventh image.

3. The image processing method according to claim 2, characterized in that, The process of converting the first image to obtain a fifth image containing only the outer contour includes: The first image is binarized to obtain the eighth image; Based on the edge texture of the target subject in the eighth image, morphological processing is performed on the eighth image to obtain the fifth image; In the eighth image, the pixel value of the target subject is a first preset value, and the pixel value of the background image is a second preset value.

4. The image processing method according to claim 2, characterized in that, The step of expanding the outer contour in the fifth image to obtain the sixth image includes: The outer contour in the fifth image is expanded outward multiple times to obtain the sixth image; and / or, The step of shrinking the outer contour in the fifth image to obtain the seventh image includes: The outer contour in the fifth image is shrunk inward multiple times to obtain the seventh image.

5. The image processing method according to claim 4, characterized in that, The second image includes the largest outer contour in the sixth image and the smallest outer contour in the seventh image. The first region is the area between the largest and smallest outer contours, and the width of the first region is greater than or equal to the blur degree of the blur kernel used for blur processing.

6. The image processing method according to claim 1, characterized in that, The step of filling pixels in the first region between the outward expansion and inward contraction of the outer contour in the second image and fusing it into the first image to obtain a third image includes: Determine the target pixel value of the first image in the second region extending beyond the outer contour; Based on the target pixel value, pixels are filled into the first region of the second image to obtain the ninth image; The image within the outer contour of the ninth image is fused with the first image to obtain the third image.

7. The image processing method according to claim 6, characterized in that, The outer contour expands outward multiple times, and the second region is the area between the largest and smallest outwardly expanded outer contours. Determining the target pixel value of the first image in the second region extending beyond the outer contour includes: Using the center point of the target entity as the center, the second region is divided under the circumference to obtain multiple first sub-regions; The average pixel value of each pixel in each first sub-region of the first image is taken as the target pixel value of the corresponding first sub-region.

8. The image processing method according to claim 7, characterized in that, The step of filling the first region of the second image with pixels according to the target pixel value to obtain the ninth image includes: Using the center point as the center, the first region is divided into multiple second sub-regions along the circumference; The pixel value of each pixel in each second sub-region of the second image is set to the target pixel value of the included first sub-region to fill the pixel, thus obtaining the ninth image.

9. The image processing method according to claim 1, characterized in that, Before blurring the third image to obtain the fourth image, the image processing method further includes: Obtain the location information and optical flow information of the target entity; Based on the position information and the optical flow information, a first motion vector of the optical flow in a first preset direction and a second motion vector in a second preset direction are determined; Based on the first motion vector and the second motion vector, determine the blur degree and blur angle of the blur kernel; The process of blurring the third image to obtain the fourth image includes: The third image is blurred based on the degree of blur and the blur angle to obtain the fourth image.

10. The image processing method according to claim 9, characterized in that, The step of blurring the third image based on the degree of blur and the blur angle to obtain the fourth image includes: Establish an identity matrix with the same number of rows and columns as the desired level of fuzziness; The identity matrix is ​​rotated according to the fuzzy angle to obtain the target matrix; The third image is blurred based on the target matrix to obtain the fourth image.

11. The image processing method according to any one of claims 1 to 10, characterized in that, The step of fusing the target subject in the first image with the fourth image to obtain the target image includes: Based on the first image and the fifth image which only contains the outer contour, a tenth image containing only the target subject is generated; The edge texture of the target subject in the tenth image is blurred to obtain the eleventh image; The target image is obtained by fusing the eleventh image as the foreground and the fourth image as the background.

12. An image processing apparatus, characterized in that, The image processing device includes: A first processing module is configured to expand and shrink the outer contour of the target subject in the first image to be processed to obtain a second image, wherein the first image is composed of the target subject and a background image. A filling module is configured to fill pixels in a first region between the outward expansion and inward contraction of the outer contour in a second image based on the first image and fuse it into the first image to obtain a third image, wherein the pixels filled in the first region are related to the background image. A second processing module is configured to blur the third image to obtain a fourth image. A generation module is configured to fuse the target subject in the first image with the fourth image to obtain a target image.

13. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to perform the image processing method as described in any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the terminal, the terminal is able to perform the image processing method as described in any one of claims 1 to 11.

Images