Image processing method and apparatus

Abstract

The present disclosure relates to an image processing method and device. The image processing method comprises: determining an object segmentation mask image corresponding to an object region in a to-be-processed image according to the object region, wherein the object segmentation mask image is an image obtained by removing a region other than the object region from the to-be-processed image; determining a boundary pixel of the object region based on the object segmentation mask image; determining a fill color of the object region in the to-be-processed image based on the boundary pixel; and rendering the object region in the to-be-processed image based on the fill color to obtain a first target image. According to the image processing method and device of the present disclosure, the boundary background pixel color of each pixel in a portrait can be found, and the background fill color can be predicted, thereby improving the background color fill efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of image and video technology. More specifically, this disclosure relates to an image processing method and apparatus. Background Technology

[0002] Removing people from an image and filling them in based on the surrounding image is a common technique in image processing. Applying this technique to special effects can provide users with a virtual effect where the person gradually disappears into the background. Common portrait removal methods often use neural networks to fill in the person's area based on semantic information from the background image. These methods have extremely high computational complexity, making them difficult to apply to real-time computing scenarios such as video effects. Summary of the Invention

[0003] Exemplary embodiments of this disclosure provide an image processing method and apparatus to at least solve the problem of low filling efficiency caused by high computational complexity when filling background color of object areas in related technologies.

[0004] According to an exemplary embodiment of this disclosure, an image processing method is provided, comprising: determining an object segmentation mask image corresponding to an object region in an image to be processed, wherein the object segmentation mask image is an image obtained by removing a region other than the object region from the image to be processed; determining boundary pixels of the object region based on the object segmentation mask image; determining a fill color of the object region in the image to be processed based on the boundary pixels; and rendering the object region in the image to be processed based on the fill color to obtain a first target image.

[0005] Optionally, after processing the object region in the image to be processed based on the fill color to obtain a first target image, the image processing method may further include: performing mean fuzzy sampling processing on the first target image according to the texture coordinates of the first target image to obtain a mean fuzzy sampling processing result; and processing the object region in the image to be processed based on the mean fuzzy sampling processing result to obtain a second target image.

[0006] Optionally, the step of performing mean blur sampling on the first target image based on the texture coordinates of the first target image to obtain the mean blur sampling result may include: performing texture coordinate offset processing on each pixel in the object region of the first target image to obtain the offset texture coordinates; and performing mean blur sampling based on the offset texture coordinates of the first target image to obtain the mean blur sampling result.

[0007] Optionally, performing texture coordinate offset processing on each pixel in the object region of the first target image may include: determining the texture coordinate offset value of the corresponding pixel in the first target image based on the color of each pixel in the object region in the image to be processed; and performing texture coordinate offset processing on the corresponding pixel in the first target image based on the texture coordinate offset value of the corresponding pixel in the first target image.

[0008] Optionally, determining the texture coordinate offset value of the corresponding pixel in the first target image based on the color of each pixel in the object region in the image to be processed may include performing the following operations for each pixel in the object region: determining the red channel value and green channel value of the color of the pixel in the object region in the image to be processed; and determining the texture coordinate offset value of the pixel corresponding to the pixel in the first target image based on the red channel value and the green channel value, thereby obtaining the texture coordinate offset value of the corresponding pixel in the first target image.

[0009] Optionally, determining the fill color of the object region in the image to be processed based on the boundary pixels may include: determining the color of the boundary pixels corresponding to each pixel in the object region; and determining the fill color of each pixel in the object region based on the color of the boundary pixels corresponding to each pixel in the object region.

[0010] Optionally, determining the fill color of each pixel in the object region based on the color of the boundary pixel corresponding to each pixel in the object region may include: determining any pixel in the object region as a target pixel; determining the mixing ratio of the colors of the boundary pixels corresponding to the target pixel to obtain the color mixing ratio corresponding to the target pixel; and mixing the colors of the boundary pixels corresponding to the target pixel with the colors of the boundary pixels corresponding to the target pixel based on the color mixing ratio corresponding to the target pixel to obtain the fill color of the target pixel.

[0011] According to an exemplary embodiment of the present disclosure, an image processing apparatus is provided, comprising: an image segmentation unit configured to determine an object segmentation mask image corresponding to an object region in an image to be processed, wherein the object segmentation mask image is an image obtained by removing a region other than the object region from the image to be processed; a boundary determination unit configured to determine boundary pixels of the object region based on the object segmentation mask image; a color determination unit configured to determine a fill color of the object region in the image to be processed based on the boundary pixels; and an image processing unit configured to process the object region in the image to be processed based on the fill color to obtain a first target image.

[0012] Optionally, the image processing apparatus may further include another processing unit configured to: perform mean fuzzy sampling processing on the first target image according to the texture coordinates of the first target image to obtain a mean fuzzy sampling processing result; and process the object region in the image to be processed based on the mean fuzzy sampling processing result to obtain a second target image.

[0013] Optionally, the other processing unit may be configured to: perform texture coordinate offset processing on each pixel in the object region of the first target image to obtain offset texture coordinates; and perform mean blur sampling processing based on the offset texture coordinates of the first target image to obtain mean blur sampling processing results.

[0014] Optionally, the other processing unit may be configured to: determine the texture coordinate offset value of the corresponding pixel in the first target image based on the color of each pixel in the object region in the image to be processed; and perform texture coordinate offset processing on the corresponding pixel in the first target image based on the texture coordinate offset value of the corresponding pixel in the first target image.

[0015] Optionally, the other processing unit may be configured to perform the following operations for each pixel in the object region: determine the red channel value and green channel value of the color of the pixel in the object region in the image to be processed; and based on the red channel value and the green channel value, determine the texture coordinate offset value of the pixel corresponding to the pixel in the first target image to obtain the texture coordinate offset value of the corresponding pixel in the first target image.

[0016] Optionally, the color determination unit may be configured to: determine the color of the boundary pixel corresponding to each pixel in the object region; and determine the fill color of each pixel in the object region based on the color of the boundary pixel corresponding to each pixel in the object region.

[0017] Optionally, the color determination unit may be configured to: determine any pixel in the object region as a target pixel; determine the color mixing ratio of the boundary pixels corresponding to the target pixel to obtain the color mixing ratio corresponding to the target pixel; and, based on the color mixing ratio corresponding to the target pixel, mix the colors of the boundary pixels corresponding to the target pixel to obtain the fill color of the target pixel.

[0018] According to an exemplary embodiment of the present disclosure, an electronic device is provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement an image processing method according to an exemplary embodiment of the present disclosure.

[0019] According to exemplary embodiments of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to perform an image processing method according to exemplary embodiments of the present disclosure.

[0020] According to exemplary embodiments of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement an image processing method according to exemplary embodiments of the present disclosure.

[0021] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0022] By first finding the colors of the nearest background pixels on both sides of each pixel in the portrait, the color after background filling is predicted, thereby improving the efficiency of background color filling.

[0023] The texture coordinates are offset based on the portrait color, and mean blur is used to create a "transparent person" effect, thereby improving the transparency of the portrait.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0026] Figure 1 Exemplary system architectures in which exemplary embodiments of this disclosure can be applied are shown.

[0027] Figure 2 A flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure is shown.

[0028] Figure 3 An example of an image to be processed according to an exemplary embodiment of this disclosure is shown.

[0029] Figure 4 An example of an object segmentation mask image is shown according to an exemplary embodiment of the present disclosure.

[0030] Figure 5 An example of color filling results according to an exemplary embodiment of this disclosure is shown.

[0031] Figure 6 A flowchart illustrating an image processing method according to another exemplary embodiment of the present disclosure is shown.

[0032] Figure 7 Examples of processing results according to exemplary embodiments of this disclosure are shown.

[0033] Figure 8 A block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure is shown.

[0034] Figure 9 A block diagram of an image processing apparatus according to another exemplary embodiment of the present disclosure is shown.

[0035] Figure 10 This is a block diagram of an electronic device 1000 according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0037] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0038] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0039] In related technologies, image features are extracted and filled into human figures using methods such as neural networks. However, these methods have high computational complexity and are difficult to apply to real-time scenarios such as video effects.

[0040] Furthermore, in related technologies, the method of removing the human figure and then filling it in produces an image that lacks a sense of three-dimensionality, makes it difficult to reflect the texture of the human figure's surface, and cannot achieve the effect of turning the human figure into transparent glass.

[0041] This disclosure proposes a solution that does not require the use of complex methods such as neural networks to extract semantic features of images. Instead, it uses a graphics processing unit (GPU) to efficiently and in parallel render the colors of the human portrait area, achieving real-time rendering efficiency and meeting the needs of application scenarios such as video effects.

[0042] This disclosure proposes a method that simulates the unevenness of a person's surface based on the color of the person in the image, giving the drawn transparent portrait a frosted glass texture, thereby achieving a three-dimensional effect in the transparent portrait.

[0043] Below, we will refer to Figures 1 to 10 A detailed description of an image processing method and apparatus according to exemplary embodiments of the present disclosure.

[0044] Figure 1 An exemplary system architecture 100 is shown that can be applied to an exemplary embodiment of this disclosure.

[0045] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables. Users can use terminal devices 101, 102, and 103 to interact with server 105 through network 104 to receive or send messages (e.g., image upload requests, image processing requests, etc.). Various image applications, such as audio / video playback software, audio / video recording software, audio / video processing software, and audio / video editing software, may be installed on terminal devices 101, 102, and 103. Terminal devices 101, 102, and 103 may be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays capable of audio and video playback, recording, and editing, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module. No specific limitations are made here.

[0046] Terminal devices 101, 102, and 103 may be equipped with image acquisition devices (e.g., cameras) to capture video data. In practice, the smallest visual unit that makes up video is a frame. Each frame is a static image. Combining temporally consecutive frame sequences creates a dynamic video. Furthermore, terminal devices 101, 102, and 103 may also be equipped with components for converting electrical signals into sound (e.g., speakers) to play sound, and may also be equipped with devices for converting analog audio signals into digital audio signals (e.g., microphones) to capture sound.

[0047] Server 105 can be a server that provides various services, such as a backend server that supports applications installed on terminal devices 101, 102, and 103. The backend server can parse, store, and process received image processing requests and other data, and can also receive image processing requests sent by terminal devices 101, 102, and 103, and feed back the processing results to terminal devices 101, 102, and 103.

[0048] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0049] It should be noted that the image processing method provided in this disclosure is typically executed by a terminal device, but it can also be executed by a server, or it can be executed collaboratively by a terminal device and a server. Accordingly, the image processing apparatus can be located in the terminal device, the server, or both.

[0050] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Any number of terminal devices, networks, and servers can be used as needed, and this disclosure places no limitation on this.

[0051] Figure 2 A flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure is shown. Figure 3 An example of an image to be processed according to an exemplary embodiment of this disclosure is shown. Figure 4 An example of an object segmentation mask image is shown according to an exemplary embodiment of the present disclosure. Figure 5 An example of color filling results according to an exemplary embodiment of this disclosure is shown.

[0052] Reference Figure 2In step S201, an object segmentation mask image corresponding to the object region is determined based on the object region in the image to be processed. Here, the object segmentation mask image is an image obtained by removing the region other than the object region from the image to be processed.

[0053] Here, objects in the object area can be people, animals, plants, etc. For example, Figure 3 The object in the image to be processed is a person. For example... Figure 4 As shown, in the object segmentation mask image, the object region is displayed normally, while the area outside the object region is black. In the background image, the object region is black, while the area outside the object region is displayed normally.

[0054] In exemplary embodiments of this disclosure, any object segmentation algorithm (or image segmentation algorithm) can be used to segment object regions in the image to be processed, and this disclosure does not impose any limitations on this.

[0055] In step S202, the boundary pixels of the object region are determined based on the object segmentation mask image. For example, the boundary pixels on two preset opposite sides of the object region are determined based on the object segmentation mask image.

[0056] In an exemplary embodiment of this disclosure, after obtaining the object segmentation mask image, an object segmentation mask texture map can also be determined based on the object segmentation mask image. In the object segmentation mask texture map, the grayscale value of each pixel represents the probability that the object segmentation algorithm predicts that the pixel location in the image to be processed belongs to an object. The boundary pixels of the object region (e.g., preset boundary pixels on opposite sides) can be determined based on the object segmentation mask texture map.

[0057] When the object in the object region is a person, the object segmentation mask texture map is also called the human body segmentation mask texture map, and the object region is also called the human image region.

[0058] As an example, when determining the preset relative boundary pixels of the portrait region based on the human body segmentation mask texture map M, for pixels P in the human body segmentation mask texture map M with a gray level greater than 0, the leftmost pixel P belonging to the portrait region is searched to the left and right respectively. left and the rightmost pixel P right The specific steps are as follows:

[0059] 2.1 Create an Open Graphics Library (OpenGL) shader (Shader1) to render the estimated color filling result of the human figure region. In shader (Shader1), the user-uploaded image to be processed I and the human body segmentation mask M are sampled according to the current texture coordinates.

[0060] 2.2 When the sampling result obtained in step 2.1 is greater than 0, the texture coordinate P is iteratively shifted to the left and right respectively, with each shift... The human body segmentation mask image M is sampled.

[0061] 2.3 Repeat step 2.2 until the color value of the sampled result is equal to 0, thus obtaining the leftmost pixel of the current portrait area with an accuracy range within 0.1. and the rightmost pixel

[0062] 2.4 Based on the leftmost and rightmost pixels when the precision range obtained in step 2.3 is within 0.1, continue to iteratively shift the texture coordinate P to the left and right respectively, with each shift... The human body segmentation mask image M is sampled, with the initial value of the offset step size s being 0.05.

[0063] 2.5 When the sampling result obtained in step 2.4 is greater than 0, shift the texture coordinate P to the left or right. When the sampling result is equal to 0, no texture coordinate offset is applied. At the same time, the offset step size s is reduced to 0.5 times the original size.

[0064] 2.6 Repeat steps 2.4 and 2.5 until the offset step size s is less than the precision 0.001. At this point, the texture coordinate P is obtained, which is the leftmost pixel P of the portrait. left and the rightmost pixel P right .

[0065] In step S203, the fill color of the object region in the image to be processed is determined based on the boundary pixels.

[0066] In an exemplary embodiment of this disclosure, when determining the fill color of the object region in the image to be processed based on the boundary pixels, the color of the boundary pixel corresponding to each pixel in the object region can be determined first, and then the fill color of each pixel in the object region can be determined based on the color of the boundary pixel corresponding to each pixel in the object region.

[0067] In an exemplary embodiment of this disclosure, when determining the fill color of each pixel in the object region based on the color of the boundary pixel corresponding to each pixel in the object region, any pixel in the object region can first be determined as a target pixel, then the mixing ratio of the colors of the boundary pixels corresponding to the target pixel is determined to obtain the color mixing ratio corresponding to the target pixel, and based on the color mixing ratio corresponding to the target pixel, the colors of the boundary pixels corresponding to the target pixel are mixed to obtain the fill color of the target pixel.

[0068] As an example, in the shader (Shader1), the user-uploaded image I to be processed is processed in P... left and P right Samples were taken separately to obtain color value I. left and I right The color mixing ratio of the leftmost and rightmost pixels in the portrait area is r.

[0069]

[0070] Based on the blending ratio, the colors of the leftmost and rightmost pixels in the portrait area are blended to obtain the fill color I at pixel P. P .

[0071] I p =I left ·(1-r)+I right ·r

[0072] In step S204, the object region in the image to be processed is processed based on the fill color to obtain the first target image.

[0073] For example, using the aforementioned shader (Shader1) to fill the object region with color (i.e., render), the resulting image O (e.g., the filled image of the portrait region) is drawn. Figure 5 (Color fill result in the image).

[0074] Figure 6 A flowchart illustrating an image processing method according to another exemplary embodiment of the present disclosure is shown. Figure 7 Examples of processing results according to exemplary embodiments of this disclosure are shown.

[0075] Reference Figure 6 In step S601, an object segmentation mask image corresponding to the object region is determined based on the object region in the image to be processed. Here, the object segmentation mask image is an image obtained by removing the region other than the object region from the image to be processed.

[0076] Here, objects in the object area can be people, animals, plants, etc. For example, Figure 3 The object in the image to be processed is a person.

[0077] In exemplary embodiments of this disclosure, any object segmentation algorithm (or image segmentation algorithm) can be used to segment object regions in the image to be processed, and this disclosure does not impose any limitations on this.

[0078] In step S602, the boundary pixels of the object region are determined based on the object segmentation mask image. For example, the boundary pixels on two preset opposite sides of the object region are determined based on the object segmentation mask image.

[0079] In an exemplary embodiment of this disclosure, after obtaining the object segmentation mask image, an object segmentation mask texture map can also be determined based on the object segmentation mask image. In the object segmentation mask texture map, the grayscale value of each pixel represents the probability that the object segmentation algorithm predicts that the pixel location in the image to be processed belongs to an object. The boundary pixels of the object region (e.g., preset boundary pixels on opposite sides) can be determined based on the object segmentation mask texture map.

[0080] When the object in the object region is a person, the object segmentation mask texture map is also called the human body segmentation mask texture map, and the object region is also called the human image region.

[0081] For example, when determining the preset boundary pixels on both sides of a human image region based on a human body segmentation mask texture map M, for pixels P in the human body segmentation mask texture map M with a gray level greater than 0, the leftmost pixel P belonging to the human image region can be searched to the left and right respectively. left and the rightmost pixel P right .

[0082] In step S603, the fill color of the object region in the image to be processed is determined based on the boundary pixels.

[0083] In an exemplary embodiment of this disclosure, when determining the fill color of the object region in the image to be processed based on the boundary pixels, the color of the boundary pixel corresponding to each pixel in the object region can be determined first, and then the fill color of each pixel in the object region can be determined based on the color of the boundary pixel corresponding to each pixel in the object region.

[0084] In an exemplary embodiment of this disclosure, when determining the fill color of each pixel in the object region based on the color of the boundary pixel corresponding to each pixel in the object region, any pixel in the object region can first be determined as a target pixel, then the mixing ratio of the colors of the boundary pixels corresponding to the target pixel is determined to obtain the color mixing ratio corresponding to the target pixel, and based on the color mixing ratio corresponding to the target pixel, the colors of the boundary pixels corresponding to the target pixel are mixed to obtain the fill color of the target pixel.

[0085] For example, in the shader (Shader1), the user-uploaded image I to be processed is processed in P... left and P right Samples were taken separately to obtain color value I. left and L right The color mixing ratio of the leftmost and rightmost pixels in the portrait area is r.

[0086]

[0087] Based on the blending ratio, the colors of the leftmost and rightmost pixels in the portrait area are blended to obtain the fill color I at pixel P. P .

[0088] I P =I left ·(1-r)+I right ·r

[0089] In step S604, the object region in the image to be processed is processed based on the fill color to obtain the first target image.

[0090] For example, using the aforementioned shader (Shader1) to render the object region, the result is as follows: Figure 5 The result of filling the human portrait area is shown in Figure O.

[0091] In step S605, the first target image is subjected to mean blur sampling processing based on the texture coordinates of the first target image to obtain the mean blur sampling processing result.

[0092] In an exemplary embodiment of this disclosure, when performing mean blur sampling on the first target image based on its texture coordinates to obtain a mean blur sampling result, the texture coordinates of each pixel in the object region of the first target image can first be offset to obtain offset texture coordinates. Then, mean blur sampling is performed based on the offset texture coordinates of the first target image to obtain a mean blur sampling result. For example, when performing mean blur sampling based on the offset texture coordinates of the first target image, the offset texture coordinates can first be sampled at preset intervals to obtain interval sampling results. Then, the mean of the interval sampling results is calculated to obtain the mean blur sampling result.

[0093] In an exemplary embodiment of this disclosure, when performing texture coordinate offset processing on each pixel in the object region of the first target image, the texture coordinate offset value of the corresponding pixel in the first target image can be determined first based on the color of each pixel in the object region in the image to be processed, and then the texture coordinate offset processing is performed on the corresponding pixel in the first target image based on the texture coordinate offset value of the corresponding pixel in the first target image.

[0094] In an exemplary embodiment of this disclosure, when determining the texture coordinate offset value of a corresponding pixel in the first target image based on the color of each pixel in the object region in the image to be processed, the following operations can be performed for each pixel in the object region: first, determine the red channel value and green channel value of the color of the pixel in the object region in the image to be processed; then, based on the red channel value and the green channel value, determine the texture coordinate offset value of the pixel corresponding to the pixel in the first target image, thereby obtaining the texture coordinate offset value of the corresponding pixel in the first target image.

[0095] As an example, an OpenGL shader (Shader2) is created to render the human figure transparency effect for a user-uploaded image I. In Shader2, the user-uploaded image I, the human body segmentation mask map M, and the human figure region filling result map O are sampled according to the current texture coordinates. For pixels P in the human body segmentation mask texture map M with a grayscale value greater than 0, the color I of pixel P in the user-uploaded image I is used as the reference point. P The texture coordinate offset value of pixel P in the portrait area filling result map O is calculated as follows: Δuv = 0.1·(I P .r,I P .g).

[0096] As an example, the color mean O′ is calculated by sampling every 90 degrees around the texture coordinates of pixel P after offset in the resulting image O filled in the portrait region. P .

[0097]

[0098] In step S606, the object region in the image to be processed is processed based on the mean fuzzy sampling processing result to obtain the second target image.

[0099] Using the above sampled color mean O′ P This is the result of the processing (i.e., rendering) by Shader2. Each pixel of the human body region in the user-uploaded image is processed (i.e., rendered) using Shader2 to create a transparent image of the human body region (e.g., ...). Figure 7 (Processing results).

[0100] The above has been combined Figures 1 to 7 An image processing method according to exemplary embodiments of the present disclosure has been described. Hereinafter, reference will be made to... Figure 8 and Figure 9 An image processing apparatus and its units according to exemplary embodiments of the present disclosure will be described.

[0101] Figure 8 A block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure is shown.

[0102] Reference Figure 8 The image processing device includes an image segmentation unit 81, a boundary determination unit 82, a color determination unit 83, and an image processing unit 84.

[0103] The image segmentation unit 81 is configured to determine an object segmentation mask image corresponding to an object region in the image to be processed, wherein the object segmentation mask image is an image obtained by removing the region other than the object region from the image to be processed.

[0104] The boundary determination unit 82 is configured to determine the boundary pixels of the object region based on the object segmentation mask image.

[0105] The color determination unit 83 is configured to determine the fill color of the object region in the image to be processed based on the boundary pixels.

[0106] In an exemplary embodiment of this disclosure, the color determination unit 83 may be configured to: determine the color of the boundary pixel corresponding to each pixel in the object region; and determine the fill color of each pixel in the object region based on the color of the boundary pixel corresponding to each pixel in the object region.

[0107] In an exemplary embodiment of this disclosure, the color determination unit 83 may be configured to: determine any pixel in the object region as a target pixel; determine the color mixing ratio of the boundary pixels corresponding to the target pixel to obtain the color mixing ratio corresponding to the target pixel; and mix the colors of the boundary pixels corresponding to the target pixel with the color mixing ratio corresponding to the target pixel to obtain the fill color of the target pixel.

[0108] The image processing unit 84 is configured to process the object region in the image to be processed based on the fill color to obtain a first target image.

[0109] Figure 9 A block diagram of an image processing apparatus according to another exemplary embodiment of the present disclosure is shown.

[0110] Reference Figure 9 The image processing apparatus includes an image segmentation unit 91, a boundary determination unit 92, a color determination unit 93, an image processing unit 94, and another processing unit 95.

[0111] The image segmentation unit 91 is configured to determine an object segmentation mask image corresponding to an object region in the image to be processed, wherein the object segmentation mask image is an image obtained by removing the region other than the object region from the image to be processed.

[0112] The boundary determination unit 92 is configured to determine the boundary pixels of the object region based on the object segmentation mask image.

[0113] The color determination unit 93 is configured to determine the fill color of the object region in the image to be processed based on the boundary pixels.

[0114] In an exemplary embodiment of this disclosure, the color determination unit 93 may be configured to: determine the color of the boundary pixel corresponding to each pixel in the object region; and determine the fill color of each pixel in the object region based on the color of the boundary pixel corresponding to each pixel in the object region.

[0115] In an exemplary embodiment of this disclosure, the color determination unit 93 may be configured to: determine any pixel in the object region as a target pixel; determine the color mixing ratio of the boundary pixels corresponding to the target pixel to obtain the color mixing ratio corresponding to the target pixel; and mix the colors of the boundary pixels corresponding to the target pixel with the color mixing ratio corresponding to the target pixel to obtain the fill color of the target pixel.

[0116] The image processing unit 94 is configured to process the object region in the image to be processed based on the fill color to obtain a first target image.

[0117] Another processing unit 95 is configured to perform mean blur sampling on the first target image according to the texture coordinates of the first target image to obtain the mean blur sampling result, and process the object region in the image to be processed based on the mean blur sampling result to obtain the second target image.

[0118] In an exemplary embodiment of this disclosure, the other processing unit 95 may be configured to: perform texture coordinate offset processing on each pixel in the object region of the first target image to obtain offset texture coordinates; and perform mean blur sampling processing based on the offset texture coordinates of the first target image to obtain mean blur sampling processing result.

[0119] In an exemplary embodiment of this disclosure, the other processing unit 95 may be configured to: determine the texture coordinate offset value of the corresponding pixel in the first target image based on the color of each pixel in the object region in the image to be processed; and perform texture coordinate offset processing on the corresponding pixel in the first target image based on the texture coordinate offset value of the corresponding pixel in the first target image.

[0120] In an exemplary embodiment of this disclosure, the other processing unit 95 may be configured to perform the following operations for each pixel in the object region: determine the red channel value and green channel value of the color of the pixel in the object region in the image to be processed; and based on the red channel value and the green channel value, determine the texture coordinate offset value of the pixel corresponding to the pixel in the first target image to obtain the texture coordinate offset value of the corresponding pixel in the first target image.

[0121] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0122] The above has been combined Figure 8 and Figure 9 An image processing apparatus according to exemplary embodiments of the present disclosure has been described. Next, in conjunction with... Figure 10 An electronic device according to exemplary embodiments of the present disclosure will be described.

[0123] Figure 10 This is a block diagram of an electronic device 1000 according to an exemplary embodiment of the present disclosure.

[0124] Reference Figure 10 The electronic device 1000 includes at least one memory 1001 and at least one processor 1002. The at least one memory 1001 stores a set of computer-executable instructions, which, when executed by the at least one processor 1002, perform an image processing method according to an exemplary embodiment of the present disclosure.

[0125] In exemplary embodiments of this disclosure, electronic device 1000 may be a PC computer, tablet device, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 1000 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 1000 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0126] In electronic device 1000, processor 1002 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0127] The processor 1002 can execute instructions or code stored in the memory 1001, which can also store data. Instructions and data can also be sent and received via a network through a network interface device, which can employ any known transmission protocol.

[0128] The memory 1001 may be integrated with the processor 1002, for example, by arranging RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 1001 may include a separate device, such as an external disk drive, a storage array, or other storage device usable by any database system. The memory 1001 and the processor 1002 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 1002 to read files stored in the memory.

[0129] In addition, the electronic device 1000 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 1000 can be interconnected via a bus and / or network.

[0130] According to exemplary embodiments of this disclosure, a computer-readable storage medium including instructions is also provided, such as a memory 1001 including instructions, which can be executed by a processor 1002 of an electronic device 1000 to perform the above-described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0131] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, the computer program product including a computer program / instructions that, when executed by a processor, implement an image processing method according to exemplary embodiments of the present disclosure.

[0132] The above has been referred to Figures 1 to 10 An image processing method and apparatus according to exemplary embodiments of the present disclosure have been described. However, it should be understood that: Figure 8 and Figure 9 The image processing apparatus and its units shown can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. Figure 10The electronic device shown is not limited to the components shown above, but some components may be added or removed as needed, and the above components may also be combined.

[0133] According to the image processing method and apparatus of this disclosure, by determining an object segmentation mask image corresponding to an object region in an image to be processed, wherein the object segmentation mask image is an image obtained by removing the region other than the object region from the image to be processed, determining the boundary pixels of the object region based on the object segmentation mask image, determining the fill color of the object region in the image to be processed based on the boundary pixels, and rendering the object region in the image to be processed based on the fill color to obtain a first target image, thereby improving the background color filling efficiency by finding the relative side boundary of the human image region pixels in the image and estimating the fill color after removing the human image based on the background color value at the boundary.

[0134] Furthermore, according to the image processing method and apparatus disclosed herein, the texture coordinates can be shifted based on the portrait color, and a "transparent person" effect can be created by combining mean blurring, and a frosted glass texture can be added to the transparent portrait, thereby improving the image processing effect.

[0135] Furthermore, the image processing method disclosed herein can be implemented in parallel on a GPU, the computation process is efficient and can meet the needs of real-time application scenarios, and the generated image is exquisite with a frosted glass three-dimensional texture.

[0136] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0137] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: Based on the object region in the image to be processed, an object segmentation mask image corresponding to the object region is determined, wherein the object segmentation mask image is an image obtained by removing the region other than the object region from the image to be processed; The boundary pixels of the object region are determined based on the object segmentation mask image; Based on the boundary pixels, determine the fill color of the object region in the image to be processed; The object region in the image to be processed is processed based on the fill color to obtain a first target image; Based on the texture coordinates of the first target image, mean blur sampling is performed on the first target image to obtain the mean blur sampling result. The object region in the image to be processed is processed based on the mean fuzzy sampling result to obtain the second target image.

2. The image processing method according to claim 1, characterized in that, The step of performing mean blur sampling on the first target image based on its texture coordinates to obtain the mean blur sampling result includes: Each pixel in the object region of the first target image is subjected to texture coordinate offset processing to obtain the offset texture coordinates of the first target image; The mean blur sampling process is performed based on the offset texture coordinates of the first target image to obtain the mean blur sampling result.

3. The image processing method according to claim 2, characterized in that, The step of performing texture coordinate offset processing on each pixel in the object region of the first target image includes: Based on the color of each pixel in the object region in the image to be processed, determine the texture coordinate offset value of the corresponding pixel in the first target image; Based on the texture coordinate offset value of the corresponding pixel in the first target image, the texture coordinate offset processing is performed on the corresponding pixel in the first target image.

4. The image processing method according to claim 3, characterized in that, The step of determining the texture coordinate offset value of the corresponding pixel in the first target image based on the color of each pixel in the object region in the image to be processed includes performing the following operations for each pixel in the object region: Determine the red channel and green channel values ​​of the pixels in the object region in the image to be processed; Based on the red channel value and the green channel value, the texture coordinate offset value of the pixel corresponding to the pixel in the first target image is determined, and the texture coordinate offset value of the corresponding pixel in the first target image is obtained.

5. The image processing method according to claim 1, characterized in that, Determining the fill color of the object region in the image to be processed based on the boundary pixels includes: Determine the color of the boundary pixel corresponding to each pixel in the object region; The fill color of each pixel in the object region is determined based on the color of the boundary pixel corresponding to each pixel in the object region.

6. The image processing method according to claim 5, characterized in that, Determining the fill color of each pixel in the object region based on the color of the boundary pixels corresponding to each pixel in the object region includes: Define any pixel in the object region as the target pixel; Determine the color mixing ratio of the boundary pixels corresponding to the target pixel to obtain the color mixing ratio corresponding to the target pixel; Based on the color mixing ratio corresponding to the target pixel, the colors of the boundary pixels corresponding to the target pixel are mixed to obtain the fill color of the target pixel.

7. An image processing apparatus, characterized in that, include: The image segmentation unit is configured to determine an object segmentation mask image corresponding to an object region in an image to be processed, wherein the object segmentation mask image is an image obtained by removing the region other than the object region from the image to be processed; A boundary determination unit is configured to determine the boundary pixels of the object region based on the object segmentation mask image; The color determination unit is configured to determine the fill color of the object region in the image to be processed based on the boundary pixels; An image processing unit is configured to process the object region in the image to be processed based on the fill color to obtain a first target image; and Another processing unit is configured to perform mean blur sampling on the first target image according to the texture coordinates of the first target image to obtain the mean blur sampling result, and process the object region in the image to be processed based on the mean blur sampling result to obtain the second target image.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor of the electronic device, the electronic device performs the image processing method as described in any one of claims 1 to 6.

Images