Image processing method, apparatus, device, storage medium and program product

Abstract

This application provides an image processing method, apparatus, device, storage medium, and program product. The method includes: acquiring probability information corresponding to each pixel in a first image; determining classification information and confidence information corresponding to each pixel in the first image based on the probability information; dividing the first image into a first person layer image and a first background layer image based on the classification information; enhancing the image quality of the first person layer image and the first background layer image by calling different parameters to obtain a second person layer image and a second background layer image; determining the image mixing weight corresponding to each pixel based on the confidence information; and mixing the second person layer image and the second background layer image based on the image mixing weight to obtain a second image. This method improves the display effect of the image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of high-resolution display devices and video processing technologies, users have increasingly higher demands for image quality. Currently, image quality can be improved by adjusting pixel display parameters such as contrast, saturation, and sharpness.

[0003] However, traditional image enhancement techniques often use global parameter adjustments (such as uniform contrast, saturation, or sharpening parameters), which makes it difficult to take into account the different needs of different areas in the image, resulting in poor image display. Summary of the Invention

[0004] This application provides an image processing method, apparatus, device, storage medium, and program product that can improve the display effect of images.

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] Obtain the probability information corresponding to each pixel in the first image. The probability information is used to represent the probability that a pixel belongs to a person pixel.

[0007] Based on the probability information corresponding to each pixel in the first image, the classification information and confidence information corresponding to each pixel in the first image are determined.

[0008] Based on the classification information corresponding to each pixel in the first image, the first image is divided into a first person layer image and a first background layer image.

[0009] By applying different parameters to enhance the image quality of the first person layer image and the first background layer image respectively, the second person layer image and the second background layer image are obtained.

[0010] Based on the confidence information corresponding to each pixel, determine the image mixing weights corresponding to each pixel;

[0011] Based on the image blending weights corresponding to each pixel, the second person layer image and the second background layer image are blended to obtain the second image.

[0012] In one possible implementation, based on the probability information corresponding to each pixel in the first image, the classification information and confidence information corresponding to each pixel in the first image are determined, including: for each pixel in the first image, if the probability information corresponding to the pixel is greater than a first preset value, the classification information corresponding to the pixel is determined to be a person pixel; if the probability information corresponding to the pixel is less than or equal to the first preset value, the classification information corresponding to the pixel is determined to be a background pixel; and the confidence information corresponding to the pixel is determined based on the classification information and probability information corresponding to the pixel.

[0013] In one possible implementation, determining the confidence information corresponding to a pixel based on the classification information and probability information corresponding to the pixel includes: if the classification information corresponding to the pixel is a person pixel, then determining a first difference between the probability information corresponding to the pixel and a first preset value, and determining the ratio of the first difference to the first preset value as the confidence information corresponding to the pixel; if the classification information corresponding to the pixel is a background pixel, then determining a second difference between the first preset value and the probability information corresponding to the pixel, and determining the ratio of the second difference to the first preset value as the confidence information corresponding to the pixel.

[0014] In one possible implementation, the method further includes: acquiring historical confidence information, which represents the confidence information of each pixel in the previous frame image; acquiring scene switching information, and determining the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information; for each pixel, performing temporal smoothing processing on the confidence information based on the historical confidence information, smoothing weight information, confidence information, and weight information corresponding to the pixel, and determining the temporally smoothed confidence information as the confidence information corresponding to the pixel.

[0015] In one possible implementation, based on scene switching information, determining the smoothing weight information corresponding to historical confidence information and the weight information corresponding to confidence information includes: if the scene switching information indicates a scene switch between the previous frame and the current frame, then determining the smoothing weight information corresponding to historical confidence information as 0 and the weight information corresponding to confidence information as 1; if the scene switching information indicates no scene switch between the previous frame and the current frame, then determining whether the confidence information is greater than a preset threshold; if it is greater than the preset threshold, then determining the smoothing weight information corresponding to historical confidence information as 0.2 and the weight information corresponding to confidence information as 0.8; if it is less than or equal to the preset threshold, then determining the smoothing weight information corresponding to historical confidence information as 0.5 and the weight information corresponding to confidence information as 0.5.

[0016] In one possible implementation, the image blending weights corresponding to each pixel are determined based on the confidence information corresponding to each pixel, including: for each pixel, obtaining the image smoothing window corresponding to the pixel; and performing a weighted summation of the confidence information corresponding to multiple target pixels in the image smoothing window to obtain the image blending weights corresponding to the pixels.

[0017] Secondly, embodiments of this application provide an image processing apparatus, comprising:

[0018] The acquisition module is used to acquire the probability information corresponding to each pixel in the first image. The probability information is used to represent the probability that a pixel belongs to a person pixel.

[0019] The first determining module is used to determine the classification information and confidence information corresponding to each pixel in the first image based on the probability information corresponding to each pixel in the first image.

[0020] The segmentation module is used to divide the first image into a first person layer image and a first background layer image based on the classification information corresponding to each pixel in the first image;

[0021] The image quality enhancement module is used to call different parameters to enhance the image quality of the first person layer image and the first background layer image respectively, so as to obtain the second person layer image and the second background layer image.

[0022] The second determining module is used to determine the image mixing weights corresponding to each pixel based on the confidence information corresponding to each pixel.

[0023] The blending module is used to blend the second person layer image and the second background layer image based on the image blending weights corresponding to each pixel to obtain the second image.

[0024] In one possible implementation, the first determining module determines the classification information and confidence information corresponding to each pixel in the first image based on the probability information corresponding to each pixel in the first image, including: for each pixel in the first image, if the probability information corresponding to the pixel is greater than a first preset value, then the classification information corresponding to the pixel is determined to be a person pixel; if the probability information corresponding to the pixel is less than or equal to the first preset value, then the classification information corresponding to the pixel is determined to be a background pixel; and the confidence information corresponding to the pixel is determined based on the classification information and probability information corresponding to the pixel.

[0025] In one possible implementation, the first determining module determines the confidence information corresponding to a pixel based on the classification information and probability information corresponding to the pixel, including: if the classification information corresponding to the pixel is a person pixel, then determining a first difference between the probability information corresponding to the pixel and a first preset value, and determining the ratio of the first difference to the first preset value as the confidence information corresponding to the pixel; if the classification information corresponding to the pixel is a background pixel, then determining a second difference between the first preset value and the probability information corresponding to the pixel, and determining the ratio of the second difference to the first preset value as the confidence information corresponding to the pixel.

[0026] In one possible implementation, the device further includes: a temporal smoothing processing module; the temporal smoothing processing module is used to acquire historical confidence information, which represents the confidence information of each pixel in the previous frame image; acquire scene switching information, and determine the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information; for each pixel, perform temporal smoothing processing on the confidence information based on the historical confidence information, smoothing weight information, confidence information, and weight information corresponding to the pixel, and determine the temporally smoothed confidence information as the confidence information corresponding to the pixel.

[0027] In one possible implementation, the temporal smoothing processing module determines the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information. This includes: if the scene switching information indicates a scene switch between the previous frame and the current frame, then the smoothing weight information corresponding to the historical confidence information is determined to be 0, and the weight information corresponding to the confidence information is determined to be 1; if the scene switching information indicates no scene switch between the previous frame and the current frame, then it is determined whether the confidence information is greater than a preset threshold. If it is greater than the preset threshold, then the smoothing weight information corresponding to the historical confidence information is determined to be 0.2, and the weight information corresponding to the confidence information is determined to be 0.8; if it is less than or equal to the preset threshold, then the smoothing weight information corresponding to the historical confidence information is determined to be 0.5, and the weight information corresponding to the confidence information is determined to be 0.5.

[0028] In one possible implementation, the second determining module determines the image mixing weight corresponding to each pixel based on the confidence information corresponding to each pixel, including: for each pixel, obtaining the image smoothing window corresponding to the pixel; and performing a weighted summation of the confidence information corresponding to multiple target pixels in the image smoothing window to obtain the image mixing weight corresponding to the pixel.

[0029] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0030] The memory stores the instructions that the computer executes;

[0031] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0032] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0033] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0034] Sixthly, embodiments of this application provide a chip including at least one processor for executing program instructions to perform the above-described image processing method.

[0035] The image processing method, apparatus, device, storage medium, and program product provided in this application include: acquiring probability information corresponding to each pixel in a first image, wherein the probability information represents the probability that a pixel belongs to a person pixel; determining classification information and confidence information corresponding to each pixel in the first image based on the probability information corresponding to each pixel in the first image; dividing the first image into a first person layer image and a first background layer image based on the classification information corresponding to each pixel in the first image; performing image quality enhancement on the first person layer image and the first background layer image respectively by calling different parameters to obtain a second person layer image and a second background layer image; determining image mixing weights corresponding to each pixel based on the confidence information corresponding to each pixel; and performing mixing processing on the second person layer image and the second background layer image based on the image mixing weights corresponding to each pixel to obtain a second image. In this embodiment, the first image is first divided into a person layer image and a background layer image based on the probability information corresponding to each pixel. Then, different parameters are applied to enhance the image quality of the person layer image and the background layer image respectively. Finally, the enhanced person layer image and the background layer image are blended using image mixing weights to obtain the overall image after image quality enhancement. Because different parameters are applied to enhance the image quality of the person layer image and the background layer image respectively, the image quality enhancement effect of different content in the image is increased, thereby improving the display effect of the overall image after image quality enhancement. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0037] Figure 1 This is a flowchart of an image processing method provided in an embodiment of this application;

[0038] Figure 2 This is a schematic diagram of an image processing method provided in an embodiment of this application;

[0039] Figure 3 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0041] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0043] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0044] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. For example, "first image" and "second image" are used only to distinguish different images and do not limit their order. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that terms such as "first" and "second" do not necessarily imply that they are different.

[0045] In this embodiment of the application, "multiple" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0046] With the rapid development of high-resolution display devices and video processing technologies, users have increasingly higher demands for image quality. Currently, image quality can be improved by adjusting pixel display parameters such as contrast, saturation, and sharpness.

[0047] However, traditional image enhancement techniques often use global parameter adjustments (such as uniform contrast, saturation, or sharpening parameters), which makes it difficult to take into account the different needs of different areas in the image, resulting in poor image display.

[0048] Therefore, how to enhance the image quality of different regions in an image, and thus improve the overall display effect of the enhanced image, is a technical problem that urgently needs to be solved.

[0049] To address the aforementioned technical problems, this application provides an image processing method: obtaining probability information corresponding to each pixel in a first image, whereby the probability information represents the probability that a pixel belongs to a person pixel; determining classification information and confidence information corresponding to each pixel in the first image based on the probability information; dividing the first image into a first person layer image and a first background layer image based on the classification information; enhancing the image quality of the first person layer image and the first background layer image by calling different parameters to obtain a second person layer image and a second background layer image; determining the image mixing weight corresponding to each pixel based on the confidence information; and performing a mixing process on the second person layer image and the second background layer image based on the image mixing weight corresponding to each pixel to obtain a second image.

[0050] In the above technical solution, the first image is first divided into a person layer image and a background layer image based on the probability information corresponding to each pixel. Then, different parameters are applied to enhance the image quality of the person layer image and the background layer image respectively. Finally, the enhanced person layer image and the background layer image are blended using image blending weights to obtain the overall image after image quality enhancement. Because different parameters are applied to enhance the image quality of the person layer image and the background layer image respectively, the image quality enhancement effect of different content in the image is increased, thereby improving the display effect of the overall image after image quality enhancement. Furthermore, by using the probability information corresponding to each pixel in the first image, the classification information, confidence information, and image blending weights corresponding to each pixel in the first image are determined, realizing image segmentation and image blending at the pixel level, thus helping to improve the accuracy of image segmentation and image blending.

[0051] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0052] The technical solutions provided in this application embodiment can rely on the existing computing power in electronic devices (such as chips and chip modules), without the need for additional hardware sensors, and meet the strict requirements of electronic devices for cost and power consumption.

[0053] In this application embodiment, the device used to implement the function of the electronic device can be an electronic device; or it can be a device that can support the electronic device to implement the function, such as a chip system, which can be installed in the electronic device.

[0054] like Figure 1 As shown, Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. In some embodiments, the image processing method includes:

[0055] S101. Obtain the probability information corresponding to each pixel in the first image. The probability information is used to represent the probability that a pixel belongs to a person pixel.

[0056] The first image can be a picture or a video frame. The sharpness of the first image is not specifically limited in this application. Optionally, such as... Figure 2 As shown, the resolution of the first image can be a high-definition image such as 2K (2048×1080) or 4K (3840×2160).

[0057] Optionally, to improve image processing efficiency and reduce image storage space, the first image can be compressed to a preset size. The preset size represents the target resolution of the first image after scaling.

[0058] For example, such as Figure 2 As shown, the first image is compressed to a 768×768 image and saved to the storage module. This storage module can be DDR (Double Data Rate Synchronous Dynamic Random Access Memory).

[0059] In some embodiments, such as Figure 2 As shown, a person segmentation model can be loaded into an NPU (Neural Processing Unit), which reads a 768×768 image from the storage module as input and processes it to generate a 768×768 probability map. This probability map includes probability information corresponding to each pixel. Optionally, the probability information can range from 0 to 1.

[0060] S102. Based on the probability information corresponding to each pixel in the first image, determine the classification information and confidence information corresponding to each pixel in the first image.

[0061] In some embodiments, this step may include: for each pixel in the first image, if the probability information corresponding to the pixel is greater than a first preset value, then the classification information corresponding to the pixel is determined to be a person pixel; if the probability information corresponding to the pixel is less than or equal to the first preset value, then the classification information corresponding to the pixel is determined to be a background pixel; and the confidence information corresponding to the pixel is determined based on the classification information and probability information corresponding to the pixel.

[0062] The value of the first preset value is not specifically limited. For example, the first preset value is 0.5. For each pixel in the probability map, if the probability is greater than 0.5, the classification information is determined to be foreground (i.e., a person pixel). If the probability is less than or equal to 0.5, the classification information is determined to be background (i.e., a background pixel).

[0063] For example, based on the 768×768 probability map corresponding to the first image, a 768×768 classification map and a 768×768 confidence map corresponding to the first image can be determined. The classification map includes classification information for each pixel. The confidence map includes confidence information for each pixel.

[0064] Optionally, the confidence information corresponding to a pixel is determined based on the classification information and probability information corresponding to the pixel, including: if the classification information corresponding to the pixel is a person pixel, then the first difference between the probability information corresponding to the pixel and a first preset value is determined, and the ratio of the first difference to the first preset value is determined as the confidence information corresponding to the pixel; if the classification information corresponding to the pixel is a background pixel, then the second difference between the first preset value and the probability information corresponding to the pixel is determined, and the ratio of the second difference to the first preset value is determined as the confidence information corresponding to the pixel.

[0065] For example, the first preset value is 0.5. For a person pixel, the formula for calculating the confidence information corresponding to the pixel is: (probability value - 0.5) / 0.5; for a background pixel, the formula for calculating the confidence information corresponding to the pixel is: (0.5 - probability value) / 0.5.

[0066] In some embodiments, confidence information can also be temporally smoothed to avoid sudden changes in the display effect of adjacent video frames, thereby eliminating screen flicker in the video.

[0067] Optionally, the method for performing time-series smoothing on confidence information may include the following steps (1) to (3):

[0068] (1) Obtain historical confidence information, which is used to represent the confidence information of each pixel in the previous frame image.

[0069] For example, such as Figure 2 As shown, historical confidence information can be obtained, which can be the confidence map corresponding to the previous frame image.

[0070] (2) Obtain scene switching information, and determine the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information.

[0071] In some embodiments, this step may include: if the scene switching information indicates that there is a scene switch between the previous frame image and the current frame image, then determine that the smoothing weight information corresponding to the historical confidence information is 0 and the weight information corresponding to the confidence information is 1; if the scene switching information indicates that there is no scene switch between the previous frame image and the current frame image, then determine whether the confidence information is greater than a preset threshold. If it is greater than the preset threshold, then determine that the smoothing weight information corresponding to the historical confidence information is 0.2 and the weight information corresponding to the confidence information is 0.8. If it is less than or equal to the preset threshold, then determine that the smoothing weight information corresponding to the historical confidence information is 0.5 and the weight information corresponding to the confidence information is 0.5.

[0072] The value of the preset threshold is not specifically limited. For example, the preset threshold is 0.5. If there is a scene change, the smoothing weight information W1 corresponding to the historical confidence information is set to 0, and the weight information W2 corresponding to the confidence information is set to 1. If there is no scene change, if the confidence information of the pixel is greater than 0.5, W1 is set to 0.2 and W2 is set to 0.8; if the confidence information of the pixel is less than or equal to 0.5, W1 is set to 0.5 and W2 is set to 0.5.

[0073] In this embodiment, the smoothing weight information is dynamically adjusted based on scene switching information and confidence information. In static scenes, a high smoothing weight is used to reduce flickering, while in dynamic scenes, the smoothing weight is reduced to preserve motion details, thus improving the accuracy of temporal smoothing processing.

[0074] (3) For each pixel, based on the historical confidence information, smoothing weight information, confidence information and weight information corresponding to the pixel, perform time-series smoothing on the confidence information, and determine the confidence information after time-series smoothing as the confidence information corresponding to the pixel.

[0075] Optionally, for each pixel, based on the historical confidence information, smoothing weight information, and confidence and weight information corresponding to the pixel, the confidence information is subjected to time-series smoothing processing using the following formula 1 to obtain the time-series smoothed confidence information.

[0076] Formula 1: Dout = (D1W1 + D2W2) / W1 + W2;

[0077] Where D1 represents the historical confidence information corresponding to the pixel, and W1 represents the smoothing weight information corresponding to the historical confidence information; D1 represents the confidence information corresponding to the pixel, and W2 represents the weight information corresponding to the confidence information.

[0078] For example, after performing time-series smoothing on the 768×768 confidence graph, a time-series smoothed 768×768 confidence graph is obtained.

[0079] S103. Based on the classification information corresponding to each pixel in the first image, the first image is divided into a first person layer image and a first background layer image.

[0080] Optionally, the first person layer image includes multiple person pixels, and the first background layer image includes multiple background pixels.

[0081] For example, the classification information corresponding to each pixel in the first image can be a 768×768 classification image corresponding to the first image. At this time, the 768×768 classification image can be enlarged to a 4K (3840×2160) classification image, and then the 4K (3840×2160) first image can be divided into a first person layer image and a first background layer image.

[0082] S104. Use different parameters to enhance the image quality of the first person layer image and the first background layer image respectively, to obtain the second person layer image and the second background layer image.

[0083] Optionally, the parameters corresponding to the first person layer image include one or more of a first contrast adjustment parameter, a first saturation adjustment parameter, and a first sharpness adjustment parameter. The parameters corresponding to the first background layer image include one or more of a second contrast adjustment parameter, a second saturation adjustment parameter, and a second sharpness adjustment parameter.

[0084] S105. Based on the confidence information corresponding to each pixel, determine the image mixing weight corresponding to each pixel.

[0085] In some embodiments, determining the image blending weights corresponding to each pixel based on the confidence information corresponding to each pixel includes: obtaining an image smoothing window corresponding to each pixel; and performing a weighted summation of the confidence information corresponding to multiple target pixels in the image smoothing window to obtain the image blending weights corresponding to the pixels.

[0086] In this process, the confidence information corresponding to multiple target pixels in the image smoothing window can be weighted and summed using mean filtering or Gaussian filtering to obtain the image mixing weight corresponding to each pixel.

[0087] For example, the image smoothing window can be a 3×3 smoothing window. For each pixel, the average confidence of all pixels within the 3×3 smoothing window surrounding that pixel can be taken as the image mixing weight for that pixel.

[0088] S106. Based on the image mixing weights corresponding to each pixel, the second person layer image and the second background layer image are mixed to obtain the second image.

[0089] Optionally, this step may include: based on the image mixing weights corresponding to each pixel, mixing the second person layer image and the second background layer image using the following formula two to obtain the second image;

[0090] Formula 2: Mixed output = M × second person layer image + (1 - M) × second background layer image; where M represents the image mixing weight and the mixed output represents the second image.

[0091] This application provides an image processing method: obtaining probability information corresponding to each pixel in a first image, where the probability information represents the probability that a pixel belongs to a person pixel; determining classification information and confidence information corresponding to each pixel in the first image based on the probability information; dividing the first image into a first person layer image and a first background layer image based on the classification information; enhancing the image quality of the first person layer image and the first background layer image by calling different parameters to obtain a second person layer image and a second background layer image; determining the image mixing weight corresponding to each pixel based on the confidence information; and mixing the second person layer image and the second background layer image based on the image mixing weight to obtain a second image. In this application embodiment, the first image is first divided into a person layer image and a background layer image using the probability information corresponding to each pixel; then, different parameters are used to enhance the image quality of the person layer image and the background layer image respectively; finally, the image mixing weight is used to mix the enhanced person layer image and the background layer image to obtain the overall image after image quality enhancement. By applying different parameters to enhance the image quality of the person and background layers respectively, the image quality enhancement effect of different content in the image is increased, thereby improving the overall display effect of the enhanced image.

[0092] Figure 3 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. (Refer to...) Figure 3 The processing apparatus includes:

[0093] The acquisition module 301 is used to acquire the probability information corresponding to each pixel in the first image. The probability information is used to represent the probability that a pixel belongs to a person pixel.

[0094] The first determining module 302 is used to determine the classification information and confidence information corresponding to each pixel in the first image based on the probability information corresponding to each pixel in the first image.

[0095] The segmentation module 303 is used to divide the first image into a first person layer image and a first background layer image based on the classification information corresponding to each pixel in the first image.

[0096] The image quality enhancement module 304 is used to call different parameters to enhance the image quality of the first person layer image and the first background layer image respectively, so as to obtain the second person layer image and the second background layer image.

[0097] The second determining module 305 is used to determine the image mixing weights corresponding to each pixel based on the confidence information corresponding to each pixel.

[0098] The mixing module 306 is used to perform mixing processing on the second person layer image and the second background layer image based on the image mixing weights corresponding to each pixel to obtain the second image.

[0099] In one possible implementation, the first determining module 302 determines the classification information and confidence information corresponding to each pixel in the first image based on the probability information corresponding to each pixel in the first image, including: for each pixel in the first image, if the probability information corresponding to the pixel is greater than a first preset value, then the classification information corresponding to the pixel is determined to be a person pixel; if the probability information corresponding to the pixel is less than or equal to the first preset value, then the classification information corresponding to the pixel is determined to be a background pixel; and the confidence information corresponding to the pixel is determined based on the classification information and probability information corresponding to the pixel.

[0100] In one possible implementation, the first determining module 302 determines the confidence information corresponding to a pixel based on the classification information and probability information corresponding to the pixel, including: if the classification information corresponding to the pixel is a person pixel, then determining a first difference between the probability information corresponding to the pixel and a first preset value, and determining the ratio of the first difference to the first preset value as the confidence information corresponding to the pixel; if the classification information corresponding to the pixel is a background pixel, then determining a second difference between the first preset value and the probability information corresponding to the pixel, and determining the ratio of the second difference to the first preset value as the confidence information corresponding to the pixel.

[0101] In one possible implementation, the device further includes: a temporal smoothing processing module; the temporal smoothing processing module is used to acquire historical confidence information, which represents the confidence information of each pixel in the previous frame image; acquire scene switching information, and determine the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information; for each pixel, perform temporal smoothing processing on the confidence information based on the historical confidence information, smoothing weight information, confidence information, and weight information corresponding to the pixel, and determine the temporally smoothed confidence information as the confidence information corresponding to the pixel.

[0102] In one possible implementation, the temporal smoothing processing module determines the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information. This includes: if the scene switching information indicates a scene switch between the previous frame and the current frame, then the smoothing weight information corresponding to the historical confidence information is determined to be 0, and the weight information corresponding to the confidence information is determined to be 1; if the scene switching information indicates no scene switch between the previous frame and the current frame, then it is determined whether the confidence information is greater than a preset threshold. If it is greater than the preset threshold, then the smoothing weight information corresponding to the historical confidence information is determined to be 0.2, and the weight information corresponding to the confidence information is determined to be 0.8; if it is less than or equal to the preset threshold, then the smoothing weight information corresponding to the historical confidence information is determined to be 0.5, and the weight information corresponding to the confidence information is determined to be 0.5.

[0103] In one possible implementation, the second determining module 305 determines the image mixing weight corresponding to each pixel based on the confidence information corresponding to each pixel, including: for each pixel, obtaining the image smoothing window corresponding to the pixel; and performing a weighted summation of the confidence information corresponding to multiple target pixels in the image smoothing window to obtain the image mixing weight corresponding to the pixel.

[0104] The image processing apparatus provided in this embodiment can execute the image processing method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0105] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the electronic device 40 further includes a communication interface 403. The processor 401, memory 402, and communication interface 403 are connected via a bus.

[0106] In the specific implementation process, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to execute the above-described image processing method.

[0107] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0108] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0109] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0110] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0111] This application also provides a chip including at least one processor for executing program instructions to perform the above-described image processing method.

[0112] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image processing method.

[0113] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described image processing method.

[0114] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0115] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0116] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

Claims

1. An image processing method, characterized in that, The method includes: Obtain probability information corresponding to each pixel in the first image, wherein the probability information is used to represent the probability that the pixel belongs to a person pixel; Based on the probability information corresponding to each pixel in the first image, the classification information and confidence information corresponding to each pixel in the first image are determined. Based on the classification information corresponding to each pixel in the first image, the first image is divided into a first person layer image and a first background layer image. By applying different parameters to enhance the image quality of the first person layer image and the first background layer image respectively, the second person layer image and the second background layer image are obtained. Based on the confidence information corresponding to each pixel, the image mixing weight corresponding to each pixel is determined; Based on the image mixing weights corresponding to each pixel, the second person layer image and the second background layer image are mixed to obtain the second image.

2. The method according to claim 1, characterized in that, The step of determining the classification information and confidence information corresponding to each pixel in the first image based on the probability information of each pixel in the first image includes: For each pixel in the first image, if the probability information corresponding to the pixel is greater than a first preset value, then the classification information corresponding to the pixel is determined to be a person pixel; if the probability information corresponding to the pixel is less than or equal to the first preset value, then the classification information corresponding to the pixel is determined to be a background pixel. Based on the classification information and probability information corresponding to the pixel, the confidence information corresponding to the pixel is determined.

3. The method according to claim 2, characterized in that, The step of determining the confidence information corresponding to the pixel based on the classification information and probability information corresponding to the pixel includes: If the classification information corresponding to the pixel is a person pixel, then the first difference between the probability information corresponding to the pixel and the first preset value is determined, and the ratio of the first difference to the first preset value is determined as the confidence information corresponding to the pixel. If the classification information corresponding to the pixel is a background pixel, then the second difference between the first preset value and the probability information corresponding to the pixel is determined, and the ratio of the second difference to the first preset value is determined as the confidence information corresponding to the pixel.

4. The method according to claim 3, characterized in that, The method further includes: Obtain historical confidence information, which is used to represent the confidence information of each pixel in the previous frame image; Obtain scene switching information, and determine the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information; For each pixel, based on the historical confidence information corresponding to the pixel, the smoothing weight information, the confidence information corresponding to the pixel, and the weight information, the confidence information is subjected to time-series smoothing processing, and the confidence information after time-series smoothing processing is determined as the confidence information corresponding to the pixel.

5. The method according to claim 4, characterized in that, The step of determining the smoothing weight information corresponding to the historical confidence information and the weight information corresponding to the confidence information based on the scene switching information includes: If the scene switching information is used to indicate that there is a scene switch between the previous frame image and the current frame image, then the smoothing weight information corresponding to the historical confidence information is determined to be 0, and the weight information corresponding to the confidence information is determined to be 1. If the scene switching information indicates that there is no scene switching between the previous frame and the current frame, then it is determined whether the confidence information is greater than a preset threshold. If it is greater than the preset threshold, then the smoothing weight information corresponding to the historical confidence information is determined to be 0.2, and the weight information corresponding to the confidence information is determined to be 0.

8. If it is less than or equal to the preset threshold, then the smoothing weight information corresponding to the historical confidence information is determined to be 0.5, and the weight information corresponding to the confidence information is determined to be 0.

5.

6. The method according to claim 1, characterized in that, The step of determining the image mixing weights corresponding to each pixel based on the confidence information corresponding to each pixel includes: For each pixel, obtain the image smoothing window corresponding to that pixel; The confidence information corresponding to multiple target pixels in the image smoothing window is weighted and summed to obtain the image mixing weight corresponding to the pixel.

7. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire probability information corresponding to each pixel in the first image, wherein the probability information is used to represent the probability that the pixel belongs to a person pixel. The first determining module is used to determine the classification information and confidence information corresponding to each pixel in the first image based on the probability information corresponding to each pixel in the first image. The segmentation module is used to divide the first image into a first person layer image and a first background layer image based on the classification information corresponding to each pixel in the first image. The image quality enhancement module is used to call different parameters to enhance the image quality of the first person layer image and the first background layer image respectively, so as to obtain the second person layer image and the second background layer image. The second determining module is used to determine the image mixing weight corresponding to each pixel based on the confidence information corresponding to each pixel. The mixing module is used to perform mixing processing on the second person layer image and the second background layer image based on the image mixing weights corresponding to each pixel, so as to obtain the second image.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the image processing method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the image processing method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the image processing method as described in any one of claims 1-6.

Images