Image processing method and apparatus, storage medium and electronic device

Abstract

Disclosed in the present description are an image processing method and apparatus, a storage medium and an electronic device. The method comprises: determining a first portrait area image in an input image, and acquiring portrait coordinates of the first portrait area image; determining a target size of the first portrait area image, and, on the basis of an adapted resolution of the target size, adjusting the first portrait area image, so as to obtain a first adapted image; on the basis of a super-resolution scaling factor, performing super-resolution processing on the first adapted image, so as to obtain a first super-resolution image; performing fusion on the first portrait area image, the first super-resolution image and a mask corresponding to the portrait coordinates, so as to obtain a second portrait area image; and, on the basis of the portrait coordinates, fusing the second portrait area image into the input image, so as to obtain a portrait close-up image. By means of the present description, a clear portrait image obtained by performing super-resolution processing on a portrait is fused into an input image, so as to obtain an image that presents a clear portrait and provides comfortable viewing experience, thereby improving the quality of generated close-up images and the performance ratio of image generation.

Description

Image processing method, device, storage medium and electronic device Technical Field

[0001] This specification relates to the field of computer technology, and in particular to an image processing method, device, storage medium, and electronic device. Background Art

[0002] Nowadays, with the popularization and continuous development of video conferencing, various additional functions in video conferencing are constantly being enriched. In the process of video conferencing, close-up of portraits is a commonly used function, but the image processing effect when taking close-up of portraits is an issue that needs to be urgently addressed.

[0003] Summary of the Invention

[0004] The embodiments of this specification provide an image processing method, device, storage medium and electronic device, which can obtain a clear portrait image by performing targeted super-resolution processing on the portrait part, and fuse the clear portrait image into the input image to obtain an image with a clear portrait and a pleasant viewing experience. It can complete a clear close-up of the portrait with limited computing power, thereby improving the generation quality of the close-up image and the performance ratio of the image generation process.

[0005] In a first aspect, an embodiment of this specification provides an image processing method, the method comprising:

[0006] Determine a first portrait region image in the input image, and obtain portrait coordinates of the first portrait region image in the input image;

[0007] determining a target size of the first portrait area image, and adjusting the first portrait area image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image;

[0008] Performing super-resolution processing on the first adapted image based on a super-resolution ratio corresponding to the target size to obtain a first super-resolution image;

[0009] Fusing the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait region image;

[0010] The second portrait region image is fused into the input image based on the portrait coordinates, and the fused input image is determined as a portrait close-up image.

[0011] In a second aspect, an embodiment of this specification provides an image processing device, the device comprising:

[0012] A region image acquisition unit, configured to determine a first portrait region image in an input image and acquire portrait coordinates of the first portrait region image in the input image;

[0013] an adapted image acquisition unit, configured to determine a target size of the first portrait region image, and adjust the first portrait region image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait region image;

[0014] a super-resolution image acquisition unit, configured to perform super-resolution processing on the first adapted image based on a super-resolution ratio corresponding to the target size, so as to obtain a first super-resolution image;

[0015] a fusion unit, configured to fuse the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait region image;

[0016] The fusion unit is further configured to fuse the second portrait region image into the input image based on the portrait coordinates, and determine the fused input image as a portrait close-up image.

[0017] In a third aspect, an embodiment of this specification provides a computer storage medium, which stores a plurality of instructions suitable for being loaded by a processor and executing the steps of the above method.

[0018] In a fourth aspect, an embodiment of this specification provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0019] In an embodiment of the present specification, the first portrait area image in the input image is super-resolutioned according to the super-resolution ratio of the target size of the first portrait area image in the input image to obtain a first super-resolution image with a clear portrait after super-resolution, the first super-resolution image is fused with the mask to obtain a second portrait area image, the second portrait area image is fused into the input image to obtain a close-up portrait image, and then a clear portrait image is obtained by targeted super-resolution processing of the portrait part, the clear portrait image is fused into the input image to obtain a clear portrait image with a comfortable viewing experience, and a clear close-up of the portrait is completed under limited computing power, thereby improving the generation quality of the close-up image and the performance ratio in the image generation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0021] FIG1 is a system architecture diagram of an image processing method provided by an embodiment of this specification;

[0022] FIG2 is a flow chart of an image processing method provided in an embodiment of this specification;

[0023] FIG3 is a flow chart of an image processing method provided in an embodiment of this specification;

[0024] FIG4 is a schematic diagram of an example of portrait coordinates provided in an embodiment of this specification;

[0025] FIG5 is a schematic diagram illustrating an example of determining an adaptive resolution according to an embodiment of this specification;

[0026] FIG6 is a schematic diagram illustrating an example of a close-up portrait image provided in an embodiment of this specification;

[0027] FIG7 is a schematic diagram illustrating an example of displaying a close-up image of a portrait provided by an embodiment of this specification;

[0028] FIG8 is a schematic structural diagram of an image processing device provided in an embodiment of this specification;

[0029] FIG9 is a schematic structural diagram of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0030] To make the features and advantages of this specification more obvious and easy to understand, the technical solutions in this specification are clearly and completely described below in conjunction with the drawings in this specification. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without making any creative efforts are within the scope of protection of this specification.

[0031] In the prior art, when taking a close-up of a portrait, the method adopted is to directly magnify the portrait portion in the image, resulting in a large loss of details in the image data and affecting the generated image quality.

[0032] The existing methods for taking close-up photos of portraits have problems of insufficient effect or high cost. For example, an image is cut out and the bilinear interpolation method is used to enlarge the image obtained by the cutout. However, since the image enlarged by the bilinear interpolation method has a large amount of information missing, the enlarged image has serious detail loss and the image quality is extremely poor. Alternatively, a method of combining a high-definition optical zoom lens and a mechanical pan-tilt head is used. After the mechanical pan-tilt head is controlled to aim at the close-up object, the optical zoom lens is used to perform optical magnification to obtain a clean close-up picture. However, the optical zoom lens is expensive and not conducive to product integration. Another method is to use a hybrid optical zoom combined with multiple cameras. Information is collected by a wide-angle camera, and then the close-up object picture is obtained by a telephoto lens. However, since the focal length of the camera is fixed, the magnification function still needs to be used to process the image during the transition stage between different focal lengths, resulting in loss of image details. The digital image processing required for multiple cameras to obtain images requires high acquisition capabilities, and the cost of multiple cameras is also high.

[0033] Based on the above technical problems, an embodiment of this specification provides an image processing method. Using the embodiment of this specification, the first portrait area image in the input image is super-resolutioned according to the super-resolution ratio of the target size of the first portrait area image in the input image to obtain a first super-resolution image with a clear portrait after super-resolution. The first super-resolution image is fused with the mask to obtain a second portrait area image, and the second portrait area image is fused into the input image to obtain a close-up portrait image. Then, a clear portrait image is obtained by targeted super-resolution of the portrait part, and the clear portrait image is fused into the input image to obtain a clear and comfortable-to-viewing portrait image. The clear close-up of the portrait is completed under limited computing power, thereby improving the generation quality of the close-up image and the performance ratio in the image generation process.

[0034] Please refer to Figure 1, which provides a system structure diagram of image processing according to an embodiment of this specification. As shown in Figure 1, the image processing method provided in the embodiment of this specification can be applied to a terminal device to perform a close-up portrait process. The system structure provided in the embodiment of this specification mainly includes a terminal device 10 and a camera device 20. The terminal device 10 can be a device with an image processing function, such as an interactive tablet, a laptop computer, a smart phone, etc. The camera device 20 can be a device for image capture, or it can be a camera module in the terminal device 10, such as a camera or a video camera.

[0035] The terminal device 10 obtains an input image from the camera device 20, determines a first portrait area image in the input image, obtains the portrait coordinates of the first portrait area image in the input image, determines a target size of the first portrait area image, adjusts the first portrait area image based on the adaptation resolution corresponding to the target size, obtains a first adapted image corresponding to the first portrait area image, super-resolution is performed on the first adapted image based on the super-resolution ratio corresponding to the target size to obtain a first super-resolution image, fuses the first portrait area image, the first super-resolution image and the mask corresponding to the portrait coordinates to obtain a second portrait area image, and fuses the second portrait area image into the input image based on the portrait coordinates to obtain a portrait close-up image corresponding to the input image.

[0036] In an embodiment of the present specification, the first portrait area image in the input image is super-resolutioned according to the super-resolution ratio of the target size of the first portrait area image in the input image to obtain a first super-resolution image with a clear portrait after super-resolution, the first super-resolution image is fused with the mask to obtain a second portrait area image, the second portrait area image is fused to the input image to obtain a close-up portrait image, and then a clear portrait image is obtained by targeted super-resolution processing of the portrait part, the clear portrait image is fused to the input image to obtain an image with a clear portrait and a smooth appearance with the input image, thereby completing a clear close-up of the portrait with limited computing power, thereby improving the generation quality of the close-up image and the performance ratio in the image generation process.

[0037] Based on the system architecture shown in FIG1 , the image processing method provided in the embodiment of this specification will be described in detail below in conjunction with FIG2 .

[0038] Please refer to Figure 2, which is a flowchart of an image processing method according to an embodiment of this specification. As shown in Figure 2, the method may include the following steps S101 to S105.

[0039] S101, determining a first portrait region image in an input image, and obtaining portrait coordinates of the first portrait region image in the input image;

[0040] In one embodiment, face recognition is used to identify a target portrait in an original image, the original image is cropped based on a set size to obtain an input image corresponding to the original image, the image is positioned in the input image, a first portrait area image is determined, and the portrait coordinates of the first portrait area image in the input image are obtained.

[0041] The original image can be an image captured by a camera that includes a person's head, or an image obtained by cropping the white edges of an image captured by a camera to obtain a close-up of a person's portrait. The original image can include a single person's portrait or multiple people's portraits. The input image can be an image for image processing obtained by cropping the original image to a set size.

[0042] The set size can be the size used to crop the original image to obtain the input image. The set size can be a number determined based on the size of the portrait, such as the portrait's height, or a fixed size. For example, the height of the input image in the set size can be three times the portrait's size, and the width can be 5.3 times the portrait's size. If the set size is a fixed value, it can be 480, etc. The specific setting can be based on actual circumstances.

[0043] The portrait size can be the size used to represent the head area in the image. For example, the portrait size can be the height of the head, or the height of the head and neck. The specific size can be determined according to the definition of the image recognition algorithm or set according to actual conditions. The unit corresponding to the portrait size can be the number of pixels, which can also be set as needed.

[0044] It is understandable that when the original image includes multiple portraits, when the original image is cropped using a set size, multiple portraits are included in the set size range, so the resulting input image includes multiple portraits.

[0045] The first portrait area image may be an image including a complete portrait, or may be an image determined in the input image based on face recognition. It should be noted that, to facilitate image processing, the first portrait area image may be a rectangular or square image.

[0046] The side length of the first portrait area image can be a value corresponding to the portrait size. The height of the first portrait area image can be the portrait size, and the width can be the same length as the portrait size, or a custom length. It is understood that in order to avoid an incomplete portrait in the first portrait area due to a too small width, the set width needs to ensure that the first portrait area image includes at least one complete portrait.

[0047] It can be understood that since the input image is obtained by cropping the original image according to the set size, there are cases where the portrait in the input is only a partial portrait. When determining the first portrait area image in the input image, the portrait that is only a partial portrait is not regarded as the first portrait area image.

[0048] The portrait coordinates of the first portrait area image can be coordinate information used to mark the first portrait area image in a coordinate system established based on the input image. It is understood that to reduce the amount of computation, the coordinates of the first portrait area image can be the coordinates of the lower left corner and upper right corner of the first portrait area, and can be set specifically based on actual circumstances. It should be noted that the portrait coordinates can be the coordinates of the head, face, head and shoulders, etc. in the first portrait area image, and can be set specifically based on actual circumstances.

[0049] S102, determining a target size of the first portrait region image, and adjusting the first portrait region image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait region image;

[0050] In one embodiment, based on the portrait size of the portrait included in the first portrait area image, the target size of the first portrait area image is determined in the portrait size, and the resolution of the first portrait area image is adjusted based on the adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image.

[0051] The target size may be data used to describe the image size, such as the height and width, used for image processing of the first portrait area image. The portrait size may be the height corresponding to the first portrait area image. When the input image is a single portrait, the portrait size is determined as the target size; when the input image includes multiple portraits, the portrait sizes of the first portrait area images in the input image are compared, and the maximum height among the portrait sizes is determined as the target size.

[0052] The first adaptive resolution can be a resolution adjacent to the target size in the super-resolution resolution. It should be noted that the resolution of the image that can be super-resolution processed is the set super-resolution resolution, and the super-resolution resolution is an integer multiple of 10. Therefore, the resolution of the first portrait area image needs to be adjusted to the first adaptive resolution so that the first adaptive image that meets the super-resolution resolution is obtained for the first portrait area image based on the first adaptive resolution.

[0053] Furthermore, to minimize detail loss in the first portrait area during resolution adjustment, the super-resolution resolution closest to the target size is selected. For example, if the target size is 170 pixels, and the super-resolution resolutions adjacent to 170 are 180 and 90, then the super-resolution resolution "180," which is closer to the target size "170," is selected as the first adaptation resolution.

[0054] When the input image includes a single portrait, the resolution of the first portrait area image is adjusted based on the first adaptive resolution to obtain a first adapted image corresponding to the first portrait area image; when the input image includes multiple portraits, the resolution of each first portrait area image in the input image is adjusted based on the second adaptive resolution of the target size to obtain a first adapted image corresponding to each first portrait area image.

[0055] S103, performing super-resolution processing on the first adapted image based on the super-resolution ratio corresponding to the target size to obtain a first super-resolution image;

[0056] In one embodiment, based on the super-resolution ratio corresponding to the target size, the color coding channel of the first adapted image is super-resolved to obtain a first super-resolved image.

[0057] The super-resolution ratio is the ratio used to super-resolution the image corresponding to the target size. The super-resolution ratio can be two, four, or eight times. To reduce computing pressure under the same computing power, the larger the target size, the smaller the corresponding super-resolution ratio. For example, if the target size is 180, the super-resolution ratio is four times; if the target size is 360, the super-resolution ratio is two times, and so on.

[0058] A coding channel can be information used in a color coding method to represent the composition of an image. A coding channel includes at least one channel. It should be noted that an image's coding channels may include only two or more coding channels, depending on the specific circumstances. The first coding channel can be one of the coding channels used to compose the image, and the second coding channel can be another coding channel used to compose the image. For example, if the image's color coding method is YUV (Y represents brightness and UV represents chrominance), the image includes a first coding channel for brightness and a second coding channel for chrominance.

[0059] Super-resolution processing can be a processing method of performing super-resolution calculation on an image to improve the image resolution. Super-resolution (SR) is referred to as super-resolution for short.

[0060] It should be noted that since the computing power required for super-resolution processing is much greater than that required for enlargement processing, in order to perform a close-up of the portrait of the input image with limited computing power, the first coding channel of the first portrait area image in the input image can be super-resolution processed, and the other coding channels in the output image except the first coding channel of the first portrait area, as well as the other parts of the input image outside the processed first portrait area, can be enlarged, so as to achieve a clear close-up effect of the portrait with limited computing power.

[0061] When the input image is a single portrait, after determining the super-resolution ratio corresponding to the target size, the first coding channel in the first adapted image is super-resolved based on the super-resolution ratio to obtain an image with a resolution of the super-resolution ratio as the first channel data. The second coding channel in the first adapted image is then amplified based on the super-resolution ratio to obtain an image with a resolution of the super-resolution ratio as the second channel data. The first channel data and the second channel data are fused to obtain a first super-resolved image corresponding to the first adapted image.

[0062] The upscaling process can be a method of increasing the resolution of an image using methods such as bilinear interpolation or unidirectional interpolation. It should be noted that upscaling is different from super-resolution. Super-resolution is a method of magnifying an image using methods such as neural networks. It requires more computing power than upscaling, but it also provides better processing results and preserves more image details.

[0063] The first channel data may be an image obtained by super-resolving the first coding channel to a super-resolution ratio of the first adaptive image resolution, and the second channel data may be an image obtained by magnifying the second coding channel to a super-resolution ratio of the first adaptive image resolution.

[0064] The first super-resolution image may be an image with a resolution equal to a super-resolution ratio of the first adapted image.

[0065] When the input image is a plurality of portraits, after determining the super-resolution ratio corresponding to the target size, the first coding channel in each first adapted image in the input image is super-resolved based on the super-resolution ratio to obtain an image with a resolution of each super-resolution ratio as the first channel data. Then, the second coding channel in each first adapted image is amplified based on the super-resolution ratio to obtain an image with a resolution of the super-resolution ratio as the second channel data. The first channel data and the second channel data are fused to obtain a first super-resolved image corresponding to each first adapted image.

[0066] Furthermore, in order to ensure that the resolution of the generated image is consistent with the original resolution of the original image, it is necessary to adjust the resolution of the first super-resolution image obtained after the super-resolution processing so that the resolution of the first super-resolution image is consistent with the original resolution. When the resolution of the first super-resolution image is inconsistent with the original resolution, a feasible method may be to, if the resolution of the first super-resolution image is different from the original resolution of the original image corresponding to the input image, scale the first super-resolution image based on the original resolution to obtain a second super-resolution image having the same resolution as the original resolution, and use the second super-resolution image as the first super-resolution image corresponding to the first adapted image.

[0067] The scaling process may be to perform resolution enlargement or reduction processing on the first super-resolution image so that the resolution of the first super-resolution image is consistent with the original resolution.

[0068] The original resolution may be the resolution of the original image, and the second super-resolution image may be an image with the same resolution as the original resolution.

[0069] S104, fusing the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait region image;

[0070] In one embodiment, calculation is performed based on the first super-resolution image and the mask corresponding to the portrait coordinates to obtain a first image to be fused corresponding to the first super-resolution image, calculation is performed based on the first portrait area image and the mask to obtain a second image to be fused, and the first image to be fused and the second image to be fused are fused to obtain a second portrait area image.

[0071] The mask may be an image calculated based on the image size, and is used to smooth the difference in peripheral information between the first super-resolution image and the first portrait area image at the location of the input image, so as to improve the visibility of the first super-resolution image in the input image. It should be noted that since the mask is calculated based only on the image size, the corresponding mask can be calculated after the image size is obtained. If the input image is a single portrait, then since the target size of the first portrait area image corresponding to the input image is known, the portrait coordinates can be obtained in step S101 to calculate the corresponding mask; if the input image is multiple portraits, then based on the portrait size of each first portrait area image in the input image, the image size of each first portrait area image is determined, and then the mask corresponding to each first portrait area image is calculated according to the image size.

[0072] The image size is a value calculated based on the image coordinates. For example, if the image coordinates are (100, 100) and (270, 270), the image size can be a rectangle with a length and width of 170.

[0073] The first image to be fused can be an image used to smooth the first super-resolved image in the input image, and is calculated using the first super-resolved image and the mask. For example, if the first super-resolved image is h_sr and the mask is mask, then the first image to be fused is calculated based on "h_sr*mask".

[0074] The second image to be fused can be an image used to represent the difference between the first portrait region image and the first image to be fused, and is calculated using the first portrait region image and the mask. For example, if the first portrait region image is h_so and the mask is mask, then the second image to be fused is calculated based on "h_so*mask".

[0075] The fusion processing can be a processing method for fusing the first image to be fused and the second image to be fused to obtain a second portrait area image, and each pixel point in the first image to be fused and each pixel point in the second image to be fused are calculated separately to obtain the second portrait area image.

[0076] The second portrait area image may be an image used to fuse the first portrait area image in the input image, and a resolution of the second portrait area image is greater than a resolution of the first portrait area image.

[0077] S105, fusing the second portrait region image into the input image based on the portrait coordinates, and determining the fused input image as a portrait close-up image;

[0078] In one embodiment, based on the portrait coordinates corresponding to the first portrait area image, the area where the first portrait area image is located is determined in the input image, and the second portrait area image is fused with the first portrait area image so that the second portrait area image is fused into the input image, and the fused input image is determined as a portrait close-up image.

[0079] The portrait close-up image may be an image with less jagged edges and less detail loss in the input image, that is, an image obtained by fusing a first portrait region image in the input image with a second portrait region image with less jagged edges and less detail loss.

[0080] In an embodiment of the present specification, the first portrait area image in the input image is super-resolutioned according to the super-resolution ratio of the target size of the first portrait area image in the input image to obtain a first super-resolution image with a clear portrait after super-resolution, the first super-resolution image is fused with the mask to obtain a second portrait area image, the second portrait area image is fused into the input image to obtain a close-up portrait image, and then a clear portrait image is obtained by targeted super-resolution processing of the portrait part, the clear portrait image is fused into the input image to obtain a clear portrait image with a comfortable viewing experience, and a clear close-up of the portrait is completed under limited computing power, thereby improving the generation quality of the close-up image and the performance ratio in the image generation process.

[0081] Please refer to Figure 3, which is a flowchart of an image processing method according to an embodiment of this specification. As shown in Figure 3, the method is applied to an interactive tablet and may include the following steps S201 to S213.

[0082] S201, determining a first portrait region image in an input image, and obtaining portrait coordinates of the first portrait region image in the input image;

[0083] In one embodiment, face recognition is used to identify a target portrait in an original image, the original image is cropped based on a set size to obtain an input image corresponding to the original image, the image is positioned in the input image, a first portrait area image is determined, and the portrait coordinates of the first portrait area image in the input image are obtained.

[0084] The original image can be an image captured by a camera that includes a person's head, or an image obtained by cropping the white edges of an image captured by a camera to obtain a close-up of a person's portrait. The original image can include a single person's portrait or multiple people's portraits. The input image can be an image for image processing obtained by cropping the original image to a set size.

[0085] The set size can be the size used to crop the original image to obtain the input image. The set size can be a number determined based on the size of the portrait, such as the portrait's height, or a fixed size. For example, the height of the input image in the set size can be three times the portrait's size, and the width can be 5.3 times the portrait's size. If the set size is a fixed value, it can be 480, etc. The specific setting can be based on actual circumstances.

[0086] The portrait size can be the size used to represent the head area in the image. For example, the portrait size can be the height of the head, or the height of the head and neck. The specific size can be determined according to the definition of the image recognition algorithm or set according to actual conditions. The unit corresponding to the portrait size can be the number of pixels, which can also be set as needed.

[0087] It is understandable that when the original image includes multiple portraits, when the original image is cropped using a set size, multiple portraits are included in the set size range, so the resulting input image includes multiple portraits.

[0088] The first portrait area image may be an image including a complete portrait, or may be an image determined in the input image based on face recognition. It should be noted that, to facilitate image processing, the first portrait area image may be a rectangular or square image.

[0089] Among them, the side length of the first portrait area image can be a value corresponding to the portrait size. The height of the first portrait area image can be the portrait size, and the width can be the same length as the portrait size, or a customized length. It can be understood that in order to avoid the portrait in the first portrait area being incomplete due to the width being too small, the set width needs to ensure that the first portrait area image includes at least one complete portrait. For example, if the portrait size is 200 pixels, the height and width of the first portrait area image can both be 200 pixels; or if the portrait size is 200 pixels and the width of the head in the first portrait area image is 150 pixels, then the height of the first portrait area image can be 200 pixels and the width can be 180 pixels.

[0090] It can be understood that since the input image is obtained by cropping the original image according to the set size, there are cases where the portrait in the input is only a partial portrait. When determining the first portrait area image in the input image, the portrait that is only a partial portrait is not regarded as the first portrait area image.

[0091] The portrait coordinates of the first portrait area image can be the coordinate information used to mark the first portrait area image in the coordinate system established based on the input image. For example, as shown in Figure 4, the resolution of the input image in Figure 4 is 480, and the height and width of the first portrait area image are both 170 pixels, then the coordinates of the first portrait area image can be (100, 100) and (270, 270). It can be understood that in order to reduce the amount of calculation, the coordinates of the first portrait area image can be the coordinates of the lower left corner and the upper right corner of the first portrait area, which can be set according to actual conditions. It should be noted that the portrait coordinates can be the coordinates of the head, face, head and shoulders, etc. in the first portrait area image, which can be set according to actual conditions.

[0092] S202: When the input image is a single portrait, determine a target size of a first portrait region image, obtain a first adapted resolution adjacent to the target size, and adjust the resolution of the first portrait region image based on the first adapted resolution to obtain a first adapted image corresponding to the first portrait region image.

[0093] In one embodiment, based on the portrait size of the portrait included in the first portrait area image, the target size of the first portrait area image is determined in the portrait size, and the resolution of the first portrait area image is adjusted based on the adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image.

[0094] The target size may be data used to describe the image size, such as the height and width, used for image processing of the first portrait area image. The portrait size may be the height corresponding to the first portrait area image. When the input image is a single portrait, the portrait size may be determined as the target size of the first portrait area image.

[0095] The first adaptive resolution can be a resolution in the super-resolution processing resolution that is adjacent to the target size. It should be noted that the resolution of the image that can be super-resolution processed is the set super-resolution processing resolution. Therefore, the resolution of the first portrait area image needs to be adjusted to the first adaptive resolution so that the first adaptive image that meets the super-resolution processing resolution is obtained for the first portrait area image based on the first adaptive resolution.

[0096] Exemplarily, a feasible method for adjusting the resolution of the first portrait area image to the first adapted resolution may be to obtain an adapted ratio between the resolution of the first portrait area image and the first adapted resolution, and scale the first portrait area image using the adapted ratio to obtain the first adapted image. For example, if the first adapted resolution is 360 and the resolution of the first portrait area image is 240, then the adapted ratio between the resolution of the first portrait area image and the first adapted resolution is 1.5 times. The resolution of the first portrait area image is scaled by 1.5 times using a method such as bilinear interpolation to obtain the first adapted image. It should be noted that the image resolution adjustment method may be a method pre-stored in the image processing chip of the terminal device. The first adapted resolution may be a pre-set fixed resolution, such as 90, 180, or 360, or may be a number in tens. It will be appreciated that a resolution in tens or a fixed number can reduce the amount of data processing, thereby increasing the image processing rate while requiring fewer computing resources.

[0097] Furthermore, in the process of determining the first adaptive resolution, if there is an intersection between the target sizes included in the adaptive resolutions, since the input image is obtained by cropping the original image in real time, there is a situation where the portrait size of the current frame is significantly different from the portrait size of the previous frame. In this case, the adaptive resolution corresponding to the range of the portrait size of the previous frame can be used as the adaptive resolution corresponding to the target size.

[0098] For example, as shown in Figure 5, the target size for the adapted resolution of 360 in Figure 5 is 180-360, and the target size for the adapted resolution of 180 is 90-210. It can be obtained that there are two adapted resolutions corresponding to the target size between 180-210. If the target size of the first portrait area image of the current frame is 200, and the target size of the previous frame of the current frame is 250, then the adapted resolution corresponding to the first portrait area image of the current frame is 360.

[0099] Furthermore, to minimize detail loss in the first portrait area during resolution adjustment, a super-resolution resolution adjacent to the target size is selected. For example, if the target size is 170 pixels, and the super-resolution resolutions adjacent to 170 are 180 and 90, then the super-resolution resolution "180," which is closer to the target size "170," is selected as the first adaptation resolution.

[0100] When the input image includes a single portrait, the resolution of the first portrait region image is adjusted based on the first adaptation resolution to obtain a first adaptation image corresponding to the first portrait region image.

[0101] S203, performing super-resolution processing on the first coding channel of the first adapted image based on the super-resolution magnification corresponding to the target size to obtain first channel data;

[0102] In one embodiment, after determining the super-resolution ratio corresponding to the target size, super-resolution processing is performed on the first coded channel in the first adapted image based on the super-resolution ratio to obtain first channel data.

[0103] The super-resolution ratio is the ratio used to super-resolution the image corresponding to the target size. The super-resolution ratio can be two, four, or eight times. To reduce computing pressure under the same computing power, the larger the target size, the smaller the corresponding super-resolution ratio. For example, if the target size is 180, the super-resolution ratio is four times; if the target size is 360, the super-resolution ratio is two times, and so on.

[0104] A coding channel can be information used in a color coding method to represent the composition of an image. A coding channel includes at least one channel. It should be noted that an image's coding channels may include only two or more coding channels, depending on the specific circumstances. The first coding channel can be one of the coding channels used to compose the image, and the second coding channel can be another coding channel used to compose the image. For example, if the image's color coding method is YUV (Y represents brightness and UV represents chrominance), the image includes a first coding channel for brightness and a second coding channel for chrominance.

[0105] The first channel data may be an image that is super-resolved by the first coding channel into a super-resolved ratio of the first adaptive image resolution.

[0106] S204, amplifying the second coding channel of the first adapted image based on the super-resolution ratio to obtain second channel data;

[0107] In one embodiment, the second coding channel in the first adapted image is amplified based on the super-resolution ratio to obtain a super-resolution ratio image with a resolution equal to the adapted resolution as the second channel data.

[0108] The enlargement process may be a process of enlarging the resolution of the image by using a bilinear interpolation or a unidirectional interpolation method.

[0109] It should be noted that magnification processing is different from super-resolution processing. Super-resolution processing is to magnify the image by multiples based on neural networks and other methods. It requires more computing power than magnification processing, and the processing effect is better, retaining more image details.

[0110] The second channel data may be an image of a super-resolution ratio magnified by the second coding channel to obtain the first adaptive image resolution.

[0111] S205, performing image fusion on the first channel data and the second channel data to obtain a first super-resolved image corresponding to the first adapted image;

[0112] In one embodiment, after obtaining the first channel data and the second channel data corresponding to the first adapted image, the first channel data and the second channel data are fused to obtain a first super-resolved image corresponding to the first adapted image.

[0113] Image fusion may be performing corresponding calculations on each pixel point in the first channel data and each pixel point in the second channel data to obtain a first super-resolution image.

[0114] The first super-resolution image may be an image with a resolution equal to a super-resolution ratio of the first adapted image.

[0115] Furthermore, in order to ensure that the resolution of the generated image is consistent with the original resolution corresponding to the original image, it is necessary to adjust the resolution of the first super-resolution image obtained after the super-resolution processing so that the resolution of the first super-resolution image is consistent with the original resolution. When the resolution of the first super-resolution image is inconsistent with the original resolution, a feasible method may be to, if the resolution of the first super-resolution image is different from the original resolution of the original image corresponding to the input image, scale the first super-resolution image based on the original resolution to obtain a second super-resolution image having the same resolution as the original resolution, and use the second super-resolution image as the first super-resolution image corresponding to the first adapted image.

[0116] The scaling process may be to perform resolution enlargement or reduction processing on the first super-resolution image so that the resolution of the first super-resolution image is consistent with the original resolution.

[0117] The original resolution may be the resolution of the original image, and the second super-resolution image may be an image with the same resolution as the original resolution.

[0118] S206: When the input image contains multiple portraits, obtain the portrait size of each portrait in the input image, determine the maximum height among the portrait sizes as the target size, obtain a second adaptive resolution with a value adjacent to the target size, and adjust the resolution of each first portrait region image in the input image based on the second adaptive resolution to obtain a first adaptive image corresponding to each first portrait region image;

[0119] In one embodiment, when the input image includes multiple portraits, the portrait sizes of each first portrait area image in the input image are compared, the maximum height of the portrait sizes is determined as the target size, and the resolution of each first portrait area image in the input image is adjusted based on the second adaptive resolution of the target size to obtain the first adaptive image corresponding to each first portrait area image.

[0120] For example, the input image includes three portraits, each portrait corresponds to a first portrait area image, and the portrait sizes of the first portrait area images are 170 pixels, 130 pixels, and 150 pixels respectively, so 170 pixels is used as the target size.

[0121] The second adaptive resolution may be an adaptive resolution adjacent to the target size in the super-resolution processing resolution when the input image includes multiple portraits.

[0122] Furthermore, in addition to adjusting the resolution of the first portrait region image to the super-resolution resolution, when the input image includes multiple portraits, since the portrait sizes may not be multiples of 10, the resolution of the first portrait region image may be rounded up to a multiple of 10 when the portrait sizes are not multiples of 10, and this resolution may be used as the second adapted resolution for the first portrait region image. For example, if the resolution of the first portrait region image is 172 pixels, the second adapted resolution for the first portrait region image is "170" obtained by rounding "172" up.

[0123] S207, determining a super-resolution ratio based on the target size, and performing super-resolution processing on the first coding channel of each first adapted image based on the super-resolution ratio to obtain first channel data;

[0124] In one embodiment, when the input image is a plurality of portraits, after determining the super-resolution ratio corresponding to the target size, the first coding channel in each first adapted image in the input image is super-resolved based on the super-resolution ratio to obtain an image with a resolution of each super-resolution ratio as the first channel data.

[0125] It is understandable that, since the input image includes multiple portraits, there is more than one first adapted image corresponding to the input image, and the first adapted image of each portrait pair needs to be super-resolution processed separately.

[0126] S208, based on the super-resolution ratio, amplify the second coding channel of each first adapted image to obtain second channel data;

[0127] In one embodiment, the second coding channel in each first adapted image is amplified based on the super-resolution ratio to obtain an image with a resolution of the super-resolution ratio as the second channel data.

[0128] S209, performing image fusion on the first channel data and the second channel data to obtain first super-resolved images corresponding to the first adapted images;

[0129] In one embodiment, the first channel data and the second channel data corresponding to each first adapted image are fused to obtain a first super-resolved image corresponding to each first adapted image.

[0130] S210, performing calculation based on the first super-resolved image and the mask corresponding to the portrait coordinates to obtain a first image to be fused;

[0131] In one embodiment, after calculating the portrait coordinates of the first portrait region image to obtain a mask, calculation is performed based on the first super-resolution image and the mask to obtain a first image to be fused.

[0132] The mask may be an image calculated based on image size, used to smooth out differences in peripheral information between the first super-resolution image and the first portrait region image at the location of the input image, thereby improving the visibility of the first super-resolution image within the input image. The mask may be calculated by, after obtaining the portrait coordinates, calculating the portrait coordinates to obtain the image size of the first portrait region image, and then calculating based on the image size using a predetermined method to obtain the mask corresponding to the portrait coordinates. The mask calculation method is not limited herein.

[0133] It should be noted that since the mask is calculated based solely on the image size, the corresponding mask can be calculated after the image size is determined. If the input image is a single portrait, then since the target size of the first portrait region image corresponding to the input image is known, the corresponding mask can be calculated after the portrait coordinates are obtained in step S201. If the input image is multiple portraits, the image size of each first portrait region image is determined based on the portrait size of each first portrait region image in the input image, and the mask corresponding to each first portrait region image is then calculated based on the image size.

[0134] The image size is a value calculated based on the image coordinates. For example, if the image coordinates are (100, 100) and (270, 270), the image size can be a rectangle with a length and width of 170.

[0135] The first image to be fused can be an image used to smooth the first super-resolved image in the input image, and is calculated using the first super-resolved image and the mask. For example, if the first super-resolved image is h_sr and the mask is mask, then the first image to be fused is calculated based on "h_sr*mask".

[0136] S211, performing calculation based on the first portrait region image and the mask to obtain a second image to be fused;

[0137] In one embodiment, when the input image is a single portrait, calculation is performed based on the first portrait area image and the mask to obtain a second image to be fused corresponding to the first portrait area image; when the input image includes multiple portraits, calculation is performed based on each first portrait area image and the corresponding mask respectively to obtain a second image to be fused corresponding to each first portrait area image.

[0138] The second image to be fused can be an image used to represent the difference between the first portrait region image and the first image to be fused, and is calculated using the first portrait region image and the mask. For example, if the first portrait region image is h_so and the mask is mask, then the second image to be fused is calculated based on "h_so*mask".

[0139] S212, performing a fusion process on the first image to be fused and the second image to be fused to obtain a second portrait region image;

[0140] In one embodiment, when the input image is a single portrait, the first image to be fused and the second image to be fused are fused to obtain a second portrait area image; when the input image includes multiple portraits, the first image to be fused and the second image to be fused of each first portrait area image are fused separately to obtain a second portrait area image corresponding to each first portrait area image.

[0141] The fusion process may be a method for fusing the first image to be fused with the second image to be fused to obtain a second portrait region image. For example, if the first image to be fused is "h_sr*mask" and the second image to be fused is based on "h_so*mask", then the fusion process uses the formula "h_sr*mask+h_so*(1-mask)" (i.e., h_sr*mask+h_so-h_so*mask)) to calculate the second portrait region image.

[0142] It can be understood that since the first image to be fused is an image used to smooth the first super-resolution image, the second image to be fused is an image that represents the difference between the first portrait area image and the first image to be fused, in the first portrait area image, the second image to be fused corresponding to the first portrait area image is calculated based on the first image to be fused to obtain the second portrait area image after smoothing the first super-resolution image.

[0143] The second portrait area image may be an image used to fuse the first portrait area image in the input image, and a resolution of the second portrait area image is greater than a resolution of the first portrait area image.

[0144] S213, fusing the second portrait region image into the input image based on the portrait coordinates, and determining the fused input image as a portrait close-up image;

[0145] In one embodiment, based on the portrait coordinates corresponding to the first portrait area image, the area where the first portrait area image is located is determined in the input image, and the second portrait area image is fused with the first portrait area image so that the second portrait area image is fused into the input image, and the fused input image is determined as a portrait close-up image.

[0146] The close-up portrait image can be an image in which the portrait edge is less jagged and detail loss is minimal in the input image. Specifically, it is an image obtained by fusing the first portrait region image in the input image with the second portrait region image, which also has less jagged edges and less detail loss. As shown in Figure 6, the portrait edge in the input image is highly jagged and suffers from significant detail loss, while the close-up portrait image has less jagged edges and is rich in detail.

[0147] Furthermore, the generated close-up portrait image can be displayed on a display interface of a terminal device to achieve a close-up of the portrait. As shown in Figure 7, an input image including a portrait is cropped from the original image, and after the input image is super-resolved to obtain a close-up portrait image, the super-resolved close-up portrait image is displayed on the right side of the display interface.

[0148] Furthermore, to ensure that the resolution of the final generated image is consistent with that of the original image, and that the output image and the original image are displayed consistently on a display device, the input image needs to be upscaled to achieve the same resolution as the original image. A feasible method for this is to obtain the original resolution of the original image corresponding to the input image, adjust the input image to the original resolution to obtain a standard image, and then fuse the second portrait region image into the standard image based on the portrait coordinates to obtain a close-up portrait image corresponding to the input image.

[0149] The standard image may be an image whose resolution is consistent with the original resolution after the resolution of the input image is adjusted.

[0150] For example, the resolution of the original image may be 4K, and the resolution of the input image may be 480, then the input image is upscaled to achieve a resolution of 4K. It should be noted that since super-resolution processing of larger images requires more computing power, the method for upscaling the resolution of the input image may be to use a method that requires less computing power, such as bilinear interpolation or unidirectional interpolation.

[0151] Furthermore, when the image height of the input image is smaller than the set height, the input image is super-resolved based on the set magnification to obtain a close-up portrait image corresponding to the input image.

[0152] The height setting may be used to determine whether the current computing capability is sufficient to perform super-resolution processing on the entire input image. The specific value of the height setting may be set according to actual needs.

[0153] It is understandable that since the size of the input image can be calculated based on the image height, and the larger the image, the more computing power is required, the comparison result between the image height and the set height is used to determine whether the input image can be directly super-resolved to obtain a close-up portrait image corresponding to the input image.

[0154] The set magnification can be the magnification used for super-resolution processing of the input image. It can be understood that since the computing power required for super-resolution processing of the entire input image is relatively high and it is the maximum image size that the processing chip can process, the set magnification can be twice, or it can be set according to actual conditions.

[0155] For example, if the set height is 270 pixels and the image height of the input image is 250 pixels, the image height is less than the set height, and the input image is super-resolution processed based on the set magnification; if the set height is 270 pixels and the image height of the input image is 300 pixels, the image height is greater than the set height, and step S201 is executed.

[0156] Furthermore, the above steps are performed when the target size is smaller than the first preset size. If the target size is larger than the first preset size, the order of executing the steps needs to be adjusted. Specifically, when the target size is larger than the first preset size, the first portrait area image is super-resolved based on the preset super-resolution ratio corresponding to the first preset size to obtain a third super-resolved image. The third super-resolved image is then adjusted based on the preset resolution to obtain a second adapted image corresponding to the third super-resolved image. The first portrait area image, the second adapted image, and the mask corresponding to the portrait coordinates are fused to obtain a third portrait area image. Finally, the third portrait area image is fused into the input image based on the portrait coordinates, and the fused input image is determined as a portrait close-up image.

[0157] It is understandable that if the target size of the first portrait area image is too large, the resolution of the portrait in the first portrait area image is already very clear, and super-resolution processing of the image is not very meaningful. Therefore, a judgment can be made based on the target size to avoid unnecessary waste of resources. If the target size of the first portrait area image is not too large and is larger than the first preset size, the first portrait area image can be super-resolution processed first, and then the image can be scaled to obtain an image that is convenient for image fusion under limited computing power.

[0158] The first preset size may be a preset size used to determine the order in which image processing is performed corresponding to the target size. When the target size is larger than the first preset size, super-resolution processing is first performed on the first portrait area image based on a preset super-resolution ratio corresponding to the first preset size to obtain a third super-resolution image with improved clarity. The preset super-resolution ratio may be a preset super-resolution magnification ratio corresponding to the first preset size.

[0159] The method for generating the third super-resolved image may refer to the corresponding part of generating the first super-resolved image in steps S203 to S205, and will not be described in detail here.

[0160] The preset resolution may be a resolution corresponding to a preset first preset size, and is used to scale the third super-resolved image. The specific steps of adjusting the third super-resolved image to obtain the second adapted image, fusing the second adapted image to obtain the third portrait region image, and obtaining the portrait close-up image may be referred to in step S202 and the relevant portions of steps S210 to S213, and are not further described here.

[0161] For example, if the first preset size is 330, the preset super-resolution ratio corresponding to the first preset size is 2 times, and the preset resolution is 720, then when the target size is 400, the first portrait area image is super-resolution processed based on the 2 times ratio to obtain a third super-resolution image, and then the third super-resolution image is reduced to obtain a second adapted image with a resolution of 720. Finally, the second adapted image, the first portrait area image and the mask are fused to obtain the third portrait area image, and the third portrait area image is fused to the input image to obtain a close-up portrait image.

[0162] Furthermore, the embodiments of this specification also provide a second preset size and a third preset size, wherein the second preset size is larger than the first preset size, and the third preset size is smaller than the first preset size.

[0163] When the target size is larger than the first preset size and smaller than or equal to the second preset size, super-resolution processing is performed on the first portrait region image based on a preset super-resolution corresponding to the first preset size to obtain a third super-resolution image, and the third super-resolution image is adjusted based on a preset resolution corresponding to the first preset size to obtain a second adapted image corresponding to the third super-resolution image;

[0164] When the target size is less than or equal to the first preset size and greater than or equal to the third preset size, the first portrait area image is adjusted based on the first adaptive resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image, and the first adapted image is super-resolved based on the super-resolution corresponding to the target size to obtain a first super-resolved image.

[0165] When the target size is larger than the second preset size or smaller than or equal to the third preset size, the first portrait area image in the input image is not processed.

[0166] It can be understood that when the target size is larger than the second preset size, it can be considered that the portrait in the first portrait area image is clear enough and does not need to be processed; when the target size is smaller than the third preset size, it is considered that the portrait in the first portrait area image is too small and includes too little information. In order to reduce unnecessary waste of computing resources, it is not super-resolution processed.

[0167] For example, if the first preset size is 330, the second preset size is 540, and the third preset size is 30, then when the target size is 600, since the target size 600 is larger than the second preset size 540, the super-resolution processing is not performed on the first portrait area image;

[0168] When the target size is 10, since the target size 10 is smaller than the third preset size 30, the first portrait area image is not super-resolved.

[0169] When the target size is 200, since the target size 250 is smaller than the first preset size 330 and larger than the third preset size 30, the first portrait area image is first adjusted based on the first adaptation resolution 360 corresponding to the target size to obtain a first adaptation image, and then the first adaptation image is super-resolved based on the super-resolution corresponding to the target size to obtain a first super-resolved image.

[0170] When the target size is 400, since the target size 400 is larger than the first preset size 330 and smaller than the second preset size 540, the first portrait area image is first super-resolution processed based on the preset super-resolution corresponding to the first preset size to obtain a third super-resolution image, and then the third super-resolution image is adjusted based on the preset resolution corresponding to the first preset size to obtain a second adapted image corresponding to the third super-resolution image.

[0171] The specific values ​​of the first preset size, the second preset size and the third preset size can be set according to actual conditions.

[0172] In an embodiment of the present disclosure, a single portrait and multiple portraits are processed separately, using processing methods corresponding to the number of portraits in an input image. The first portrait region image in the input image is super-resolved at a super-resolution ratio based on a target size of the first portrait region image to obtain a first super-resolved image with a clear portrait. The first super-resolved image is fused with a mask to obtain a second portrait region image, which is then fused into the input image to obtain a close-up portrait image. Furthermore, a clear portrait image is obtained by performing targeted super-resolving on the portrait portion, which is then fused into the input image to obtain a clear and visually pleasing portrait image. This allows for clear close-up portraits to be captured with limited computing power, improving the quality of close-up image generation and the performance ratio of the image generation process. Furthermore, the target size is determined based on multiple preset sizes to determine the image processing method corresponding to the first portrait region image. This fully utilizes the image processing capabilities of a terminal device under limited computing power, improving the image processing capabilities of the terminal device for close-up portraits.

[0173] Based on the system architecture shown in Figure 1, the image processing device provided in the embodiments of this specification will be described in detail below in conjunction with Figure 8. It should be noted that the image processing device in Figure 8 is used to execute the method of the embodiments shown in Figures 2 to 7 of the embodiments of this specification. For ease of explanation, only the parts related to the embodiments of this specification are shown. For specific technical details not disclosed, please refer to the embodiments shown in Figures 2 to 7 of the embodiments of this specification.

[0174] Please refer to Figure 8, which is a schematic diagram of the structure of an image processing device according to an embodiment of this specification. As shown in Figure 8, the image processing device 1 according to the embodiment of this specification may include: a regional image acquisition unit 11, an adapted image acquisition unit 12, a super-resolution image acquisition unit 13, and a fusion unit 14.

[0175] The region image acquisition unit 11 is configured to determine a first portrait region image in the input image and obtain the portrait coordinates of the first portrait region image in the input image;

[0176] The adapted image acquisition unit 12 is configured to determine a target size of the first portrait region image, and adjust the first portrait region image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait region image;

[0177] The super-resolution image acquisition unit 13 is configured to perform super-resolution processing on the first adapted image based on the super-resolution ratio corresponding to the target size to obtain a first super-resolution image;

[0178] a fusion unit 14 for fusing the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait region image;

[0179] The fusion unit 14 is further configured to fuse the second portrait region image into the input image based on the portrait coordinates, and determine the fused input image as a portrait close-up image.

[0180] Optionally, the adapted image acquisition unit 12 is further configured to:

[0181] When the input image is a single portrait, determining a target size of a first portrait region image, obtaining a first adapted resolution adjacent to the target size value, and adjusting the resolution of the first portrait region image based on the first adapted resolution to obtain a first adapted image corresponding to the first portrait region image;

[0182] When the input image contains multiple portraits, the portrait size of each portrait in the input image is obtained, the maximum height among the portrait sizes is determined as the target size, a second adaptive resolution adjacent to the target size value is obtained, and the resolution of each first portrait area image in the first portrait area image is adjusted based on the second adaptive resolution to obtain a first adaptive image corresponding to each first portrait area image.

[0183] Optionally, the super-resolution image acquisition unit 13 is further configured to:

[0184] When the first portrait region image is a single portrait, super-resolution processing is performed on the first coding channel of the first adapted image based on the super-resolution magnification corresponding to the target size to obtain first channel data;

[0185] Amplifying the second coding channel of the first adapted image based on the super-resolution ratio to obtain second channel data;

[0186] The first channel data and the second channel data are fused to obtain a first super-resolved image corresponding to the first adapted image.

[0187] Optionally, the super-resolution image acquisition unit 13 is further configured to:

[0188] When the input image contains multiple portraits, a super-resolution ratio is determined based on the target size, and a super-resolution process is performed on the first coding channel of each first adapted image based on the super-resolution ratio to obtain first channel data.

[0189] Based on the super-resolution ratio, amplifying the second coding channel of each first adapted image to obtain second channel data;

[0190] The first channel data and the second channel data of each first adapted image are fused to obtain a first super-resolved image corresponding to each first adapted image.

[0191] Optionally, the fusion unit 14 is further configured to:

[0192] Calculating based on the first super-resolution image and the mask corresponding to the portrait coordinates to obtain a first image to be fused;

[0193] Calculating based on the first portrait area image and the mask to obtain a second image to be fused;

[0194] The first image to be fused and the second image to be fused are fused to obtain a second portrait area image.

[0195] Optionally, the image processing device 1 further includes:

[0196] The mask calculation unit 16 is configured to calculate the portrait coordinates to obtain the image size of the first portrait region image, and to calculate based on the image size in a set manner to obtain a mask corresponding to the portrait coordinates.

[0197] Optionally, the image processing device 1 further includes:

[0198] The resolution adjustment unit 17 is used to obtain the original resolution of the original image corresponding to the input image, adjust the input image to the original resolution, and obtain a standard image, where the resolution of the standard image is consistent with the original resolution;

[0199] The fusion unit 14 is further configured to fuse the second portrait region image into the standard image based on the portrait coordinates to obtain a portrait close-up image corresponding to the input image.

[0200] Optionally, the image processing device 1 further includes:

[0201] The resolution adjustment unit 17 is further configured to, when the resolution of the first super-resolution image is different from the original resolution of the original image corresponding to the input image, scale the first super-resolution image based on the original resolution to obtain a second super-resolution image having the same resolution as the original resolution, and use the second super-resolution image as the first super-resolution image corresponding to the first adapted image.

[0202] Optionally, the image processing device 1 further includes:

[0203] The super-resolution image acquisition unit 13 is further configured to perform super-resolution processing on the input image based on a set magnification when the image height of the input image is less than a set height, so as to obtain a close-up portrait image corresponding to the input image.

[0204] Optionally, the super-resolution image acquisition unit 13 is further configured to, when the target size is larger than a first preset size, perform super-resolution processing on the first portrait region image based on a preset super-resolution ratio corresponding to the first preset size to obtain a third super-resolution image;

[0205] The adapted image acquisition unit 12 is further configured to adjust the third super-resolved image based on a preset resolution to obtain a second adapted image corresponding to the third super-resolved image;

[0206] The fusion unit 14 is further configured to fuse the first portrait area image, the second adapted image, and the mask corresponding to the portrait coordinates to obtain a third portrait area image;

[0207] The fusion unit 14 is further configured to fuse the third portrait region image into the input image based on the portrait coordinates, and determine the fused input image as a portrait close-up image.

[0208] In the embodiments of this specification, a single portrait and multiple portraits are processed separately, using processing methods corresponding to the number of portraits in an input image. The first portrait region image in the input image is super-resolved at a super-resolution ratio based on a target size of the first portrait region image, resulting in a first super-resolved image with a clear portrait. The first super-resolved image is fused with a mask to obtain a second portrait region image, which is then fused into the input image to obtain a close-up portrait image. Furthermore, a clear portrait image is obtained by performing targeted super-resolving on the portrait portion, which is then fused into the input image to obtain a clear and visually pleasing portrait image. This allows for clear close-up portraits to be captured with limited computing power, improving the quality of close-up images and the performance ratio of the image generation process. Furthermore, the target size is determined based on multiple preset sizes to determine the image processing method corresponding to the first portrait region image. This fully utilizes the image processing capabilities of a terminal device under limited computing power, improving the image processing capabilities of the terminal device for close-up portraits.

[0209] The embodiments of this specification also provide a computer storage medium, which can store multiple program instructions. The program instructions are suitable for being loaded by a processor and executing the method steps of the embodiments shown in Figures 1 to 7 above. The specific execution process can be found in the specific description of the embodiments shown in Figures 1 to 7, and will not be repeated here.

[0210] Please refer to Figure 9, which provides a structural diagram of an electronic device for an embodiment of this specification. As shown in Figure 9, the electronic device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, an input / output interface 1003, a memory 1005, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 9, the memory 1005 as a computer storage medium may include an operating system, a network communication module, an input / output interface module, and an image processing application.

[0211] In the electronic device 1000 shown in FIG9 , the input / output interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user.

[0212] In one embodiment, the processor 1001 may be configured to call an image processing application stored in the memory 1005 and specifically perform the following operations:

[0213] Determine a first portrait region image in the input image, and obtain portrait coordinates of the first portrait region image in the input image;

[0214] determining a target size of the first portrait area image, and adjusting the first portrait area image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image;

[0215] Performing super-resolution processing on the first adapted image based on a super-resolution ratio corresponding to the target size to obtain a first super-resolution image;

[0216] Fusing the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait region image;

[0217] The second portrait region image is fused into the input image based on the portrait coordinates, and the fused input image is determined as a portrait close-up image.

[0218] Optionally, when the processor 1001 performs super-resolution processing on the first adapted image based on the super-resolution ratio corresponding to the target size to obtain the first super-resolution image, the processor 1001 specifically performs the following operations:

[0219] When the input image is a single portrait, determining a target size of a first portrait region image, obtaining a first adapted resolution adjacent to the target size value, and adjusting the resolution of the first portrait region image based on the first adapted resolution to obtain a first adapted image corresponding to the first portrait region image;

[0220] When the input image contains multiple portraits, the portrait size of each portrait in the input image is obtained, the maximum height among the portrait sizes is determined as the target size, a second adaptive resolution adjacent to the target size value is obtained, and the resolution of each first portrait area image in the input image is adjusted based on the second adaptive resolution to obtain a first adaptive image corresponding to each first portrait area image.

[0221] Optionally, when the processor 1001 performs super-resolution processing on the first adapted image based on the super-resolution ratio corresponding to the target size to obtain the first super-resolution image, the processor 1001 specifically performs the following operations:

[0222] When the first portrait region image is a single portrait, super-resolution processing is performed on the first coding channel of the first adapted image based on the super-resolution magnification corresponding to the target size to obtain first channel data;

[0223] Amplifying the second coding channel of the first adapted image based on the super-resolution ratio to obtain second channel data;

[0224] The first channel data and the second channel data are fused to obtain a first super-resolved image corresponding to the first adapted image.

[0225] Optionally, when the processor 1001 performs super-resolution processing on the first adapted image based on the super-resolution ratio corresponding to the target size to obtain the first super-resolution image, the processor 1001 specifically performs the following operations:

[0226] When the input image contains multiple portraits, a super-resolution ratio is determined based on the target size, and a super-resolution process is performed on the first coding channel of each first adapted image based on the super-resolution ratio to obtain first channel data.

[0227] Based on the super-resolution ratio, amplifying the second coding channel of each first adapted image to obtain second channel data;

[0228] The first channel data and the second channel data of each first adapted image are fused to obtain a first super-resolved image corresponding to each first adapted image.

[0229] Optionally, when the processor 1001 performs fusion processing on the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain the second portrait region image, the processor 1001 specifically performs the following operations:

[0230] Calculating based on the first super-resolution image and the mask corresponding to the portrait coordinates to obtain a first image to be fused;

[0231] Calculating based on the first portrait area image and the mask to obtain a second image to be fused;

[0232] The first image to be fused and the second image to be fused are fused to obtain a second portrait area image.

[0233] Optionally, the processor 1001 further performs the following operations:

[0234] The portrait coordinates are calculated to obtain the image size of the portrait area image, and a mask corresponding to the portrait coordinates is obtained by performing calculation based on the image size in a set manner.

[0235] Optionally, the processor 1001 further performs the following operations:

[0236] Obtain the original resolution of the original image corresponding to the input image, adjust the input image to the original resolution, and obtain a standard image, where the resolution of the standard image is consistent with the original resolution;

[0237] The second portrait region image is fused into the standard image based on the portrait coordinates to obtain a portrait close-up image corresponding to the input image.

[0238] Optionally, the processor 1001 further performs the following operations:

[0239] When the resolution of the first super-resolution image is different from the original resolution of the original image corresponding to the input image, the first super-resolution image is scaled based on the original resolution to obtain a second super-resolution image with the same resolution as the original resolution, and the second super-resolution image is used as the first super-resolution image corresponding to the first adapted image.

[0240] Optionally, the processor 1001 further performs the following operations:

[0241] When the image height of the input image is less than the set height, the input image is super-resolved based on the set magnification to obtain a close-up portrait image corresponding to the input image.

[0242] Optionally, after determining the target size of the first portrait area image, the processor 1001 further performs the following operations:

[0243] When the target size is larger than a first preset size, super-resolution processing is performed on the first portrait area image based on a preset super-resolution ratio corresponding to the first preset size to obtain a third super-resolution image;

[0244] Adjusting the third super-resolved image based on a preset resolution to obtain a second adapted image corresponding to the third super-resolved image;

[0245] fusing the first portrait area image, the second adapted image, and the mask corresponding to the portrait coordinates to obtain a third portrait area image;

[0246] The third portrait region image is fused into the input image based on the portrait coordinates, and the fused input image is determined as a portrait close-up image.

[0247] In an embodiment of the present disclosure, a single portrait and multiple portraits are processed separately, using processing methods corresponding to the number of portraits in an input image. The first portrait region image in the input image is super-resolved at a super-resolution ratio based on a target size of the first portrait region image to obtain a first super-resolved image with a clear portrait. The first super-resolved image is fused with a mask to obtain a second portrait region image, which is then fused into the input image to obtain a close-up portrait image. Furthermore, a clear portrait image is obtained by performing targeted super-resolving on the portrait portion, which is then fused into the input image to obtain a clear and visually pleasing portrait image. This allows for clear close-up portraits to be captured with limited computing power, improving the quality of close-up image generation and the performance ratio of the image generation process. Furthermore, the target size is determined based on multiple preset sizes to determine the image processing method corresponding to the first portrait region image. This fully utilizes the image processing capabilities of a terminal device under limited computing power, improving the image processing capabilities of the terminal device for close-up portraits.

[0248] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0249] The above disclosure is only a preferred embodiment of this specification, and certainly cannot be used to limit the scope of rights of this specification. Therefore, equivalent changes made according to the claims of this specification are still within the scope covered by this specification.

Claims

1. An image processing method, characterized in that: The method comprises: Determine a first portrait region image in an input image, and obtain portrait coordinates of the first portrait region image in the input image; determining a target size of the first portrait area image, and adjusting the first portrait area image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image; Performing super-resolution processing on the first adapted image based on the super-resolution ratio corresponding to the target size to obtain a first super-resolution image; fusing the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait region image; The second portrait region image is fused into the input image based on the portrait coordinates, and the fused input image is determined as a portrait close-up image.

2. The method according to claim 1, characterized in that The determining a target size of the first portrait area image, and adjusting the first portrait area image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image, includes: When the input image is a single portrait, determining a target size of the first portrait region image, obtaining a first adapted resolution adjacent to a value of the target size, and adjusting the resolution of the first portrait region image based on the first adapted resolution to obtain a first adapted image corresponding to the first portrait region image; When the input image includes multiple portraits, the portrait sizes of each first portrait area image in the input image are obtained, the maximum height among the portrait sizes is determined as the target size, a second adaptive resolution with a numerical value adjacent to the target size is obtained, and the resolution of each first portrait area image in the input image is adjusted based on the second adaptive resolution to obtain a first adaptive image corresponding to each first portrait area image.

3. The method according to claim 1, characterized in that The super-resolution processing of the first adapted image based on the super-resolution ratio corresponding to the target size to obtain a first super-resolution image includes: When the input image is a single person portrait, super-resolution processing is performed on the first coding channel of the first adapted image based on the super-resolution magnification corresponding to the target size to obtain first channel data; Based on the super-resolution ratio, amplifying the second coding channel of the first adapted image to obtain second channel data; The first channel data and the second channel data are subjected to image fusion to obtain a first super-resolved image corresponding to the first adapted image.

4. The method according to claim 1, wherein The super-resolution processing of the first adapted image based on the super-resolution ratio corresponding to the target height and target size to obtain a first super-resolution image includes: When the input image contains a plurality of portraits, determining a super-resolution ratio based on the target size, and performing super-resolution processing on the first coding channel of each first adapted image based on the super-resolution ratio to obtain first channel data; Based on the super-resolution ratio, amplifying the second coding channel of each first adapted image to obtain second channel data; The first channel data and the second channel data of each first adapted image are respectively fused to obtain a first super-resolved image corresponding to each first adapted image.

5. The method according to claim 1, wherein The fusing the first portrait area image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait area image includes: Calculating based on the first super-resolved image and the mask corresponding to the portrait coordinates to obtain a first image to be fused; Performing calculation based on the first portrait area image and the mask to obtain a second image to be fused; The first image to be fused and the second image to be fused are fused to obtain a second portrait area image.

6. The method according to claim 1, characterized in that After obtaining the portrait coordinates of the first portrait area image in the input image, the method further includes: The portrait coordinates are calculated to obtain an image size of the first portrait region image, and a mask corresponding to the portrait coordinates is obtained by performing calculation based on the image size in a set manner.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining an original resolution of an original image corresponding to the input image, adjusting the input image to the original resolution, and obtaining a standard image, wherein the resolution of the standard image is consistent with the original resolution; The second portrait region image is fused into the standard image based on the portrait coordinates to obtain a portrait close-up image corresponding to the input image.

8. The method according to claim 1, characterized in that After obtaining the first super-resolved image, the method further includes: When the resolution of the first super-resolution image is different from the original resolution of the original image corresponding to the input image, the first super-resolution image is scaled based on the original resolution to obtain a second super-resolution image with the same resolution as the original resolution, and the second super-resolution image is used as the first super-resolution image corresponding to the first adapted image.

9. The method according to claim 1, characterized in that The method further comprises: When the image height of the input image is smaller than the set height, super-resolution processing is performed on the input image based on a set magnification to obtain a close-up portrait image corresponding to the input image.

10. The method according to claim 1, characterized in that After determining the target size of the first portrait area image, the method further includes: When the target size is larger than a first preset size, super-resolution processing is performed on the first portrait area image based on a preset super-resolution ratio corresponding to the first preset size to obtain a third super-resolution image; Adjusting the third super-resolved image based on a preset resolution to obtain a second adapted image corresponding to the third super-resolved image; fusing the first portrait area image, the second adapted image, and the mask corresponding to the portrait coordinates to obtain a third portrait area image; The third portrait region image is fused into the input image based on the portrait coordinates, and the fused input image is determined as a portrait close-up image.

11. An image processing device, characterized in that: The device comprises: a region image acquisition unit, configured to determine a first portrait region image in an input image and acquire portrait coordinates of the first portrait region image in the input image; an adapted image acquisition unit, configured to determine a target size of the first portrait area image, and adjust the first portrait area image based on an adapted resolution corresponding to the target size to obtain a first adapted image corresponding to the first portrait area image; a super-resolution image acquisition unit, configured to perform super-resolution processing on the first adapted image based on the super-resolution ratio corresponding to the target size, so as to obtain a first super-resolution image; a fusion unit, configured to fuse the first portrait region image, the first super-resolution image, and the mask corresponding to the portrait coordinates to obtain a second portrait region image; a fusion unit, further configured to fuse the second portrait region image into the input image based on the portrait coordinates, The fused input image is determined as a portrait close-up image.

12. A computer storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 10.

13. An electronic device comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method according to any one of claims 1 to 10.

Images