Image enhancement method and device, electronic equipment and computer program product

Abstract

The invention relates to an image enhancement method and device, electronic equipment and a computer program product, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first quality enhanced image and a second quality enhanced image corresponding to an original image, wherein the image quality of the first quality enhanced image is higher than that of the second quality enhanced image; obtaining a main edge guide graph corresponding to the original image according to the first quality enhanced image; and fusing the first quality enhanced image and the second quality enhanced image according to the main edge guide map to obtain a target enhanced image of the original image. According to the image quality enhancement method and device, the first quality enhancement image with an excessively strong processing effect and the second quality enhancement image with a relatively weak processing effect are fused, so that the problem of excessive image sharpening can be avoided while the definition is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image enhancement method, an image enhancement device, an electronic device, and a computer program product. Background Art

[0002] With the popularization of deep learning and the development of large models, the computing resource cost required for deep learning models is getting higher and higher. In actual application scenarios, if better image enhancement effects are to be achieved, higher requirements are placed on the complexity of the model.

[0003] In contrast, lightweight models can bring greater benefits and cost-effectiveness, and are receiving more and more attention. However, the complexity of lightweight models is constrained, and it is difficult to achieve a balance between smoothness and sharpness during image processing, and it is impossible to improve clarity while avoiding the problem of over-sharpening.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide an image enhancement method, an image enhancement device, an electronic device and a computer program product, which can at least to a certain extent improve the clarity while avoiding the problem of excessive image sharpening.

[0006] According to a first aspect of the present disclosure, there is provided an image enhancement method, comprising:

[0007] Acquire a first quality-enhanced image and a second quality-enhanced image corresponding to the original image, wherein the image quality of the first quality-enhanced image is higher than that of the second quality-enhanced image;

[0008] Obtaining a main edge guide map corresponding to the original image according to the first quality enhanced image;

[0009] The first quality enhanced image and the second quality enhanced image are fused according to the main edge guide map to obtain a target enhanced image of the original image.

[0010] In an exemplary embodiment of the present disclosure, the fusing the first quality enhanced image and the second quality enhanced image according to the main edge guide map to obtain a target enhanced image of the original image includes:

[0011] Performing a negation operation on the main edge guide map to obtain a reverse main edge guide map corresponding to the main edge guide map;

[0012] Obtain a first pixel map according to the first quality enhanced image and the reverse main edge guide map, and obtain a second pixel map according to the second quality enhanced image and the main edge guide map;

[0013] The first pixel map and the second pixel map are fused to obtain a target enhanced image of the original image.

[0014] In an exemplary embodiment of the present disclosure, the first quality-enhanced image has a higher degree of sharpness than the second quality-enhanced image.

[0015] In an exemplary embodiment of the present disclosure, acquiring a first quality enhanced image and a second quality enhanced image corresponding to the original image includes:

[0016] Inputting the original image into a first quality enhancement model to obtain a first quality enhanced image corresponding to the original image;

[0017] The original image is input into a second quality enhancement model to obtain a second quality enhanced image corresponding to the original image.

[0018] In an exemplary embodiment of the present disclosure, the first quality enhancement model includes an enhancement model obtained by training with strong degradation degree data, or an enhancement model obtained by training with mean absolute error loss; the second quality enhancement model includes an enhancement model obtained by training with weak degradation degree data, or an enhancement model obtained by training with mean square error loss.

[0019] In an exemplary embodiment of the present disclosure, obtaining a main edge guide map corresponding to the original image according to the first quality enhanced image includes:

[0020] Performing Gaussian blur processing on the first quality enhanced image to obtain a Gaussian blurred image corresponding to the first quality enhanced image;

[0021] Performing filtering processing on the Gaussian blurred image to obtain a filtered image corresponding to the first quality enhanced image;

[0022] Normalizing the filtered image to obtain a normalized image corresponding to the first quality enhanced image;

[0023] Threshold truncation processing is performed on the pixels in the normalized image to obtain a main edge guide map corresponding to the original image.

[0024] In an exemplary embodiment of the present disclosure, filtering the Gaussian blurred image to obtain a filtered image corresponding to the first quality enhanced image includes:

[0025] Filtering the Gaussian blurred image using a horizontal edge filter to obtain a horizontal filtered image;

[0026] Performing filtering processing on the Gaussian blurred image by using an edge filter in a vertical direction to obtain a vertical filtered image;

[0027] The horizontal filtered image and the vertical filtered image are fused to obtain a filtered image corresponding to the first quality enhanced image.

[0028] In an exemplary embodiment of the present disclosure, performing threshold truncation processing on pixel points in the normalized image to obtain a main edge guide map corresponding to the original image includes:

[0029] The pixels in the normalized image are subjected to threshold truncation processing through a preset pixel threshold, the pixels whose values ​​are greater than or equal to the pixel threshold are retained, and the pixels whose values ​​are less than the pixel threshold are filtered out to obtain the main edge guide map corresponding to the original image.

[0030] According to a second aspect of the present disclosure, there is provided an image enhancement device, comprising:

[0031] An image acquisition module is configured to acquire a first quality enhanced image and a second quality enhanced image corresponding to the original image, wherein the image quality of the first quality enhanced image is higher than that of the second quality enhanced image;

[0032] A guide map generation module, configured to obtain a main edge guide map corresponding to the original image according to the first quality enhanced image;

[0033] The image fusion module is configured to fuse the first quality enhanced image and the second quality enhanced image according to the main edge guidance map to obtain a target enhanced image of the original image.

[0034] In an exemplary embodiment of the present disclosure, the image fusion module includes:

[0035] A main edge guide map inversion unit is configured to perform an inversion operation on the main edge guide map to obtain a reverse main edge guide map corresponding to the main edge guide map;

[0036] A pixel map obtaining unit is configured to obtain a first pixel map according to the first quality enhanced image and the reverse main edge guide map, and obtain a second pixel map according to the second quality enhanced image and the main edge guide map;

[0037] The pixel map fusion unit is configured to fuse the first pixel map and the second pixel map to obtain a target enhanced image of the original image.

[0038] In an exemplary embodiment of the present disclosure, the image acquisition module includes:

[0039] A first quality enhanced image acquisition unit is configured to input the original image into a first quality enhancement model to obtain a first quality enhanced image corresponding to the original image;

[0040] The second quality enhanced image acquisition unit is configured to input the original image into a second quality enhancement model to obtain a second quality enhanced image corresponding to the original image.

[0041] In an exemplary embodiment of the present disclosure, the guide map generation module includes:

[0042] a Gaussian blurred image obtaining unit, configured to perform Gaussian blur processing on the first quality enhanced image to obtain a Gaussian blurred image corresponding to the first quality enhanced image;

[0043] a filtered image obtaining unit, configured to perform filtering processing on the Gaussian blurred image to obtain a filtered image corresponding to the first quality enhanced image;

[0044] a normalized image obtaining unit, configured to perform normalization processing on the filtered image to obtain a normalized image corresponding to the first quality enhanced image;

[0045] The main edge guide map obtaining unit is configured to perform threshold truncation processing on the pixels in the normalized image to obtain the main edge guide map corresponding to the original image.

[0046] In an exemplary embodiment of the present disclosure, the filtered image obtaining unit includes:

[0047] A horizontal filtered image obtaining unit is configured to perform filtering processing on the Gaussian blurred image through an edge filter in a horizontal direction to obtain a horizontal filtered image;

[0048] A vertical filtered image obtaining unit is configured to perform filtering processing on the Gaussian blurred image through an edge filter in a vertical direction to obtain a vertical filtered image;

[0049] The filtered image fusion unit is configured to fuse the horizontal filtered image and the vertical filtered image to obtain a filtered image corresponding to the first quality enhanced image.

[0050] In an exemplary embodiment of the present disclosure, the main edge guide map obtaining unit includes:

[0051] The threshold truncation processing unit is configured to perform threshold truncation processing on the pixels in the normalized image through a preset pixel threshold, retain the pixels whose pixel values ​​are greater than or equal to the pixel threshold, and filter the pixels whose pixel values ​​are less than the pixel threshold, so as to obtain the main edge guide map corresponding to the original image.

[0052] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any one of the above-mentioned image enhancement methods.

[0053] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the image enhancement method described in any one of the above is implemented.

[0054] The exemplary embodiments of the present disclosure may have the following beneficial effects:

[0055] In the image enhancement method of the example implementation mode of the present disclosure, by fusing the first quality enhanced image with too strong processing effect and the second quality enhanced image with weaker processing effect, on the one hand, the over-sharpening correction can be performed on the output effect of any lightweight model, while improving the clarity, avoiding the problem of excessive picture sharpening, maintaining the effect of picture details, taking into account the overall picture quality, and achieving subjectively excellent picture quality effect under the background of the lightweight model; on the other hand, there is no need to train a complex deep learning model. When the expression ability of the lightweight deep learning model is limited, the computational cost and subjective picture quality are taken into account to quickly optimize the problem of over-sharpening of the picture.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0058] Figure 1 A schematic diagram of the training process of an enhancement / super-resolution model in a related embodiment of the present disclosure is shown;

[0059] Figure 2 A schematic diagram showing a flow chart of an image enhancement method according to an exemplary embodiment of the present disclosure;

[0060] Figure 3A schematic diagram of a process of obtaining a main edge guide map according to an exemplary embodiment of the present disclosure is shown;

[0061] Figure 4 A schematic diagram showing the process of image fusion in an exemplary embodiment of the present disclosure is shown;

[0062] Figure 5 A schematic diagram of a process for constructing a main edge guide graph in a specific embodiment of the present disclosure is shown;

[0063] Figure 6 A schematic diagram showing a flow chart of an image enhancement method in a specific embodiment of the present disclosure is shown;

[0064] Figure 7 A schematic diagram of a fusion process based on a main edge guide graph in a specific embodiment of the present disclosure is shown;

[0065] Figure 8 An image enhancement effect diagram in a specific embodiment of the present disclosure is shown;

[0066] Fig. 9 A block diagram of an image enhancement device according to an exemplary embodiment of the present disclosure is shown;

[0067] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0068] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0069] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein.

[0070] The following example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0071] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0072] The scaling law in deep learning means that the larger the model, the better the effect, while the smaller the model, the worse the effect. This is especially true for computationally intensive enhancement processing tasks such as image and video super-resolution.

[0073] The super-resolution task is an ill-posed problem, that is, for a given low-resolution image, it can be obtained by sampling multiple high-resolution images, and the super-resolution task is to generate such a high-resolution image that matches the low-resolution image. Since the input low-resolution image often has all details lost, the high-resolution image can only be reconstructed by relying on the network's effective learning of a large amount of natural structural information of the image, which leads to the need to design a more complex model structure, which is contrary to the requirements of a lightweight model.

[0074] In actual application scenarios, such as short video application scenarios, the implementation of mobile super-resolution algorithms is more direct for the business. As the last link in user video consumption, the image quality of mobile super-resolution algorithms directly affects the user's consumption experience, especially since the display screens of mobile devices are often small. Therefore, short video users have a stronger perception of over-sharp images. Once the image is over-sharp, even if the video content is interesting, problems such as white edges and over-smoothness in the image will seriously affect the user's impression of the platform, which in turn has a negative impact on the user's sustainable consumption.

[0075] In some related embodiments, image enhancement can be achieved by training an enhancement / super-resolution model. The training process of the enhancement / super-resolution model is as follows: Figure 1 As shown in the figure, a low-quality image is input, and a pseudo high-quality image is obtained through neural network processing. Then, the loss is calculated for the pseudo high-quality image and the real high-quality image corresponding to the low-quality image. The gradient is calculated according to the value of the loss function, and the parameters of the neural network are updated until the final loss is basically unchanged. This is considered convergence. In the case where the neural network is a lightweight model, the model's own expression and learning capabilities are limited, and enhancement / super-resolution is a computationally intensive pixel generation task. Therefore, the ability of the lightweight model can easily reach its upper limit, making the final output of the model either the main lines are very clear but other dense textures are too sharp, or the overall clarity is relatively uniform but not clear enough.

[0076] Since the model is small, has few layers and channels, it is difficult for a lightweight model to have a global perception field of view, so L N The paradigm makes it difficult for lightweight models to have the ability of local adaptive enhancement, and it is also difficult to achieve a balance between smoothness and sharpness to improve overall clarity and increase details. The complexity of lightweight models is constrained, and the loss function is also difficult to achieve satisfactory results.

[0077] In other related embodiments, starting from the perspective of data degradation, by controlling the degree of degradation, the clarity of the final result can be improved with physical perception without being too sharp.

[0078] However, data degradation methods are sensitive to application scenarios and require precise control of the degree of degradation, which places high demands on developers. The model has limited learning capabilities and may be able to learn a certain degree of degradation, but it cannot adaptively enhance the uneven image quality in short video scenes, for example. This may result in weak processing of some videos or excessive processing of others, failing to maximize benefits.

[0079] Based on the above problems, this exemplary embodiment first provides an image enhancement method. Figure 2 As shown, the above image enhancement method may include the following steps:

[0080] Step S210: Obtain a first quality enhanced image and a second quality enhanced image corresponding to the original image, wherein the image quality of the first quality enhanced image is higher than that of the second quality enhanced image.

[0081] Step S220: Obtain a main edge guide map corresponding to the original image according to the first quality enhanced image.

[0082] Step S230: Fusing the first quality enhanced image and the second quality enhanced image according to the main edge guide map to obtain a target enhanced image of the original image.

[0083] In the image enhancement method of the example implementation mode of the present disclosure, by fusing the first quality enhanced image with too strong processing effect and the second quality enhanced image with weaker processing effect, on the one hand, the over-sharpening correction can be performed on the output effect of any lightweight model, while improving the clarity, avoiding the problem of excessive picture sharpening, maintaining the effect of picture details, taking into account the overall picture quality, and achieving subjectively excellent picture quality effect under the background of the lightweight model; on the other hand, there is no need to train a complex deep learning model. When the expression ability of the lightweight deep learning model is limited, the computational cost and subjective picture quality are taken into account to quickly optimize the problem of over-sharpening of the picture.

[0084] Next, combine Figures 3 to 8 The above steps of this exemplary embodiment are described in more detail.

[0085] In step S210, a first quality enhanced image and a second quality enhanced image corresponding to the original image are obtained, and the image quality of the first quality enhanced image is higher than that of the second quality enhanced image.

[0086] In this example implementation, the original image is a low-quality image with low clarity or low resolution, and the first quality enhanced image and the second quality enhanced image corresponding to the original image are enhanced images obtained after the original image is enhanced to different degrees, wherein the image quality of the first quality enhanced image is higher than that of the second quality enhanced image, and the sharpening degree of the first quality enhanced image is also higher than that of the second quality enhanced image. For example, the first quality enhanced image is a strong effect image with clear main lines but too sharp in other dense textures, and the second quality enhanced image is a weak effect image with relatively uniform overall clarity but insufficient clarity.

[0087] In this example implementation, the original image may be input into a first quality enhancement model to obtain a first quality enhanced image corresponding to the original image, wherein the first quality enhancement model includes an enhancement model trained by strong degradation degree data or an enhancement model trained by mean absolute error loss.

[0088] In the application scenario of the lightweight model, the first quality enhancement model refers to a strong model with a strong image enhancement effect, which can be a model trained using L1 loss or a model trained with strong degradation degree data. By inputting the original image into the strong model, a strong effect image can be obtained.

[0089] In this example implementation, the original image may be input into the second quality enhancement model to obtain a second quality enhanced image corresponding to the original image, wherein the second quality enhancement model includes an enhancement model trained by weak degradation degree data or an enhancement model trained by mean square error loss.

[0090] In the application scenario of lightweight models, the second quality enhancement model refers to a weak model with weak image enhancement effect, which can be a model trained using L2 loss or a model trained with weak degradation degree data. Inputting the original image into the weak model can obtain a weak effect image.

[0091] The weak model and the strong model in this example implementation can be any lightweight model with performance bottlenecks, as long as the weak model effect is more conservative and the strong model effect is sharpened. Both models cannot achieve a subjective balanced effect, either the overall clarity is relatively uniform but the clarity is insufficient, or the main lines are very clear but other dense textures are too sharp.

[0092] In step S220, a main edge guide map corresponding to the original image is obtained according to the first quality enhanced image.

[0093] In this example implementation, the main edge guide map can be constructed based on the first quality enhanced image, that is, the oversharp image output by the strong model. Figure 3 As shown, obtaining a main edge guide map corresponding to the original image according to the first quality enhanced image may specifically include the following steps:

[0094] Step S310: Perform Gaussian blur processing on the first quality enhanced image to obtain a Gaussian blurred image corresponding to the first quality enhanced image.

[0095] First, the first quality enhanced image, that is, the strong effect image or the oversharp image, may be subjected to Gaussian blur processing to weaken the influence of oversharpness and obtain a Gaussian blurred image corresponding to the first quality enhanced image.

[0096] Step S320: Filter the Gaussian blurred image to obtain a filtered image corresponding to the first quality enhanced image.

[0097] In this example implementation, the Gaussian blurred image can be filtered using a horizontal edge filter to obtain a horizontal filtered image, and the Gaussian blurred image can be filtered using a vertical edge filter to obtain a vertical filtered image. The horizontal filtered image and the vertical filtered image are then fused to obtain a filtered image corresponding to the first quality enhanced image.

[0098] For example, the Gaussian blurred image can be filtered using Sobel filters in the horizontal and vertical directions respectively, and the filter images in the two directions are weightedly fused to obtain a filtered image corresponding to the first quality enhanced image. The Sobel operator is a linear filter used for edge detection. Filtering in the horizontal and vertical directions and then fusing can improve the filtering effect of the image.

[0099] Step S330: normalize the filtered image to obtain a normalized image corresponding to the first quality enhanced image.

[0100] The fused filtered image is normalized to obtain a normalized image corresponding to the first quality enhanced image.

[0101] Step S340: Perform threshold truncation processing on the pixels in the normalized image to obtain a main edge guide map corresponding to the original image.

[0102] In this example implementation, the pixels in the normalized image can be threshold truncate using a preset pixel threshold, pixels whose values ​​are greater than or equal to the pixel threshold are retained, and pixels whose values ​​are less than the pixel threshold are filtered out to obtain a main edge guide map corresponding to the original image.

[0103] For example, the pixel values ​​in the normalized image are 0 to 255, and the pixel threshold is 120. Pixels greater than or equal to 120 are retained, and other pixels are filtered. The main edge of the image can be retained through threshold truncation processing, and small edges and noise can be filtered.

[0104] The main edge guidance map is a binary image. The physical meaning of the non-zero pixel value in the main edge guidance map is the extraction of the edge of the main body in the picture. It can be considered as a structural area that the human eye perceives more strongly. If it is too sharp, it is easy to be noticed by the human eye. The part with a zero pixel value in the main edge guidance map represents a flat area or a weak texture area. These areas are too smooth and easy to be perceived as false by the human eye.

[0105] In step S230, the first quality enhanced image and the second quality enhanced image are fused according to the main edge guide map to obtain a target enhanced image of the original image.

[0106] In this example implementation, the first quality enhanced image and the second quality enhanced image may be fused based on the main edge guide map to obtain a target enhanced image that is not over-sharpened but has details preserved after the advantages and disadvantages of the two images are complemented.

[0107] In this example implementation, Figure 4 As shown, the first quality enhanced image and the second quality enhanced image are fused according to the main edge guide map to obtain a target enhanced image of the original image, which may specifically include the following steps:

[0108] Step S410: Perform an inversion operation on the main edge guidance map to obtain a reverse main edge guidance map corresponding to the main edge guidance map.

[0109] The inversion operation is performed on all the pixels on the main edge guide map, the non-zero part of the pixel value is converted to 0, and the part of the pixel value 0 is converted to non-0, so that the reverse main edge guide map corresponding to the main edge guide map can be obtained.

[0110] Step S420: Obtain a first pixel map according to the first quality enhanced image and the reverse main edge guide map, and obtain a second pixel map according to the second quality enhanced image and the main edge guide map.

[0111] The first quality enhanced image and the reverse main edge guide map are multiplied to obtain a first pixel map. The second quality enhanced image and the main edge guide map are multiplied to obtain a second pixel map.

[0112] Step S430: Fuse the first pixel map and the second pixel map to obtain a target enhanced image of the original image.

[0113] Finally, the first pixel map and the second pixel map are superimposed to obtain a target enhanced image of the original image.

[0114] The final image effect is that the edges of the main body are clear but not sharp, while the weak texture area is not smooth and rich in details. The pixels in the weak effect image are retained for the edge area of ​​the main body, and the pixels in the strong effect (oversharp) image are retained for the weak texture and flat area, so as to avoid oversharpening but improve clarity.

[0115] like Figure 5 FIG. 1 is a flow chart of constructing a main edge guide graph in a specific embodiment of the present disclosure. The specific steps of the flow chart are as follows:

[0116] Step S510: Gaussian blur.

[0117] For the strong effect image (i.e. oversharp image), Gaussian blur is first performed to weaken the effect of oversharpness.

[0118] Step S520: Horizontal filtering.

[0119] Step S530: Vertical filtering.

[0120] The blurred image is processed using Sobel filtering in horizontal and vertical directions respectively.

[0121] Step S540: Weighted fusion.

[0122] The filter images in two directions are weightedly fused.

[0123] Step S550: Normalization.

[0124] Normalize the fused filter image.

[0125] Step S560: Threshold truncation.

[0126] Finally, threshold truncation is performed to obtain the main edge guide map.

[0127] Figure 6 A schematic flow chart of an image enhancement method in a specific embodiment of the present disclosure is shown. A weak effect image is obtained through a weak model, a strong effect image is obtained through a strong model, and then the weak effect image and the strong effect image are fused based on the main edge guide image, and finally a clear but not sharp effect image is obtained.

[0128] Figure 7 A schematic diagram of the fusion process based on the main edge guide map in a specific embodiment of the present disclosure is shown. The strong effect map is multiplied by the reverse main edge guide map, the weak effect map is multiplied by the main edge guide map, and then superimposed to obtain the final effect map.

[0129] Figure 8 An image enhancement effect diagram in a specific embodiment of the present disclosure is shown. Among them, the upper left corner of each figure is the objective image quality score of the corresponding image (the score range is 0-5, and the higher the score, the higher the image quality). It can be seen that given a low-resolution image, the weak effect super-resolution image processed by the weak model is too smooth and lacks clarity, while the effect processed by the strong model is too sharp and the white edges are relatively serious. The optimized super-resolution image obtained by the image enhancement method in this example embodiment can complement the advantages and disadvantages of the two to obtain an image that is not over-sharpened but has details preserved, and its objective image quality score is higher than the scores of the effect diagrams obtained by the two models.

[0130] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0131] Furthermore, the present disclosure also provides an image enhancement device. Fig. 9 As shown, the image enhancement device may include an image acquisition module 910, a guide map generation module 920, and an image fusion module 930. Among them:

[0132] The image acquisition module 910 is configured to acquire a first quality enhanced image and a second quality enhanced image corresponding to the original image, wherein the image quality of the first quality enhanced image is higher than that of the second quality enhanced image;

[0133] The guide map generation module 920 is configured to obtain a main edge guide map corresponding to the original image according to the first quality enhanced image;

[0134] The image fusion module 930 is configured to fuse the first quality enhanced image and the second quality enhanced image according to the main edge guidance map to obtain a target enhanced image of the original image.

[0135] In some exemplary embodiments of the present disclosure, the first quality-enhanced image has a higher degree of sharpness than the second quality-enhanced image.

[0136] In some exemplary embodiments of the present disclosure, the image acquisition module 910 may include a first quality enhanced image acquisition unit and a second quality enhanced image acquisition unit.

[0137] The first quality enhanced image acquisition unit is configured to input the original image into the first quality enhancement model to obtain a first quality enhanced image corresponding to the original image;

[0138] The second quality enhanced image acquisition unit is configured to input the original image into the second quality enhancement model to obtain a second quality enhanced image corresponding to the original image.

[0139] In some exemplary embodiments of the present disclosure, the first quality enhancement model includes an enhancement model trained by data with a strong degree of degradation, or an enhancement model trained by a mean absolute error loss; the second quality enhancement model includes an enhancement model trained by data with a weak degree of degradation, or an enhancement model trained by a mean square error loss.

[0140] In some exemplary embodiments of the present disclosure, the guide map generation module 920 may include a Gaussian blur image acquisition unit, a filter image acquisition unit, a normalized image acquisition unit, and a main edge guide map acquisition unit.

[0141] The Gaussian blurred image obtaining unit is configured to perform Gaussian blur processing on the first quality enhanced image to obtain a Gaussian blurred image corresponding to the first quality enhanced image;

[0142] The filtered image obtaining unit is configured to perform filtering processing on the Gaussian blurred image to obtain a filtered image corresponding to the first quality enhanced image;

[0143] The normalized image obtaining unit is configured to perform normalization processing on the filtered image to obtain a normalized image corresponding to the first quality enhanced image;

[0144] The main edge guide map obtaining unit is configured to perform threshold truncation processing on pixel points in the normalized image to obtain a main edge guide map corresponding to the original image.

[0145] In some exemplary embodiments of the present disclosure, the filtered image acquisition unit may include a horizontal filtered image acquisition unit, a vertical filtered image acquisition unit, and a filtered image fusion unit.

[0146] The horizontal filtered image obtaining unit is configured to filter the Gaussian blurred image through an edge filter in a horizontal direction to obtain a horizontal filtered image;

[0147] The vertical filtered image obtaining unit is configured to filter the Gaussian blurred image through an edge filter in a vertical direction to obtain a vertical filtered image;

[0148] The filtered image fusion unit is configured to fuse the horizontal filtered image and the vertical filtered image to obtain a filtered image corresponding to the first quality enhanced image.

[0149] In some exemplary embodiments of the present disclosure, the main edge guide map obtaining unit may include a threshold truncation processing unit, which is configured to perform threshold truncation processing on pixels in the normalized image through a preset pixel threshold, retain pixels whose pixel values ​​are greater than or equal to the pixel threshold, and filter pixels whose pixel values ​​are less than the pixel threshold, to obtain the main edge guide map corresponding to the original image.

[0150] In some exemplary embodiments of the present disclosure, the image fusion module 930 may include a main edge guide map inversion unit, a pixel map acquisition unit, and a pixel map fusion unit.

[0151] The main edge guide map inversion unit is configured to perform an inversion operation on the main edge guide map to obtain a reverse main edge guide map corresponding to the main edge guide map;

[0152] The pixel map obtaining unit is configured to obtain a first pixel map according to the first quality enhanced image and the reverse main edge guide map, and obtain a second pixel map according to the second quality enhanced image and the main edge guide map;

[0153] The pixel map fusion unit is configured to fuse the first pixel map and the second pixel map to obtain a target enhanced image of the original image.

[0154] The specific details of each module / unit in the above-mentioned image enhancement device have been described in detail in the corresponding method embodiment part, and will not be repeated here.

[0155] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown.

[0156] It should be noted that Fig.10 The computer system 1000 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0157] like Fig.10As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. Various programs and data required for system operation are also stored in the RAM 1003. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0158] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0159] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 1009, and / or installed from a removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, various functions defined in the system of the present disclosure are executed.

[0160] The exemplary embodiment of the present disclosure also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the information prompt method for reward redemption is implemented.

[0161] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0162] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0163] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0164] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on the electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the information prompt method for reward redemption described above.

[0165] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0166] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules described above can be concretized in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be concretized.

[0167] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0168] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image enhancement method, characterized in that: include: Acquire a first quality-enhanced image and a second quality-enhanced image corresponding to the original image, wherein the image quality of the first quality-enhanced image is higher than that of the second quality-enhanced image; Obtaining a main edge guide map corresponding to the original image according to the first quality enhanced image; The first quality enhanced image and the second quality enhanced image are fused according to the main edge guide map to obtain a target enhanced image of the original image.

2. The image enhancement method according to claim 1, characterized in that: The step of fusing the first quality enhanced image and the second quality enhanced image according to the main edge guide map to obtain a target enhanced image of the original image includes: Performing a negation operation on the main edge guide map to obtain a reverse main edge guide map corresponding to the main edge guide map; Obtain a first pixel map according to the first quality enhanced image and the reverse main edge guide map, and obtain a second pixel map according to the second quality enhanced image and the main edge guide map; The first pixel map and the second pixel map are fused to obtain a target enhanced image of the original image.

3. The image enhancement method according to claim 1, characterized in that: The first quality-enhanced image has a higher sharpness than the second quality-enhanced image.

4. The image enhancement method according to claim 1, characterized in that: The step of acquiring a first quality enhanced image and a second quality enhanced image corresponding to the original image comprises: Inputting the original image into a first quality enhancement model to obtain a first quality enhanced image corresponding to the original image; The original image is input into a second quality enhancement model to obtain a second quality enhanced image corresponding to the original image.

5. The image enhancement method according to claim 4, characterized in that: The first quality enhancement model includes an enhancement model obtained by training with data of strong degradation degree, or an enhancement model obtained by training with mean absolute error loss; the second quality enhancement model includes an enhancement model obtained by training with data of weak degradation degree, or an enhancement model obtained by training with mean square error loss.

6. The image enhancement method according to claim 1, characterized in that: The step of obtaining a main edge guide map corresponding to the original image according to the first quality enhanced image includes: Performing Gaussian blur processing on the first quality enhanced image to obtain a Gaussian blurred image corresponding to the first quality enhanced image; Performing filtering processing on the Gaussian blurred image to obtain a filtered image corresponding to the first quality enhanced image; Normalizing the filtered image to obtain a normalized image corresponding to the first quality enhanced image; Threshold truncation processing is performed on the pixels in the normalized image to obtain a main edge guide map corresponding to the original image.

7. The image enhancement method according to claim 6, characterized in that: The filtering process is performed on the Gaussian blurred image to obtain a filtered image corresponding to the first quality enhanced image, including: Filtering the Gaussian blurred image using a horizontal edge filter to obtain a horizontal filtered image; Performing filtering processing on the Gaussian blurred image by using an edge filter in a vertical direction to obtain a vertical filtered image; The horizontal filtered image and the vertical filtered image are fused to obtain a filtered image corresponding to the first quality enhanced image.

8. The image enhancement method according to claim 6, characterized in that: The step of performing threshold truncation processing on the pixels in the normalized image to obtain a main edge guide map corresponding to the original image includes: The pixels in the normalized image are subjected to threshold truncation processing through a preset pixel threshold, the pixels whose values ​​are greater than or equal to the pixel threshold are retained, and the pixels whose values ​​are less than the pixel threshold are filtered out to obtain the main edge guide map corresponding to the original image.

9. An image enhancement device, characterized in that: include: An image acquisition module is configured to acquire a first quality enhanced image and a second quality enhanced image corresponding to the original image, wherein the image quality of the first quality enhanced image is higher than that of the second quality enhanced image; A guide map generation module, configured to obtain a main edge guide map corresponding to the original image according to the first quality enhanced image; The image fusion module is configured to fuse the first quality enhanced image and the second quality enhanced image according to the main edge guidance map to obtain a target enhanced image of the original image.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image enhancement method as described in any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image enhancement method according to any one of claims 1 to 8 is implemented.

Images