A photographic image enhancement method and system

Abstract

The application discloses a kind of photographic image enhancement method and system, obtain blurred image, the blurred image is obtained by contour structure, and training fuzzy contour position is obtained.Based on the blurred image and the training fuzzy contour image, clear image is obtained by enhancement image generation network. With contour as benchmark, a plurality of regions are obtained, and the regions are respectively subjected to clear processing. Finally, the clear image is obtained by merging the regions and the image subjected to clear processing with the complete image, so that the contour of the saved image is not lost in the clear processing process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a photography image enhancement method and system. BACKGROUND

[0002] At present, due to many images, especially the old photos, often appear low pixel, image is not clear, so usually need to clear the image processing, but now many image processing process is not accurate, often blurred boundary, blurred outline, resulting in the outline of the image and the original outline deviation, resulting in the image and the original image is not similar enough. SUMMARY

[0003] The present application provides a photography image enhancement method and system to solve the above problems in the prior art.

[0004] In a first aspect, the present application provides a photography image enhancement method, comprising:

[0005] obtaining a blurred image; the blurred image is an image that needs to be enhanced;

[0006] obtaining a training blurred outline position by a contour structure based on the blurred image;

[0007] obtaining a clear image by an enhanced image generation network based on the blurred image and the training blurred outline image;

[0008] The image enhancement model comprises a contour structure, an enhanced image generation network and an enhanced image discrimination network.

[0009] Optionally, the training method of the image enhancement model comprises:

[0010] obtaining a training set; the training set comprises training images and labeled data; the training images comprise training blurred images and training clear images, the training blurred images are images obtained by adding noise to the training clear images; the labeled data comprises labeled outline positions;

[0011] obtaining a training blurred outline position by a contour structure based on the training blurred image;

[0012] obtaining a training generated image by an enhanced image generation network based on the training blurred outline position and the training blurred image;

[0013] inputting the training generated image and the training clear image into an enhanced image discrimination network to obtain a generated image discrimination value;

[0014] obtaining an outline loss value based on the training blurred outline position and the labeled outline position;

[0015] obtaining a total loss value, the total loss value being a sum of the contour loss value and the generated image discrimination value;

[0016] obtaining a current training iteration number of the image enhancement model and a preset maximum iteration number of the image enhancement model training;

[0017] stopping the training when the total loss value is less than or equal to a threshold value or the training iteration number reaches the maximum iteration number, to obtain the trained image enhancement model.

[0018] Optionally, the training fuzzy contour position based on the fuzzy image includes:

[0019] obtaining a grayscale fuzzy image based on the fuzzy image;

[0020] obtaining a contrast-enhanced fuzzy image based on the fuzzy image through gamma transformation;

[0021] obtaining a grayscale increase value; the grayscale increase value is used to increase the grayscale value in the grayscale fuzzy image;

[0022] obtaining a grayscale fuzzy increase matrix; the value in the grayscale fuzzy increase matrix is the product of the grayscale increase value and the value in the contrast-enhanced fuzzy image;

[0023] obtaining a training fuzzy contour image based on the grayscale fuzzy increase matrix using Fourier transform.

[0024] Optionally, the training generated image based on the training fuzzy contour position and the training fuzzy image through the enhanced image generation network includes:

[0025] obtaining a plurality of fuzzy regions based on the training fuzzy contour position and the training fuzzy image; the fuzzy region is a region obtained by dividing the fuzzy image according to the training fuzzy contour image;

[0026] obtaining a complete region enhanced image based on the plurality of fuzzy regions and the training fuzzy image through the enhanced image generation network;

[0027] obtaining an overall enhanced image based on the training fuzzy image through the enhanced image generation network;

[0028] obtaining a training generated image based on the overall enhanced image and the complete region enhanced image.

[0029] Optionally, the complete region enhanced image based on the plurality of fuzzy regions and the training fuzzy image through the enhanced image generation network includes:

[0030] Based on the plurality of blur regions and the training blur image, a plurality of blur region images is obtained;

[0031] Based on the blur region image, feature extraction is performed through a blur convolution network to obtain a region blur feature map;

[0032] Based on the region blur feature map, enhancement is performed through an enhanced image generation network to obtain a region enhanced image;

[0033] By inputting a plurality of blur region images into the blur convolution network multiple times, a plurality of region blur feature maps is obtained; and the plurality of region blur feature maps is enhanced through the enhanced image generation network to obtain a plurality of region enhanced images;

[0034] Based on the plurality of region enhanced images, a complete region enhanced image is obtained.

[0035] Optionally, the obtaining of the plurality of blur region images based on the plurality of blur regions and the training blur image comprises:

[0036] Based on the blur region, a region difference point position is obtained; the region difference point comprises a region left lower point position and a region right upper point position;

[0037] Based on the region difference point position, a region size is obtained; the region size comprises a region width and a region height; the region width is a difference between a horizontal coordinate of the region right upper point position and a horizontal coordinate of the region left lower point position; and the region height is a difference between a vertical coordinate of the region right upper point position and a vertical coordinate of the region left lower point position;

[0038] Based on the region size, a zeroed three-dimensional tensor is obtained;

[0039] Based on the blur region, the training blur image and the zeroed three-dimensional tensor, a blur region image is obtained;

[0040] The plurality of blur region images is obtained by multiple times of calculation on the plurality of blur regions.

[0041] Optionally, the enhancement of the region enhanced image based on the region blur feature map through the enhanced image generation network comprises:

[0042] Based on the region blur feature map, a first region blur up-convolution image is obtained through a first up-convolution enhancement network;

[0043] Based on the first region blur up-convolution image, a first region enhanced image is obtained through a second up-convolution enhancement network;

[0044] The first region enhanced image is input into an interactive convolution network to obtain an interactive feature image;

[0045] Based on the region blur feature map and the interaction feature image, a first new region blur up-convolution image is obtained through a first up-convolution enhancement network.

[0046] Based on the first new region blur up-convolution image, a region enhancement image is obtained through a second up-convolution enhancement network.

[0047] Optionally, the obtaining of the training generation image based on the overall enhancement image and the complete region enhancement image comprises:

[0048] The overall enhancement image is obtained to obtain an R-layer overall enhancement matrix.

[0049] Based on the complete region enhancement image, an R-layer complete region enhancement matrix is obtained.

[0050] Based on the R-layer overall enhancement matrix and the R-layer complete region enhancement matrix, an R-layer median enhancement matrix is obtained.

[0051] The training generation image is obtained by calculating the median value of RGB in the overall enhancement image and the complete region enhancement image multiple times.

[0052] In a second aspect, an embodiment of the present application provides a photographic image enhancement system, comprising:

[0053] An acquisition module is configured to obtain a blur image; the blur image is an image that needs to be enhanced;

[0054] A contour detection module is configured to obtain a training blur contour position of the blur image through a contour structure.

[0055] A clear image generation module is configured to obtain a clear image based on the blur image and the training blur contour image through an enhancement image generation network.

[0056] The image enhancement model comprises a contour structure, an enhancement image generation network and an enhancement image discrimination network.

[0057] Optionally, the training method of the image enhancement model comprises:

[0058] A training set is obtained; the training set comprises training images and labeled data; the training images comprise training blur images and training clear images; the training blur images are images obtained by adding noise to the training clear images; and the labeled data comprises labeled contour positions.

[0059] The training blur contour position is obtained based on the training blur image through the contour structure.

[0060] The training generation image is obtained based on the training blur contour position and the training blur image through the enhancement image generation network.

[0061] input the training generated image and the training clear image into an enhanced image discrimination network to obtain a generated image discrimination value;

[0062] obtain a contour loss value based on the training fuzzy contour position and the labeled contour position;

[0063] obtain a total loss value, which is a sum of the contour loss value and the generated image discrimination value;

[0064] obtain a current training iteration number of the image enhancement model and a preset maximum iteration number of the image enhancement model training;

[0065] stop the training when the total loss value is less than or equal to a threshold value or the training iteration number reaches the maximum iteration number, to obtain a trained image enhancement model.

[0066] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0067] The embodiment of the present application also provides a photographic image enhancement method and system, the method comprising: obtaining a fuzzy image. The fuzzy image is an image that needs to be enhanced. The fuzzy image is input into a contour structure to obtain a training fuzzy contour position. Based on the fuzzy image and the training fuzzy contour image, a clear image is obtained through an enhanced image generation network. The image enhancement model comprises a contour structure, an enhanced image generation network and an enhanced image discrimination network.

[0068] The contour is used as a reference to obtain a plurality of regions, and the regions are respectively processed to be clear. Finally, the regions are merged, and the clear image after the clear processing of the complete image is obtained together, so that the contour of the saved image is not lost in the clear processing process. The contour of the fuzzy image is obtained through the contour structure. The contour structure is used for gamma transformation, and then multiplied by the gray value increase value to increase the difference between the values, so that the regions can be more accurately distinguished. The Fourier transform method is used to retain the high frequency to obtain the contour. The obtained contour is subjected to loss calculation, so as to train the gray value increase value, so that the contour can be better extracted. The image is divided into a plurality of regions through the contour, each region is subjected to clear training, and the whole fuzzy image is also subjected to clear training, so that a clearer image is obtained by combining the two. When the regions are operated separately, the regions are taken out to establish a matrix for operation, and the values in the original regions are set to 0. A clear region image of the separate region is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 is a photographic image enhancement method flowchart provided by the embodiment of the present application.

[0070] Figure 2This is a schematic diagram of the structure of an enhanced image generation network in a photographic image enhancement system provided by an embodiment of the present invention.

[0071] Figure 3 This is a block structure diagram of an electronic device provided in an embodiment of the present invention.

[0072] The diagram shows: Bus 500; Receiver 501; Processor 502; Transmitter 503; Memory 504; Bus Interface 505. Detailed Implementation

[0073] The present invention will now be described in detail with reference to the accompanying drawings.

[0074] Example 1

[0075] like Figure 1 As shown, this embodiment of the invention provides a photographic image enhancement method, the method comprising:

[0076] S101: Obtain a blurred image; the blurred image is an image that needs to be enhanced.

[0077] S102: Obtain the training blurred contour position from the blurred image through the contour structure.

[0078] S103: Based on the blurred image and the trained blurred contour image, a clear image is obtained by enhancing the generative network.

[0079] The image enhancement model includes a contour structure, an enhanced image generation network, and an enhanced image discrimination network.

[0080] Optional training methods for image augmentation models:

[0081] Obtain a training set; the training set includes training images and labeled data; the training images include training blurred images and training sharp images, the training blurred images being images of the desired training sharp images with noise added; the labeled data includes labeled contour positions.

[0082] Based on the training blurred image, the position of the training blurred contour is obtained through the contour structure.

[0083] Based on the training blurred contour positions and the training blurred image, a training generated image is obtained by enhancing the image generation network;

[0084] The generated training image and the clear training image are input into the enhanced image discrimination network to obtain the discrimination value of the generated image.

[0085] Based on the positions of the trained fuzzy contours and the labeled contours, the contour loss value is obtained.

[0086] obtaining a total loss value, the total loss value being a sum of the contour loss value and the generated image discrimination value.

[0087] obtaining a current training iteration number of the image enhancement model and a preset maximum iteration number of the image enhancement model training.

[0088] In the embodiment, the preset maximum iteration number of the image enhancement model training is 1200.

[0089] stopping the training when the total loss value is less than or equal to a threshold value or the training iteration number reaches the maximum iteration number, to obtain the trained image enhancement model.

[0090] Optionally, the training fuzzy contour position is obtained based on the fuzzy image through a contour structure, including:

[0091] obtaining a grayscale fuzzy image based on the fuzzy image.

[0092] In the embodiment, a weight brightness conversion method 0.21*r+0.72*g+0.07*b is used to convert the fuzzy image into the grayscale fuzzy image.

[0093] obtaining a contrast-enhanced fuzzy image based on the fuzzy image through gamma transformation.

[0094] In the embodiment, the gamma transformation algorithm is output_image=c*pow(input_image,gamma).

[0095] In the embodiment, input_image is an input image, output_image is an output image, c is a gamma transformation hyperparameter, and gamma is a gamma value, which is set to 0.32.

[0096] obtaining a grayscale increase value, the grayscale increase value being used to increase the grayscale value in the grayscale fuzzy image.

[0097] obtaining a grayscale fuzzy increase matrix, the value in the grayscale fuzzy increase matrix being a product of the grayscale increase value and the value in the contrast-enhanced fuzzy image.

[0098] obtaining the training fuzzy contour position based on the grayscale fuzzy image.

[0099] obtaining a training fuzzy contour image based on the grayscale fuzzy increase matrix using Fourier transform.

[0100] In the embodiment, the Fourier transform is used to only retain high frequencies, which can enhance the image details, thereby obtaining the training fuzzy contour image.

[0101] Through the above method, the purpose of image contour extraction is to find the position with large gray scale difference in the image. The method of using gray scale increase value is to obtain more information. Using Fourier transform to retain only high frequency can enhance the details of the image, thereby obtaining a training fuzzy contour image. The loss of the obtained contour is calculated to train the gray scale increase value, so that the contour can be extracted better.

[0102] Optionally, based on the training fuzzy contour position and the training fuzzy image, a training generated image is obtained through an enhanced image generation network, including:

[0103] Based on the training fuzzy contour position and the training fuzzy image, a plurality of fuzzy regions are obtained; the fuzzy region is divided into a plurality of regions according to the training fuzzy contour image.

[0104] Among them, the plurality of fuzzy regions are labeled fuzzy regions, and in this embodiment, the labeling is performed from left to right and from bottom to top, and the first fuzzy region is labeled as 1.

[0105] Based on the plurality of fuzzy regions and the training fuzzy image, a complete region enhanced image is obtained through an enhanced image generation network.

[0106] Among them, the plurality of region enhanced images are filled with values according to the labeled region positions to obtain a complete region enhanced image.

[0107] Based on the training fuzzy image, an overall enhanced image is obtained through an enhanced image generation network.

[0108] Among them, the method of obtaining the overall enhanced image through the enhanced image generation network from the training fuzzy image is similar to the method of obtaining the region enhanced image through the enhanced image generation network based on the region fuzzy feature map.

[0109] Based on the overall enhanced image and the complete region enhanced image, a training generated image is obtained.

[0110] Through the above method, the image is divided into different blocks by the contour, which can well maintain the position information of the contour, so that the contour does not change in the up convolution, and the obtained clear image is more accurate. And through the overall enhanced image and the complete region enhanced image, accurate information can be obtained in the training process.

[0111] Optionally, based on the plurality of fuzzy regions and the training fuzzy image, a complete region enhanced image is obtained through an enhanced image generation network, including:

[0112] Based on the plurality of fuzzy regions, a plurality of fuzzy region images are obtained;

[0113] Based on the fuzzy region image, feature extraction is performed through a fuzzy convolution network to obtain a region fuzzy feature map.

[0114] Based on the region fuzzy feature map, enhancement is performed through an enhanced image generation network to obtain a region enhanced image.

[0115] By inputting multiple fuzzy region images into the fuzzy convolution network multiple times, multiple region fuzzy feature maps are obtained. The multiple region fuzzy feature maps are enhanced through the enhanced image generation network to obtain multiple region enhanced images.

[0116] Based on the multiple region enhanced images, a complete region enhanced image is obtained.

[0117] Wherein, a fixed size three-dimensional tensor obtained according to network design is taken as the size of the complete region enhanced image. The multiple region enhanced images are filled into the complete region enhanced image according to the labeled region position, and the value of the region enhanced image is obtained.

[0118] Through the above method, the image is divided into multiple regions by contour, each region is trained to be clear, and the whole fuzzy image is also trained to be clear, so that a clearer image is obtained by combining the two.

[0119] Optionally, the multiple fuzzy region images are obtained based on the multiple fuzzy regions and the training fuzzy image, comprising:

[0120] Based on the fuzzy region, a region difference point position is obtained; the region difference point includes a region left lower point position and a region right upper point position.

[0121] Based on the region difference point position, a region size is obtained; the region size includes a region width and a region height; the region width is the difference between the horizontal coordinate of the region right upper point position and the horizontal coordinate of the region left lower point position; the region height is the difference between the vertical coordinate of the region right upper point position and the vertical coordinate of the region left lower point position.

[0122] Based on the region size, a zeroed three-dimensional tensor is obtained.

[0123] Wherein, the size of the zeroed three-dimensional vector is (a, b, 3), a represents the region width in the region size; b represents the region height in the region size; and 3 represents RGB three channels.

[0124] Based on the fuzzy region, the training fuzzy image and the zeroed three-dimensional tensor, a fuzzy region image is obtained.

[0125] By calculating multiple fuzzy regions multiple times, multiple fuzzy region images are obtained.

[0126] Through the above method, the region is taken out alone to establish a matrix for operation, and the values not in the original region are set to 0. A clear region image of the single region is obtained.

[0127] Optionally, based on the region fuzzy feature map, an enhanced image is obtained through an enhanced image generation network.

[0128] Based on the region fuzzy feature map, a first region fuzzy up-convolution image is obtained through a first up-convolution enhancement network.

[0129] Based on the first region fuzzy up-convolution image, a first region enhanced image is obtained through a second up-convolution enhancement network.

[0130] The first region enhanced image is input into an interactive convolution network to obtain an interactive feature image.

[0131] Based on the region fuzzy feature map and the interactive feature image, a first new region fuzzy up-convolution image is obtained through the first up-convolution enhancement network.

[0132] Based on the first new region fuzzy up-convolution image, a region enhanced image is obtained through the second up-convolution enhancement network.

[0133] In the process of obtaining the training generated image, the first region fuzzy up-convolution image, the region enhanced image and the overall enhanced image are obtained, and the same is judged in the training process.

[0134] The image is as shown in the image. Figure 2

[0135] Through the above method, when the convolution network is constructed, the clear image obtained through the first up-convolution is input into the interactive network, and the second up-convolution is performed together with the fuzzy image to obtain an accurate image.

[0136] Optionally, based on the overall enhanced image and the complete region enhanced image, a training generated image is obtained, including:

[0137] The overall enhanced image is obtained to obtain an R-layer overall enhanced matrix.

[0138] Based on the complete region enhanced image, an R-layer complete region enhanced matrix is obtained.

[0139] Based on the R-layer overall enhanced matrix and the R-layer complete region enhanced matrix, an R-layer median enhanced matrix is obtained.

[0140] Through multiple calculations of the median values of RGB in the overall enhanced image and the complete region enhanced image, a training generated image is obtained.

[0141] ​Through the above method, each region is trained to become clear, and the whole fuzzy image is also trained to be clear, so that a clearer image is obtained by combining the two.

[0142] Through the above method, the fuzzy image is converted into a clear image by using the trained image enhancement model. The image enhancement uses a method of generating network and discriminant network to train together to obtain an image enhancement model capable of converting the fuzzy image into a clear image. The contour structure is used to obtain the contour of the fuzzy image, and the contour structure is then subjected to gamma transformation, and then multiplied by the gray value increase value to increase the difference between the numerical values, so that the contour can be more accurately distinguished. The Fourier transform method is used to retain the high frequency to obtain the contour. The contour is used to divide the image into multiple regions, and each region is trained to become clear, and the whole fuzzy image is also trained to be clear, so that a clearer image is obtained by combining the two. When the region is operated alone, the region is taken out to establish a matrix for operation, and the values outside the original region are set to 0. A clear region image of the single region is obtained. When the convolution network is constructed, the clear image obtained by the first up-convolution is subjected to the second up-convolution together with the fuzzy image through the interaction network to obtain an accurate image.

[0143] Embodiment 2

[0144] Based on the above-mentioned image enhancement method, the embodiment of the present application further provides an image enhancement system, which comprises an acquisition module, a contour detection module and a clear image generation module.

[0145] The acquisition module is used to obtain a fuzzy image; the fuzzy image is an image that needs to be enhanced.

[0146] The contour detection module is used to obtain the training fuzzy contour position of the fuzzy image through the contour structure.

[0147] The clear image generation module is used to obtain a clear image through the enhancement image generation network based on the fuzzy image and the training fuzzy contour image.

[0148] The image enhancement model comprises a contour structure, an enhancement image generation network and an enhancement image discriminant network.

[0149] Optionally, the training method of the image enhancement model comprises the following steps:

[0150] A training set is obtained; the training set comprises training images and labeled data; the training images comprise training fuzzy images and training clear images, the training fuzzy images are images obtained by adding noise to the training clear images; and the labeled data comprises labeled contour positions.

[0151] Based on the training fuzzy image, a training fuzzy contour position is obtained through a contour structure.

[0152] Based on the training fuzzy contour position and the training fuzzy image, a training generated image is obtained through an enhanced image generation network.

[0153] The training generated image and the training clear image are input into an enhanced image discrimination network to obtain a generated image discrimination value.

[0154] Based on the training fuzzy contour position and a labeled contour position, a contour loss value is obtained.

[0155] A total loss value is obtained, which is the sum of the contour loss value and the generated image discrimination value.

[0156] A current training iteration number of the image enhancement model and a maximum iteration number of the image enhancement model training which is set in advance are obtained.

[0157] When the total loss value is less than or equal to a threshold value or the training iteration number reaches the maximum iteration number, the training is stopped to obtain a trained image enhancement model.

[0158] Here, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0159] The embodiment of the present application also provides an electronic device, such as Figure 3 As shown in the figure, the electronic device comprises a memory 504, a processor 502, and a computer program stored in the memory 504 and capable of running on the processor 502, and the processor 502 implements the steps of any method of the photograph image enhancement method described above when executing the program.

[0160] Wherein, in Figure 3 the bus architecture (represented by bus 500), the bus 500 can include any number of interconnected buses and bridges, which link various circuits including one or more processors represented by the processor 502 and the memory represented by the memory 504. The bus 500 can also link various other circuits such as peripheral devices, voltage stabilizers and power management circuits, which are well known in the art, and therefore, they will not be described further herein. The bus interface 505 provides an interface between the bus 500 and the receiver 501 and the transmitter 503. The receiver 501 and the transmitter 503 can be the same element, i.e. a transceiver, which provides a unit for communicating with various other devices on a transmission medium. The processor 502 is responsible for managing the bus 500 and general processing, while the memory 504 can be used to store data used by the processor 502 in performing operations.

[0161] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above for photographic image enhancement and the data involved therein.

[0162] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0163] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0164] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0165] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0166] Furthermore, those skilled in the art will recognize that, while certain embodiments described herein include certain features, not all embodiments need necessarily include those certain features. To the contrarv, features of different embodiments can be combined to create embodiments that are not necessarily included in the summary above. For example, in the following claims, any of the claimed embodiments can be used in any combination. Accordingly, other implementations are within the scope of the following claims.

[0167] Various component embodiments of the present application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. As will be appreciated by those skilled in the art, a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some or all of the components in an apparatus according to embodiments of the present application. The present application can also be implemented as a program of instructions for performing part or all of the methods described herein, e.g., a computer program and a computer program product. Such program of instructions of the present application can be stored on a computer readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0168] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unitary claim, several of the devices mentioned in the groups of devices can be embodied by one and the same thing or element. The usage of the words 'first','second', and 'third', etc. do not limit the scope of the independent claims, but they are used to distinguish between different features. Furthermore, the foregoing description of embodiments of the application is not intended to be exhaustive or to limit the application to the precise form disclosed. Changes, modifications and variations are possible in the light of the above teachings. The order of steps recited as part of the disclosed examples can depend on the implementation to achieve the described results. The sole focus of the claims is to cover all embodiments falling within the scope of the claims and their equivalents.

Claims

1. A photographic image enhancement method characterized by, include: Obtain a blurred image; The blurred image is the image that needs to be enhanced; The blurred image is analyzed using a contour structure to obtain the position of the blurred contour. Based on the blurred image and the location of the blurred contour, a clear image is obtained by using an enhanced image generation network; Image enhancement models include contour structures, enhanced image generation networks, and enhanced image discrimination networks; Training methods for image enhancement models: Obtain a training set; the training set includes training images and labeled data; the training images include training blurred images and training sharp images, the training blurred images being images of the desired training sharp images with noise added; The annotation data includes the location of the annotation outline; Based on the training blurred image, the position of the training blurred contour is obtained through the contour structure; Based on the training blurred contour positions and the training blurred image, a training generated image is obtained by enhancing the image generation network; The generated training image and the clear training image are input into the enhanced image discrimination network to obtain the discrimination value of the generated image; Based on the trained fuzzy contour positions and the labeled contour positions, the contour loss value is obtained; Obtain the total loss value, which is the sum of the contour loss value and the generated image discrimination value; Obtain the current number of training iterations of the image enhancement model and the preset maximum number of training iterations of the image enhancement model; Training stops when the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, and a trained image enhancement model is obtained.

2. The photographic image enhancement method of claim 1, wherein, The step of obtaining the training blurred contour position based on the training blurred image and through the contour structure includes: Based on the trained blurred image, a grayscale blurred image is obtained; Based on the trained blurred image, a contrast-enhanced blurred image is obtained through gamma transformation; Obtain a grayscale increase value; the grayscale increase value is used to increase the grayscale value in a grayscale blurred image; Obtain a grayscale blur enhancement matrix; the values ​​in the grayscale blur enhancement matrix are the product of the grayscale enhancement value and the value in the contrast-enhanced blurred image; Based on the grayscale blur augmentation matrix, the position of the training blur contour is obtained using Fourier transform.

3. The photographic image enhancement method of claim 1, wherein, The step of obtaining a trained generated image by enhancing the image generation network based on the trained blurred contour position and the trained blurred image includes: Based on the training blurred contour position and the training blurred image, multiple blurred regions are obtained; the blurred regions are obtained by dividing the blurred image into multiple regions according to the training blurred contour image. Based on the multiple blurred regions and the trained blurred image, an enhanced image of the complete region is obtained through an enhanced image generation network; Based on the trained blurred image, an overall enhanced image is obtained through an enhanced image generation network; Based on the overall enhanced image and the complete region enhanced image, the trained generated image is obtained.

4. The photographic image enhancement method of claim 3, wherein, The process of obtaining a complete region-enhanced image through an image enhancement generation network based on the multiple blurred regions and the trained blurred image includes: Based on the multiple blurred regions and the training blurred image, multiple blurred region images are obtained; Based on the blurred region image, feature extraction is performed through a blurred convolutional network to obtain a blurred feature map of the region; Based on the region blur feature map, an enhanced image is generated by an enhanced image generation network to obtain a region enhanced image; A plurality of region blur feature maps are obtained by inputting a plurality of blur region images into the blur convolution network multiple times; the plurality of region blur feature maps are enhanced by the enhanced image generation network to obtain a plurality of region enhanced images; Based on the plurality of region enhanced images, a complete region enhanced image is obtained.

5. The photographic image enhancement method of claim 4, wherein, The plurality of blur region images are obtained based on the plurality of blur regions and the training blur image, including: Based on the blur region, a region difference point position is obtained; the region difference point includes a region left lower point position and a region right upper point position; Based on the region difference point position, a region size is obtained; the region size includes a region width and a region height; the region width is the difference between the horizontal coordinate of the region right upper point position and the horizontal coordinate of the region left lower point position; the region height is the difference between the vertical coordinate of the region right upper point position and the vertical coordinate of the region left lower point position; Based on the region size, a zeroed three-dimensional tensor is obtained; Based on the blur region, the training blur image and the zeroed three-dimensional tensor, a blur region image is obtained; A plurality of blur region images are obtained by calculating a plurality of blur regions multiple times.

6. The photographic image enhancement method of claim 4, wherein, The region enhanced image is obtained based on the region blur feature map by enhancing the image generation network, including: Based on the region blur feature map, a first region blur up-convolution image is obtained by a first up-convolution enhancement network; Based on the first region blur up-convolution image, a first region enhanced image is obtained by a second up-convolution enhancement network; The first region enhanced image is input into an interactive convolution network to obtain an interactive feature image; Based on the region blur feature map and the interactive feature image, a first new region blur up-convolution image is obtained by the first up-convolution enhancement network; Based on the first new region blur up-convolution image, a region enhanced image is obtained by the second up-convolution enhancement network.

7. The photographic image enhancement method of claim 3, wherein, The training generated image is obtained based on the overall enhanced image and the complete region enhanced image, including: The overall enhanced image is obtained to obtain an R-layer overall enhanced matrix; Based on the complete region enhanced image, an R-layer complete region enhanced matrix is obtained; Based on the R-layer overall enhanced matrix and the R-layer complete region enhanced matrix, an R-layer median enhanced matrix is obtained; The training generated image is obtained by calculating the intermediate value of RGB in the overall enhanced image and the complete region enhanced image multiple times.

8. A photographic image enhancement system characterized by, It includes: An acquisition module: obtaining a blur image; The blur image is an image that needs to be enhanced; An outline detection module: obtaining a blur outline position by a contour structure based on the blur image; A clear image generation module: obtaining a clear image by an enhanced image generation network based on the blur image and the blur outline position; The image enhancement model includes a contour structure, an enhanced image generation network and an enhanced image discrimination network; The training method of the image enhancement model: A training set is obtained; the training set includes training images and labeled data; the training images include training blur images and training clear images, and the training blur images are images obtained by adding noise to the training clear images required to be trained; The labeled data comprises a labeled contour position; Based on the training fuzzy image, a training fuzzy contour position is obtained through a contour structure; Based on the training fuzzy contour position and the training fuzzy image, a training generated image is obtained through an enhanced image generation network; The training generated image and the training clear image are input into an enhanced image discrimination network to obtain a generated image discrimination value; Based on the training fuzzy contour position and the labeled contour position, a contour loss value is obtained; A total loss value is obtained, which is the sum of the contour loss value and the generated image discrimination value; A current training iteration number of the image enhancement model and a maximum iteration number of the image enhancement model training which is set in advance are obtained; When the total loss value is less than or equal to a threshold value or the training iteration number reaches the maximum iteration number, the training is stopped, and a trained image enhancement model is obtained.

Images