Image enhancement method and its device, equipment, medium and product

Abstract

The present application discloses an image enhancement method, device, computer equipment and storage medium, including: obtaining a second target image converted from a first target image, wherein the information carrying capacity of the second target image is greater than that of the first target image; performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate multiple enhanced images; based on a preset weight calculation formula, respectively calculating the gradient weight and exposure weight corresponding to the second target image and each enhanced image; based on the gradient weight and exposure weight, performing image synthesis processing on the second target image and each enhanced image to generate a synthetic image; performing image fusion on the synthetic image according to a preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image. The second target image is subjected to three distortion repairs to improve the image quality of the second target image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image enhancement method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the development and progress of science and technology, the widespread popularization of smart phones and the rapid development of the Internet, massive amounts of image data are constantly being generated and shared. At the same time, people's requirements for image quality are constantly increasing, and more people are pursuing higher-definition and more colorful display data.

[0003] The applicant of the present invention has found in the research that in the prior art, as a shooting device commonly used by users, mobile terminals can only shoot some pictures with low information load due to the limitation of hardware performance, and then synthesize the pictures with low information load by format conversion to obtain pictures with high information load. However, the images obtained after format conversion often have a large degree of distortion and poor image quality.

[0004] Application Contents

[0005] The present application provides an image enhancement method, device, electronic device and computer-readable storage medium capable of performing image enhancement on a converted image.

[0006] To achieve the above object, the present application provides an image enhancement method, comprising:

[0007] Acquire a second target image converted from the first target image, wherein the information carrying capacity of the second target image is greater than the information carrying capacity of the first target image;

[0008] Performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images;

[0009] Based on a preset weight calculation formula, respectively calculate the gradient weight and exposure weight corresponding to the second target image and each enhanced image;

[0010] Based on the gradient weight and the exposure weight, performing image synthesis processing on the second target image and each of the enhanced images to generate a synthesized image;

[0011] The composite image is fused according to a preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image.

[0012] Optionally, performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images includes:

[0013] Reading the grayscale value of the second target image;

[0014] Performing gamma correction processing on the grayscale value according to a preset gamma mapping algorithm to generate a first enhanced image corresponding to the second target image;

[0015] The grayscale values ​​are mapped according to a preset nonlinear algorithm, and a second enhanced image corresponding to the second target image is generated according to a result of the mapping.

[0016] Optionally, after performing value mapping on the grayscale value according to a preset nonlinear algorithm and generating a second enhanced image corresponding to the second target image according to a result of the value mapping, the method further comprises:

[0017] performing linear compensation processing on the second enhanced image according to a preset linear algorithm;

[0018] The second enhanced image after the linear compensation processing is subjected to image cropping processing, so that the tone value of the second enhanced image is mapped within a preset value range, and a third enhanced image corresponding to the second target image is generated.

[0019] Optionally, performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images includes:

[0020] Reading the grayscale value of the second target image;

[0021] The grayscale value is logarithmically mapped according to a preset logarithmic mapping algorithm to generate a fourth enhanced image corresponding to the second target image.

[0022] Optionally, the step of respectively calculating the gradient weight and exposure weight corresponding to the second target image and each enhanced image based on a preset weight calculation formula includes:

[0023] Reading grayscale images of the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image;

[0024] A first directional gradient and a second directional gradient of each of the grayscale images are subjected to a preset Sobel operator, and a gradient weight of each of the grayscale images is generated according to the first directional gradient and the second directional gradient;

[0025] According to a preset exposure calculation formula, three-channel exposure values ​​of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are calculated, and exposure weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are generated according to the three-channel exposure values.

[0026] Optionally, performing image synthesis processing on the second target image and each of the enhanced images to generate a synthetic image based on the gradient weight and the exposure weight includes:

[0027] Calculating image weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image according to the gradient weight and the exposure weight;

[0028] The second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are fused according to the image weight to generate the composite image.

[0029] Optionally, the performing image fusion on the synthesized image according to a preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image includes:

[0030] Inputting the synthesized image into the Gaussian pyramid for sampling processing to generate a plurality of first sampling images;

[0031] Inputting the top layer image in the Gaussian pyramid into the Laplacian pyramid for sampling processing to generate a plurality of second sampling images, wherein the plurality of first sampling images correspond to the second sampling images one by one;

[0032] Performing weighted operations on the plurality of first sampling images and the corresponding plurality of second sampling images respectively to generate a plurality of weighted images;

[0033] The plurality of weighted images are sequentially fused along the reverse direction of the Laplacian pyramid to generate the enhanced image.

[0034] To achieve the above object, the present application also provides an image enhancement device, comprising:

[0035] An acquisition module, used for acquiring a second target image converted from the first target image, wherein the information carrying capacity of the second target image is greater than the information carrying capacity of the first target image;

[0036] A processing module, used for performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images;

[0037] A calculation module, used to calculate the gradient weight and exposure weight corresponding to the second target image and each enhanced image respectively based on a preset weight calculation formula;

[0038] A synthesis module, used for performing image synthesis processing on the second target image and each of the enhanced images to generate a synthesized image based on the gradient weight and the exposure weight;

[0039] The execution module is used to perform image fusion on the synthetic image according to the preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image.

[0040] Optionally, the image enhancement device further includes:

[0041] A first reading submodule, used for reading the grayscale value of the second target image;

[0042] A first processing submodule, configured to perform gamma correction processing on the grayscale value according to a preset gamma mapping algorithm to generate a first enhanced image corresponding to the second target image;

[0043] The second processing submodule is used to perform value mapping on the grayscale value according to a preset nonlinear algorithm, and generate a second enhanced image corresponding to the second target image according to a result of the value mapping.

[0044] Optionally, the image enhancement device further includes:

[0045] A third processing submodule, configured to perform linear compensation processing on the second enhanced image according to a preset linear algorithm;

[0046] The first execution submodule is used to perform image cropping processing on the second enhanced image after the linear compensation processing, so that the tone value of the second enhanced image is mapped within a preset value range, and generate a third enhanced image corresponding to the second target image.

[0047] Optionally, the image enhancement device further includes:

[0048] A second reading submodule, used for reading the grayscale value of the second target image;

[0049] The fourth processing submodule is used to perform logarithmic mapping processing on the grayscale value according to a preset logarithmic mapping algorithm to generate a fourth enhanced image corresponding to the second target image.

[0050] Optionally, the image enhancement device further includes:

[0051] A third reading submodule, used for reading the grayscale images of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image;

[0052] a fifth processing submodule, configured to perform a first directional gradient and a second directional gradient on each of the grayscale images by using a preset Sobel operator, and generate a gradient weight of each of the grayscale images according to the first directional gradient and the second directional gradient;

[0053] The second execution submodule is used to calculate the three-channel exposure values ​​of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to a preset exposure calculation formula, and generate the exposure weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to the three-channel exposure values.

[0054] Optionally, the image enhancement device further includes:

[0055] A first calculation submodule, used for calculating the image weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to the gradient weight and the exposure weight;

[0056] The first synthesis submodule is used to perform image fusion on the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to the image weight to generate the synthesized image.

[0057] Optionally, the image enhancement device further includes:

[0058] A first sampling submodule, used for inputting the synthetic image into the Gaussian pyramid for sampling processing to generate a plurality of first sampling images;

[0059] A second sampling submodule is used to input the top layer image in the Gaussian pyramid into the Laplacian pyramid for sampling processing to generate a plurality of second sampling images, wherein the plurality of first sampling images and the second sampling images correspond one to one;

[0060] A second calculation submodule, used for performing weighted operations on the plurality of first sampled images and the corresponding plurality of second sampled images respectively to generate a plurality of weighted images;

[0061] The third execution submodule is used to perform image fusion on the multiple weighted images in sequence along the reverse direction of the Laplacian pyramid to generate the enhanced image.

[0062] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the above-mentioned image enhancement method.

[0063] In order to solve the above technical problems, an embodiment of the present application further provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the above-mentioned image enhancement method.

[0064] A computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the image enhancement method described in any embodiment of the present application.

[0065] The beneficial effect of the embodiment of the present application is: by performing image enhancement processing on the second target image obtained by converting the first target image, the image distortion caused by format conversion in the second target image is enhanced from different dimensions. Then, by calculating the gradient weight and exposure weight of each enhanced image, and calculating the fusion weight of each enhanced image from the gradient weight and exposure weight, multiple enhanced images are synthesized to generate a synthetic image with the fusion weight, and the image distortion of the second target image is further repaired. Finally, the synthesized image is fused for multiple frames through the Gaussian pyramid and the Laplacian pyramid, and the second target image is repaired for the third time, thereby improving the image quality of the second target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0067] Figure 1 A schematic diagram of the basic flow of an image enhancement method according to a specific embodiment of the present application;

[0068] Figure 2 A schematic diagram of a first process of generating an enhanced image according to a specific embodiment of the present application;

[0069] Figure 3 A schematic diagram of a second process of generating an enhanced image according to a specific embodiment of the present application;

[0070] Figure 4 A third flow chart of generating an enhanced image according to a specific embodiment of the present application;

[0071] Figure 5 A schematic diagram of a flow chart of generating gradient weights and exposure weights for each image according to a specific embodiment of the present application;

[0072] Figure 6 A schematic diagram of a process for generating a composite image according to a specific embodiment of the present application;

[0073] Figure 7 A schematic diagram of a process of generating an imposed image according to an embodiment of the present application;

[0074] Figure 8 A schematic diagram of the basic structure of an image enhancement device according to an embodiment of the present application;

[0075] Fig. 9This is a basic structural block diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0077] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0078] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0079] It will be understood by those skilled in the art that the term "terminal" as used herein includes both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of performing two-way communications over a two-way communication link. Such devices may include: cellular or other communication devices, which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which may combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / intranet access, a web browser, a notepad, a calendar and / or a GPS (Global Positioning System) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "terminal" used here can be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, running at any other location on the earth and / or in space. The "terminal" used here can also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0080] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0081] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0082] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0083] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0084] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0085] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be executed independently. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same application concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different concept expressions, should be understood as equivalent.

[0086] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0087] See also Figure 1 , Figure 1 Schematic diagram of the basic flow of the image enhancement method of this embodiment.

[0088] like Figure 1 As shown, an image enhancement method comprises:

[0089] S1100, acquiring a second target image converted from a first target image, wherein the information carrying capacity of the second target image is greater than the information carrying capacity of the first target image;

[0090] The second target image is a composite image obtained by performing image synthesis processing on a plurality of first target images.

[0091] In some implementations, the first target image can be converted into the second target image according to the neural network model, and the conversion method is as follows:

[0092] S2100, acquiring a first target image in a first data format;

[0093] In this embodiment, the shooting device used for image shooting is a mobile terminal, such as a mobile phone, a tablet computer or a DV device.

[0094] The mobile terminal needs to capture images in a specific shooting mode. The configuration parameters can be stored on the server side. After receiving the device information sent by the mobile terminal, the server side identifies the operating system or device information of the mobile terminal based on the device information, and then further configures the configuration parameters that match the mobile terminal based on the device information.

[0095] In some embodiments, the matching of configuration parameters needs to be matched according to the camera interface carried by the mobile terminal. First, the API interface of the mobile terminal camera is collected, searched in the configuration database, and after obtaining the parameters corresponding to the API interface, the configuration parameters are sent to the mobile terminal. When the corresponding configuration parameters cannot be found through the API interface, the SDK information of the shooting module in the mobile terminal device information is read, and the SDK information is searched in the configuration database. After obtaining the configuration parameters corresponding to the SDK, the corresponding configuration parameters are sent to the mobile terminal. Furthermore, when the configuration parameters cannot be matched through the SDK information, the configuration parameters can be matched according to the operating system type of the mobile terminal.

[0096] When the server receives the device information sent by the mobile terminal, it matches the configuration parameters corresponding to the mobile terminal according to the device information and sends the configuration parameters to the mobile terminal. After the mobile terminal obtains the configuration parameters, it configures the shooting parameters of the mobile terminal in the following manner: setting the shooting parameters of the mobile terminal.

[0097] The mobile terminal enters the set target shooting mode according to the configuration parameter setting. It should be noted that the target shooting mode is not a necessary mode for the mobile terminal to capture images. In some embodiments, the mobile terminal can capture the first target image in any shooting mode.

[0098] The image captured by the mobile terminal is the first target image, and the first data format is the SDR format. It should be pointed out that the first data format is not limited to the SDR format, but can also be a conventional image format such as JPG, PNG, etc.

[0099] S2200, inputting the first target image into a preset image format conversion model, wherein the image format conversion model is pre-constrained to a convergence state through linear loss and nonlinear loss, and is a neural network model for performing format conversion on an image;

[0100] After the first target image is acquired, the first target image is input into the image format conversion model. The image format conversion model is a neural network model that has been trained to a convergence state through supervised training in advance and can perform format conversion on the image input therein.

[0101] In some implementations, the image format conversion model is deployed in a mobile terminal. After the mobile terminal captures the first target image, the first target image is input into the local storage of the mobile terminal, and the image format conversion model converts the image format locally.

[0102] In some embodiments, the image format conversion model is deployed on the server side. After the mobile terminal acquires the first target image, the mobile terminal sends the first target image to the server side. The server side converts the image format of the first target image through the image format conversion model. After the image format conversion is completed, the server side sends the converted image to the mobile terminal.

[0103] In some embodiments, after the first target image is acquired, the acquired first target image needs to be screened. The screening method is: screening by the average brightness of the first target image. Specifically, a standard brightness interval value is set. The standard brightness interval value can be [30,230], but the standard brightness interval value is not limited thereto. Depending on the specific application scenario, the critical value of the standard brightness interval value can be larger or smaller.

[0104] The average brightness of the first target image is calculated, and the average brightness value corresponding to the first target image is compared to see whether it falls within the standard brightness range. If the average brightness value corresponding to the first target image falls within the standard brightness range, the first target image is input into the image format conversion model; otherwise, the first target image is deleted and the image is retaken.

[0105] The value range of the standard brightness interval value can be dynamically adjusted according to different shooting environments. Specifically, the first target image is input into a neural network model that is pre-trained to a convergence state and is used to recognize the scene in which the image is located, and the neural network model recognizes the shooting environment represented by the first target image, for example, different image scenes such as indoor, outdoor, cloudy, sunny, etc.

[0106] According to the image scene, a standard brightness interval value matching the image scene is searched in a preset scene database. By scene recognition of the first target image, the adaptability of the standard brightness interval value to the environment can be improved.

[0107] In some implementations, the image format conversion model is deployed in a mobile terminal. Since the performance of the mobile terminal is limited and a large-scale neural network model cannot be run, the image format conversion model needs to be lightweight. Specifically, the structure of the image format conversion model is as follows:

[0108] The image format conversion model includes: a first convolution channel and a second convolution channel, and the feature output by the first convolution channel is a mask feature vector, and the feature output by the second convolution channel is a convolution feature vector.

[0109] The first convolution channel includes: a first convolution layer and a first attention layer, the first convolution layer is cascaded with the first attention layer, and the first attention layer is connected to the output end of the first convolution layer, and the first attention layer includes a channel attention layer.

[0110] The second convolution channel includes: a plurality of cascaded feature layers, the output end of each feature layer is connected to a second attention layer, and the second attention layer includes a channel attention layer and a spatial attention layer. In some embodiments, the second convolution channel includes: 3 groups of cascaded feature layers and 3 groups of second attention layers, but the number of feature layers and second attention layers included in the second convolution channel is not limited thereto. Depending on the specific application scenario, in some embodiments, the number of feature layers and second attention layers included in the second convolution channel can be the maximum or less.

[0111] Each of the feature layers includes: a plurality of cascaded second convolutional layers, the output end of each of the second convolutional layers is connected to a linear rectification layer, and the output of any linear rectification layer is used as the input of all the second convolutional layers arranged after any of the linear rectification layers. In some embodiments, each feature layer includes: 5 groups of cascaded second convolutional layers and linear rectification layers, and a linear rectification function is provided in the linear rectification layer. The output of any linear rectification layer is used as the input of all the second convolutional layers arranged after any of the linear rectification layers. For example, the output of the first-ranked linear rectification layer is used as the input of the second, third, fourth and fifth second convolutional layers, and the output of the second-ranked linear rectification layer is used as the input of the third, fourth and fifth second convolutional layers, and so on, until the fourth-ranked linear rectification layer is input to the fifth-ranked second convolutional layer. It should be pointed out that the number of second convolutional layers and linear rectification layers included in the feature layer is not limited thereto. Depending on the specific application scenario, in some embodiments, the number of second convolutional layers and linear rectification layers can be more or less.

[0112] In this implementation, the image format conversion model includes a loss function, and the loss function includes a linear loss function and a nonlinear loss function.

[0113] Loss=Loss linear +Loss nonlinear

[0114] Among them, Loss represents the loss function, Loss linear Expressed as a linear loss function, Loss nonlinear It is expressed as a nonlinear loss function.

[0115] Both linear loss function and nonlinear loss function include color gamut loss function Loss color And the perceptual loss function Loss perceptual . Compared with the existing technology, the loss function is a single nonlinear loss function or a linear loss function. However, a single linear loss function cannot make the image format conversion model converge, and a single nonlinear loss function will put the image format conversion model in a distorted state. By mixing linear loss function and nonlinear loss function, the linear loss function can compensate for the model distortion caused by the nonlinear loss function, making the image format conversion model more robust and the converted image more stable.

[0116] Loss linear =Loss color +Loss perceptual

[0117] Loss nonlinear =Loss color +Loss perceptual

[0118] Perceptual loss refers to the difference in results between different feature layers of the image format conversion model. The difference is calculated using the L1 paradigm, where y is the labeled feature and y′ is the output of the image format conversion model.

[0119] Loss perceptual =L1|VGG 19 (y′-y)|

[0120] Color gamut loss uses three color gamuts for loss calculation, namely RGB, HSV, and LAB. RGB is the three primary color space, corresponding to the three colors of red, green, and blue respectively; HSV is the hue (H), saturation (S), and brightness (V) space, and the color is adjusted through the H and S channels to be closer to the color of the annotated image; in LAB, L is brightness, and AB is red and blue hue. Similar to HSV, the difference between the A and B channels is calculated to improve the accuracy of network color. The calculation of color gamut loss adopts the L1 paradigm, as shown in the following formula:

[0121] Loss color =L1|(y′ rgb -y rgb )|+L1|(y′ hsv -y hsv )|+L1|(y′ lab -y lab )|

[0122] The above-mentioned image format conversion model format is a lightweight model that can be deployed on mobile terminals, and the combination of linear and nonlinear losses reduces image distortion and over-enhancement, and increases the expressiveness and stability of the image output by the image format conversion model.

[0123] S2300. Read a second target image output by the image format conversion model, wherein the second target image format is a second data format, and an information carrying capacity of the second data format is greater than an information carrying capacity of the first data format.

[0124] The first target image is input into the image format conversion model, the input channel of the image format conversion model converts the first target image into an array vector matrix, and then the array vector matrix is ​​respectively input into the first convolution channel, the second convolution channel, and the output ends of the first convolution channel and the second convolution channel.

[0125] Among them, the first convolution layer and the first attention layer in the first convolution channel extract the features of the array vector matrix, and the extracted feature vector is the mask feature vector.

[0126] The feature layer and the second attention layer in the second convolution channel extract the features of the array vector matrix in a hierarchical manner. Among them, each second convolution layer and linear rectification layer in the feature layer extracts the convolution features in the array vector matrix in a hierarchical manner. Finally, the feature vector output by the second convolution channel is the convolution feature vector.

[0127] At the output end of the first convolution channel and the second convolution channel, the mask feature vector output by the first convolution channel is subjected to a dot product operation with the array vector matrix, that is, the eye mask feature matrix represented by the mask feature vector is multiplied with the array vector matrix.

[0128] After the mask feature vector output by the first convolution channel is dot-producted with the array vector matrix to obtain a dot product result, the vector matrix obtained by the dot product result is added to the feature vector output by the second convolution channel.

[0129] After the calculation result is obtained by the addition operation, the calculation result needs to be mapped. The mapping method is: the calculation result is mapped by the hyperbolic tangent function. The hyperbolic tangent function (tanh) is the ratio of the hyperbolic sine function (sinh) to the hyperbolic cosine function (cosh), which can map the calculation result obtained by the addition operation within the value range of [-1,1]. Finally, the mapped vector matrix is ​​pixelated to generate the second target image.

[0130] The data format of the second target image is the second data format. In this embodiment, the second data format is the HDR format. It should be noted that the format range of the second data format is not limited thereto. Depending on the specific application scenario, the second data format can also be (not limited to): TGA, BMP and other image formats.

[0131] The above-mentioned implementation method pre-trains the image format conversion model, which is a neural network model, and can convert the format of the image input therein, thereby increasing the information carrying capacity of the input image and the amount of information in the image. Therefore, when a user uses a shooting device to shoot a first target image in a first data format, the first target image is input into the image format conversion model, and the image format conversion model performs format conversion on the first target image, so that the first target image is converted into a second target image in a second data format. The information carrying capacity of the second data format is greater than the information carrying capacity of the first data format, and the second target image after format conversion has a higher image quality. The above-mentioned method can quickly improve the image quality, break through the limitations of the hardware performance of the shooting device, and meet the needs of users.

[0132] In some implementations, after the first target image is input into the image format conversion model, the first target image needs to be processed through the first convolution channel and the second convolution channel.

[0133] S2300 includes:

[0134] S2311, reading the mask feature vector output by the first convolution channel;

[0135] The first target image is input into the image format conversion model, the input channel of the image format conversion model converts the first target image into an array vector matrix, and then the array vector matrix is ​​respectively input into the first convolution channel, the second convolution channel, and the output ends of the first convolution channel and the second convolution channel.

[0136] Among them, the first convolution layer and the first attention layer in the first convolution channel extract the features of the array vector matrix, and the extracted feature vector is the mask feature vector.

[0137] S2312, performing a dot product operation on the mask feature vector and the array vector matrix of the first target image;

[0138] The feature layer and the second attention layer in the second convolution channel extract the features of the array vector matrix in a hierarchical manner. Among them, each second convolution layer and linear rectification layer in the feature layer extracts the convolution features in the array vector matrix in a hierarchical manner. Finally, the feature vector output by the second convolution channel is the convolution feature vector.

[0139] S2313, performing an addition operation on the result of the dot product operation and the feature vector output by the second convolution channel;

[0140] At the output end of the first convolution channel and the second convolution channel, the mask feature vector output by the first convolution channel is subjected to a dot product operation with the array vector matrix, that is, the eye mask feature matrix represented by the mask feature vector is multiplied with the array vector matrix.

[0141] S2314: Map the result obtained by the addition operation through a preset hyperbolic tangent function to generate the second target image.

[0142] After the mask feature vector output by the first convolution channel is dot-producted with the array vector matrix to obtain a dot product result, the vector matrix obtained by the dot product result is added to the feature vector output by the second convolution channel.

[0143] After the calculation result is obtained by the addition operation, the calculation result needs to be mapped. The mapping method is: the calculation result is mapped by the hyperbolic tangent function. The hyperbolic tangent function (tanh) is the ratio of the hyperbolic sine function (sinh) to the hyperbolic cosine function (cosh), which can map the calculation result obtained by the addition operation within the value range of [-1,1]. Finally, the mapped vector matrix is ​​pixelated to generate the second target image.

[0144] S1200, performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images;

[0145] The second target image is subjected to image enhancement processing by a preset image enhancement strategy. The preset image enhancement strategy is to perform mapping processing on the second target image by a set mapping function.

[0146] In some embodiments, the image enhancement strategy includes: one or more combinations of gamma mapping algorithms, nonlinear algorithms, linear algorithms, and logarithmic mapping algorithms. It should be noted that the image enhancement strategy is not limited thereto, and depending on the specific application scenario, the second target image can also be enhanced by a mapping function adapted to the needs of the specific scenario.

[0147] In this embodiment, the number of enhanced images is 4. However, the number of enhanced images is not limited thereto, and according to different specific application scenarios, the number of enhanced images can be 2, 3, 5, 6, 7 or more.

[0148] S1300, based on a preset weight calculation formula, respectively calculating the gradient weight and exposure weight corresponding to the second target image and each enhanced image;

[0149] After the second target image is enhanced by the image enhancement strategy, the image weights of each enhanced image and the second target image need to be calculated.

[0150] The image weight includes: the gradient weight and exposure weight of the enhanced image. The gradient weight is calculated by the preset Sobel operator, while the exposure weight is calculated by the preset exposure calculation formula.

[0151] S1400, performing image synthesis processing on the second target image and each of the enhanced images based on the gradient weight and the exposure weight to generate a synthesized image;

[0152] After calculating the gradient weight and exposure weight of each enhanced image, the image weight of each enhanced image is calculated based on the gradient weight and exposure weight, where image weight = gradient weight * exposure weight. The image weight represents the weight value of each enhanced image and the second target image in the new fused image. Therefore, after calculating the image weight of each enhanced image and the second target image, the second target image and multiple enhanced images are superimposed according to the image weight, and the pixels of each corresponding pixel point are weighted based on the image weight to generate a composite image synthesized by the second target image and multiple enhanced images. Since the enhanced images are all derived from the second target image, there is no need to align the second target image and multiple enhanced images during this image fusion process, which improves the efficiency of synthesis.

[0153] S1500: Perform image fusion on the synthesized image according to a preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image.

[0154] After the synthesized image is synthesized, it is necessary to perform image processing on the synthesized image according to the Gaussian pyramid and the Laplacian pyramid. The processing method of the Gaussian pyramid is: down-sampling the synthesized image multiple times, and each down-sampling uses a Gaussian blur method to process the image.

[0155] Since the Gaussian pyramid will lose some high-frequency features in the image during image processing, in order to compensate for the high-frequency features, a Laplacian pyramid is constructed. In each layer of the Laplacian pyramid, the predicted image after upsampling and Gaussian convolution of the image on the previous layer is subtracted from the image corresponding to each layer of the Gaussian pyramid to generate a sampled image of the Laplacian pyramid. Furthermore, the number of layers of the Laplacian pyramid corresponds to the number of layers of the Gaussian pyramid. And the image of each layer of the Laplacian pyramid is a high-frequency representation of the image of the corresponding layer of the Gaussian pyramid.

[0156] After obtaining multiple layers of sampled images through Gaussian pyramid and Laplacian pyramid, dot product operation is performed on the images of corresponding layers of Gaussian pyramid and Laplacian pyramid to obtain multiple weighted images. The weighted images make up for the high-frequency features of Gaussian pyramid images.

[0157] After calculating and obtaining multiple weighted images, the multiple weighted images are fused. During fusion, since the image specifications of each weighted image are different, it is necessary to upsample the bottom layer image (the weighted image with the smallest size) in the weighted image, and then upsample the weighted image of the previous layer and the bottom layer image to generate the first fused image. The first fused image is then upsampled, and the upsampled image is fused with the weighted image of the previous layer to generate the second fused image. Similarly, the weighted image of the current layer and the upsampled image of the fused image of the previous layer are fused until the last layer of weighted images is fused, and the fused image obtained is the enhanced image.

[0158] In this implementation, the number of layers of the Gaussian pyramid is 4, and the corresponding Laplacian pyramid is also 4, so 4 first sampled images, 4 second sampled images, and 4 weighted images are generated. However, the number of layers of the Gaussian pyramid and the number of layers of the Laplacian pyramid are not limited thereto. According to different specific application scenarios, in some implementations, the number of layers of the Gaussian pyramid and the number of layers of the Laplacian pyramid can be: 2 layers, 3 layers, 5 layers, 6 layers or more layers.

[0159] The above implementation method performs image enhancement processing on the second target image obtained by converting the first target image, thereby enhancing the image distortion caused by format conversion in the second target image from different dimensions. Then, by calculating the gradient weight and exposure weight of each enhanced image, and calculating the fusion weight of each enhanced image from the gradient weight and exposure weight, multiple enhanced images are synthesized by the fusion weight to generate a synthetic image, and the image distortion of the second target image is further repaired. Finally, the synthetic image is fused for multiple frames through the Gaussian pyramid and the Laplacian pyramid, and the second target image is subjected to a third distortion repair, thereby improving the image quality of the second target image.

[0160] In some embodiments, the image enhancement strategy includes: a gamma mapping algorithm and a nonlinear algorithm. Figure 2 , Figure 2 This is a schematic diagram of a first flow chart for generating an enhanced image in this embodiment.

[0161] like Figure 2 As shown, S1200 includes:

[0162] S1211, reading the grayscale value of the second target image;

[0163] Grayscale processing is performed on the second target image to convert the second target image into a grayscale image, and then the grayscale values ​​of the pixel values ​​in the grayscale image are read.

[0164] S1212, performing gamma correction processing on the grayscale value according to a preset gamma mapping algorithm to generate a first enhanced image corresponding to the second target image;

[0165] The grayscale values ​​in the grayscale image are gamma corrected by a preset gamma mapping algorithm, and the image obtained after the gamma correction is the first enhanced image. The gamma mapping algorithm can effectively suppress the grayscale value of the non-bright area, maintain the grayscale value of the highlight area, and enhance the contrast of the highlight area.

[0166] Specifically, the formula of the gamma mapping algorithm is as follows:

[0167]

[0168] Among them, the gamma value is 0.8, x represents the pixel value, y gamma Represents the pixel value after gamma correction.

[0169] S1213: Perform value mapping on the grayscale value according to a preset nonlinear algorithm, and generate a second enhanced image corresponding to the second target image according to a result of the value mapping.

[0170] The nonlinear algorithm used in this embodiment is: S-curve mapping. The S-curve is a mapping curve that increases the grayscale value of the bright area and reduces the grayscale value of the dark area. After mapping, it can effectively improve the image contrast, remove the fog caused by the format conversion of the second target image, and obtain a clearer image.

[0171] The formula for the nonlinear algorithm is as follows:

[0172]

[0173]

[0174] Among them, x represents the pixel value, y s The value after nonlinear algorithm mapping, α needs to be greater than 1 to improve the contrast. This application sets α=1.3.

[0175] It should be pointed out that there is no obvious order relationship between the generation of the first enhanced image and the second enhanced image. In a specific application scenario, the generation order of the above images can be adjusted according to actual needs.

[0176] In some embodiments, the image enhancement strategy includes linear compensation. Figure 3 , Figure 3 This is a schematic diagram of a second flow chart for generating an enhanced image in this embodiment.

[0177] like Figure 3 As shown, after S1213, it includes:

[0178] S1221. Performing linear compensation processing on the second enhanced image according to a preset linear algorithm;

[0179] In this embodiment, a linear compensation process is performed on the second enhanced image by a linear algorithm, and the linear compensation can be: direct linear stretching, cropped linear stretching or segmented stretching. Image linear stretching can improve image contrast.

[0180] The formula of the linear algorithm is expressed as:

[0181] y l =clip(y s *ratio,0,1)

[0182]

[0183]

[0184] Among them, x represents the pixel value, y s The value after nonlinear algorithm mapping, α needs to be greater than 1 to improve contrast. This application sets α = 1.3, y lIndicates the value after clipping, the ratio value is 1.5, and clip indicates clipping processing.

[0185] S1222: Perform image cropping processing on the second enhanced image after the linear compensation processing, so that the tone value of the second enhanced image is mapped within a preset value range, and generate a third enhanced image corresponding to the second target image.

[0186] The image hue value after linear compensation will exceed the value range of [0,1]. Therefore, it is necessary to further perform image cropping on the excess portion, and crop the excess portion so that the hue value of the cropped image returns to the value range of [0,1]. The image obtained by cropping is the third target image.

[0187] In some embodiments, the image enhancement strategy includes a logarithmic mapping algorithm. Figure 4 , Figure 4 This is a schematic diagram of a third process for generating an enhanced image in this embodiment.

[0188] like Figure 4 As shown, S1200 includes:

[0189] S1231, reading the gray value of the second target image;

[0190] Grayscale processing is performed on the second target image to convert the second target image into a grayscale image, and then the grayscale values ​​of the pixel values ​​in the grayscale image are read.

[0191] S1232. Perform logarithmic mapping processing on the grayscale value according to a preset logarithmic mapping algorithm to generate a fourth enhanced image corresponding to the second target image.

[0192] In this embodiment, the grayscale value is logarithmically mapped by the μ-log mapping function, and the image information is mapped to a low dynamic range, especially in the dark area and the normal exposure area, the mapping is almost linear, and only the bright area is suppressed to a certain extent. The advantage is that the image is contrast enhanced and the brightness of the dark area is enhanced, and the color is brighter. The fourth enhanced image is generated by logarithmically mapping the grayscale value by the μ-log mapping function.

[0193] The formula for the μ-log mapping function is as follows:

[0194] if mean(x)<0.5:μ=50

[0195] if mean(x)≥0.5:μ=500

[0196]

[0197] Among them, x represents the pixel value and μ is a constant value.

[0198] In some implementations, after multiple enhanced images are calculated, it is necessary to perform weight calculation on the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image. Figure 5 , Figure 5 A schematic diagram of the process of generating gradient weights and exposure weights for each image in this embodiment.

[0199] like Figure 5 As shown, S1300 includes:

[0200] S1311, reading grayscale images of the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image;

[0201] The second target image is subjected to image enhancement processing by using an image enhancement strategy to generate a first enhanced image, a second enhanced image, a third enhanced image and a fourth enhanced image. Then, the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are subjected to grayscale processing to generate grayscale images corresponding to the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image.

[0202] S1312, performing a first directional gradient and a second directional gradient on each of the grayscale images by using a preset Sobel operator, and generating a gradient weight of each of the grayscale images according to the first directional gradient and the second directional gradient;

[0203] After the grayscale images corresponding to the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are generated, the first direction gradient and the second direction gradient of each grayscale image are calculated by the Sobel operator. The first direction gradient refers to the X direction gradient in the coordinate system; the second direction gradient refers to the Y direction gradient in the coordinate system.

[0204] The Sobel operator calculates the gradients of the grayscale image in the x-direction and the y-direction, and calculates the square root of the gradients in the two directions as the gradient weight. In this implementation, a smoothing coefficient is added on the basis of the Sobel operator. The smoothing coefficient helps to reduce the occurrence of faults or artifacts (false shadow images) caused by excessive differences in grayscale values ​​in multi-frame fusion, and ensures that the image weight can be weighted.

[0205] After the first directional gradient and the second directional gradient are calculated, the square root of the first directional gradient and the second directional gradient is calculated, and the value of the square root is the gradient weight of each grayscale image.

[0206] The Sobel operator formula is written as follows:

[0207] Gradient in x direction: Gradient in y direction:

[0208] Total gradient:

[0209] Among them, g represents the gradient weight.

[0210] S1313. According to a preset exposure calculation formula, three-channel exposure values ​​of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are calculated, and exposure weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are generated according to the three-channel exposure values.

[0211] The gradient weights of the grayscale images corresponding to the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image are calculated. The exposure weights corresponding to the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image are calculated according to the exposure calculation formula. In the RGB color mode, the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image all have three channels. The exposure value of a single channel of each image is calculated separately by the exposure calculation formula, and then the exposure weight of each image is obtained by multiplying the exposure values ​​of the three channels.

[0212] The exposure calculation formula is as follows:

[0213]

[0214] Among them, x represents the pixel value, ex is the exposure matching coefficient, σ is the weight difference variance, λ e Smoothing coefficient.

[0215] In some implementations, after the gradient weight and exposure weight of each image are calculated, each image needs to be synthesized according to the gradient weight and exposure weight. Figure 6 , Figure 6 The figure is a schematic diagram of the process of generating a composite image according to the present embodiment.

[0216] like Figure 6 As shown, S1400 includes:

[0217] S1411. Calculate image weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image according to the gradient weight and the exposure weight;

[0218] After the gradient weights and exposure weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image are calculated, the image weight of each image is calculated according to the gradient weight and the exposure weight. The calculation method is: the value obtained by multiplying the gradient weight and the exposure weight of each image is used as the image weight of each image.

[0219] S1412. Perform image fusion on the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image according to the image weights to generate the composite image.

[0220] The image weight represents the weight value of each enhanced image and the second target image in the new fused image. Therefore, after calculating the image weight of each enhanced image and the second target image, the second target image and multiple enhanced images are superimposed according to the image weight, and the pixels of each corresponding pixel point are weighted based on the image weight to generate a composite image synthesized by the second target image and multiple enhanced images. Since the enhanced images are all derived from the second target image, there is no need to align the second target image and multiple enhanced images during this image fusion process, which improves the efficiency of synthesis.

[0221] In some embodiments, after the composite image is generated, it is necessary to further enhance the composite image. Figure 7 , Figure 7 The following is a schematic diagram of the process of generating an imposed image in this embodiment.

[0222] like Figure 7 As shown, S1500 includes:

[0223] S1511, inputting the synthesized image into the Gaussian pyramid for sampling processing to generate a plurality of first sampling images;

[0224] After the synthesized image is synthesized, it is necessary to perform image processing on the synthesized image according to the Gaussian pyramid and the Laplacian pyramid. The processing method of the Gaussian pyramid is: down-sample the synthesized image multiple times, and each down-sampling uses a Gaussian blur method to process the image. Multiple first sampled images are obtained through Gaussian pyramid processing.

[0225] S1512, inputting the top layer image in the Gaussian pyramid into the Laplacian pyramid for sampling processing to generate a plurality of second sampling images, wherein the plurality of first sampling images correspond to the second sampling images one by one;

[0226] Since the Gaussian pyramid will lose some high-frequency features in the image during image processing, in order to compensate for the high-frequency features, a Laplacian pyramid is constructed. In each layer of the Laplacian pyramid, the predicted image after upsampling and Gaussian convolution of the image on the previous layer is subtracted from each layer image corresponding to the Gaussian pyramid to generate a sampled image of the Laplacian pyramid. Furthermore, the number of layers of the Laplacian pyramid corresponds to the number of layers of the Gaussian pyramid. And the image of each layer of the Laplacian pyramid is a high-frequency representation of the image of the corresponding layer of the Gaussian pyramid. Through Laplacian pyramid processing, multiple second sampled images are generated. The first sampled image and the second sampled image have a one-to-one correspondence, and the corresponding method is that images with the same size correspond to each other.

[0227] S1513, performing weighted operations on the plurality of first sampling images and the corresponding plurality of second sampling images respectively to generate a plurality of weighted images;

[0228] After obtaining multiple layers of sampled images through Gaussian pyramid and Laplacian pyramid, dot product operation is performed on the images of corresponding layers of Gaussian pyramid and Laplacian pyramid to obtain multiple weighted images. The weighted images make up for the high-frequency features of Gaussian pyramid images.

[0229] S1514: performing image fusion on the plurality of weighted images in sequence along the reverse direction of the Laplacian pyramid to generate the enhanced image.

[0230] After calculating and obtaining multiple weighted images, the multiple weighted images are fused. During fusion, since the image specifications of each weighted image are different, it is necessary to upsample the bottom layer image (the weighted image with the smallest size) in the weighted image, and then upsample the weighted image of the previous layer and the bottom layer image to generate the first fused image. The first fused image is then upsampled, and the upsampled image is fused with the weighted image of the previous layer to generate the second fused image. Similarly, the weighted image of the current layer and the upsampled image of the fused image of the previous layer are fused until the last layer of weighted images is fused, and the fused image obtained is the enhanced image.

[0231] In this implementation, the number of layers of the Gaussian pyramid is 4, and the corresponding Laplacian pyramid is also 4, so 4 first sampled images, 4 second sampled images, and 4 weighted images are generated. However, the number of layers of the Gaussian pyramid and the number of layers of the Laplacian pyramid are not limited thereto. According to different specific application scenarios, in some implementations, the number of layers of the Gaussian pyramid and the number of layers of the Laplacian pyramid can be: 2 layers, 3 layers, 5 layers, 6 layers or more layers.

[0232] Please refer to Figure 8 , Figure 8Schematic diagram of the basic structure of the image enhancement device of this embodiment.

[0233] like Figure 8 As shown, an image enhancement device includes: an acquisition module 1100, a processing module 1200, a calculation module 1300, a synthesis module 1400 and an execution module 1500. Among them:

[0234] The acquisition module 1100 is used to acquire a second target image converted from the first target image, wherein the information carrying capacity of the second target image is greater than the information carrying capacity of the first target image;

[0235] The processing module 1200 is used to perform image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images;

[0236] The calculation module 1300 is used to calculate the gradient weight and exposure weight corresponding to the second target image and each enhanced image respectively based on a preset weight calculation formula;

[0237] The synthesis module 1400 is used to perform image synthesis processing on the second target image and each of the enhanced images to generate a synthesized image based on the gradient weight and the exposure weight;

[0238] The execution module 1500 is used to perform image fusion on the composite image according to the preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image.

[0239] The image enhancement device performs image enhancement processing on the second target image obtained by converting the first target image, and enhances the image distortion caused by format conversion in the second target image from different dimensions. Then, by calculating the gradient weight and exposure weight of each enhanced image, and calculating the fusion weight of each enhanced image from the gradient weight and exposure weight, multiple enhanced images are synthesized to generate a synthetic image with the fusion weight, and the image distortion of the second target image is further repaired. Finally, the synthesized image is fused with multiple frames through the Gaussian pyramid and the Laplacian pyramid, and the second target image is repaired for the third time, thereby improving the image quality of the second target image.

[0240] In some embodiments, the image enhancement device further comprises:

[0241] A first reading submodule, used for reading the grayscale value of the second target image;

[0242] A first processing submodule, configured to perform gamma correction processing on the grayscale value according to a preset gamma mapping algorithm to generate a first enhanced image corresponding to the second target image;

[0243] The second processing submodule is used to perform value mapping on the grayscale value according to a preset nonlinear algorithm, and generate a second enhanced image corresponding to the second target image according to a result of the value mapping.

[0244] In some embodiments, the image enhancement device further comprises:

[0245] A third processing submodule, configured to perform linear compensation processing on the second enhanced image according to a preset linear algorithm;

[0246] The first execution submodule is used to perform image cropping processing on the second enhanced image after the linear compensation processing, so that the tone value of the second enhanced image is mapped within a preset value range, and generate a third enhanced image corresponding to the second target image.

[0247] In some embodiments, the image enhancement device further comprises:

[0248] A second reading submodule, used for reading the grayscale value of the second target image;

[0249] The fourth processing submodule is used to perform logarithmic mapping processing on the grayscale value according to a preset logarithmic mapping algorithm to generate a fourth enhanced image corresponding to the second target image.

[0250] In some embodiments, the image enhancement device further comprises:

[0251] A third reading submodule, used for reading the grayscale images of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image;

[0252] a fifth processing submodule, configured to perform a first directional gradient and a second directional gradient on each of the grayscale images by using a preset Sobel operator, and generate a gradient weight of each of the grayscale images according to the first directional gradient and the second directional gradient;

[0253] The second execution submodule is used to calculate the three-channel exposure values ​​of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to a preset exposure calculation formula, and generate the exposure weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to the three-channel exposure values.

[0254] In some embodiments, the image enhancement device further comprises:

[0255] A first calculation submodule, used for calculating the image weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to the gradient weight and the exposure weight;

[0256] The first synthesis submodule is used to perform image fusion on the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image according to the image weight to generate the synthesized image.

[0257] In some embodiments, the image enhancement device further comprises:

[0258] A first sampling submodule, used for inputting the synthetic image into the Gaussian pyramid for sampling processing to generate a plurality of first sampling images;

[0259] A second sampling submodule is used to input the top layer image in the Gaussian pyramid into the Laplacian pyramid for sampling processing to generate a plurality of second sampling images, wherein the plurality of first sampling images and the second sampling images correspond one to one;

[0260] A second calculation submodule, used for performing weighted operations on the plurality of first sampled images and the corresponding plurality of second sampled images respectively to generate a plurality of weighted images;

[0261] The third execution submodule is used to perform image fusion on the multiple weighted images in sequence along the reverse direction of the Laplacian pyramid to generate the enhanced image.

[0262] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0263] like Fig. 9 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected via a system bus. Among them, the non-volatile storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement an image enhancement method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute an image enhancement method. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0264] In this embodiment, the processor is used to execute Figure 8 The specific functions of the acquisition module 1100, the processing module 1200, the calculation module 1300, the synthesis module 1400 and the execution module 1500 are stored in the memory, and the program code and various data required to execute the above modules are stored. The network interface is used to transmit data between the user terminal or the server. The memory in this embodiment stores the program code and data required to execute all sub-modules in the image enhancement device, and the server can call the program code and data of the server to execute the functions of all sub-modules.

[0265] The computer device performs image enhancement processing on the second target image obtained by converting the first target image, and enhances the image distortion caused by format conversion in the second target image from different dimensions. Then, by calculating the gradient weight and exposure weight of each enhanced image, and calculating the fusion weight of each enhanced image from the gradient weight and exposure weight, multiple enhanced images are synthesized to generate a synthetic image with the fusion weight, and the image distortion of the second target image is further repaired. Finally, the synthesized image is fused for multiple frames through the Gaussian pyramid and the Laplacian pyramid, and the second target image is repaired for the third time, thereby improving the image quality of the second target image.

[0266] The present application also provides a computer storage medium, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the image enhancement method of any of the above embodiments.

[0267] The present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the image enhancement method described in any embodiment of the present application.

[0268] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0269] Those skilled in the art will appreciate that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, altered, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted.

[0270] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image enhancement method, characterized in that: include: Acquire a second target image converted from the first target image, wherein the information carrying capacity of the second target image is greater than the information carrying capacity of the first target image; Performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images, wherein the preset image enhancement strategy is to perform mapping processing on the second target image through a set mapping function to generate the enhanced image, and the image enhancement strategy includes: one or more combination algorithms among a gamma mapping algorithm, a nonlinear algorithm, a linear algorithm and a logarithmic mapping algorithm; Based on a preset weight calculation formula, respectively calculate the gradient weight and exposure weight corresponding to the second target image and each enhanced image; Based on the gradient weight and the exposure weight, performing image synthesis processing on the second target image and each of the enhanced images to generate a synthesized image; The composite image is fused according to a preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image.

2. The image enhancement method according to claim 1, characterized in that: The performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images comprises: Reading the grayscale value of the second target image; Performing gamma correction processing on the grayscale value according to a preset gamma mapping algorithm to generate a first enhanced image corresponding to the second target image; The grayscale values ​​are mapped according to a preset nonlinear algorithm, and a second enhanced image corresponding to the second target image is generated according to a result of the mapping.

3. The image enhancement method according to claim 2, characterized in that: After performing value mapping on the grayscale value according to a preset nonlinear algorithm and generating a second enhanced image corresponding to the second target image according to a result of the value mapping, the method further comprises: performing linear compensation processing on the second enhanced image according to a preset linear algorithm; The second enhanced image after the linear compensation processing is subjected to image cropping processing, so that the tone value of the second enhanced image is mapped within a preset value range, and a third enhanced image corresponding to the second target image is generated.

4. The image enhancement method according to claim 3, characterized in that: The performing image enhancement processing on the second target image according to a preset image enhancement strategy to generate a plurality of enhanced images comprises: Reading the grayscale value of the second target image; The grayscale value is logarithmically mapped according to a preset logarithmic mapping algorithm to generate a fourth enhanced image corresponding to the second target image.

5. The image enhancement method according to claim 4, characterized in that: The step of calculating the gradient weight and exposure weight corresponding to the second target image and each enhanced image based on a preset weight calculation formula includes: Reading grayscale images of the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image; A first directional gradient and a second directional gradient of each of the grayscale images are subjected to a preset Sobel operator, and a gradient weight of each of the grayscale images is generated according to the first directional gradient and the second directional gradient; According to a preset exposure calculation formula, three-channel exposure values ​​of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are calculated, and exposure weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are generated according to the three-channel exposure values.

6. The image enhancement method according to claim 5, characterized in that: The performing image synthesis processing on the second target image and each of the enhanced images to generate a synthetic image based on the gradient weight and the exposure weight comprises: Calculating image weights of the second target image, the first enhanced image, the second enhanced image, the third enhanced image, and the fourth enhanced image according to the gradient weight and the exposure weight; The second target image, the first enhanced image, the second enhanced image, the third enhanced image and the fourth enhanced image are fused according to the image weight to generate the composite image.

7. The image enhancement method according to claim 1, characterized in that: The performing image fusion on the synthesized image according to the preset Gaussian pyramid and Laplacian pyramid to generate an enhanced image comprises: Inputting the synthesized image into the Gaussian pyramid for sampling processing to generate a plurality of first sampling images; Inputting the top layer image in the Gaussian pyramid into the Laplacian pyramid for sampling processing to generate a plurality of second sampling images, wherein the plurality of first sampling images correspond to the second sampling images one by one; Performing weighted operations on the plurality of first sampling images and the corresponding plurality of second sampling images respectively to generate a plurality of weighted images; The plurality of weighted images are sequentially fused along the reverse direction of the Laplacian pyramid to generate the enhanced image.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the image enhancement method according to any one of claims 1 to 7.

9. A computer storage medium, characterized in that When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to perform the steps of the image enhancement method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the image enhancement method described in any one of claims 1 to 7 are implemented.

Images