Construction method of image enhancement model, image enhancement method, device, equipment and medium

Abstract

The invention discloses an image enhancement model construction method, an image enhancement method, an image enhancement device, equipment and a medium. The construction method of the image enhancement model comprises the steps that an original sample set is acquired, and an original sample comprises at least one target image and an enhanced image matched with the target image; forming a training sample set according to each target image and an enhanced image matched with the target image, wherein the enhanced image is used as annotation data in a training sample; and training the convolutional neural network model by using the training sample set to obtain an image enhancement model. Through the technical scheme of the invention, the construction of the image enhancement model and the enhancement of the image can be realized, the construction difficulty of the image enhancement model is reduced, and the efficiency of image enhancement work and the quality of an image enhancement result are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method for constructing an image enhancement model, an image enhancement method, a device, equipment and a medium. Background Art

[0002] Underwater image processing and understanding are playing an increasingly critical role in many fields of today's society, such as underwater archaeology, marine ecological research, and the application of underwater robots. However, due to the complexity of the water environment itself and the absorption and scattering effects of light in water, directly collected underwater images usually face the problem of image quality degradation, which seriously affects the accurate acquisition and effective analysis of underwater information.

[0003] In order to improve the quality of underwater images, traditional methods such as white balance, histogram equalization and physical model-based methods have been adopted, which have improved the visibility and information content of underwater images to a certain extent. At the same time, the rapid development of deep neural networks has spawned many underwater image enhancement methods based on deep learning, which can be roughly divided into two categories: generative adversarial networks (GANs) and convolutional neural networks (CNNs). However, at a time when unmanned underwater vehicles (UUVs) and underwater robots are widely used, these underwater devices are often constrained by limited computing resources, making it difficult for traditional deep learning models to effectively achieve underwater image enhancement on these platforms.

[0004] Traditional underwater image enhancement technology is often limited to a single scene and is difficult to adapt to the complex and changing underwater environment. Underwater image enhancement methods based on deep learning generally have problems such as complex network structure, large number of parameters and calculations, high training difficulty and poor stability, which greatly limit its effectiveness and promotion in practical applications. In summary, the existing image enhancement methods have the problems of difficult model construction, low efficiency of image enhancement work and low quality of image enhancement results. Summary of the invention

[0005] The present invention provides a method for constructing an image enhancement model, an image enhancement method, a device, equipment and a medium, which can solve the problems that the existing image enhancement method model is difficult to construct, the efficiency of image enhancement work and the quality of image enhancement results are both low.

[0006] In a first aspect, an embodiment of the present invention provides a method for constructing an image enhancement model, the method comprising:

[0007] Acquire an original sample set, wherein the original sample includes at least one target image and an enhanced image matching the target image;

[0008] Forming a training sample set according to each target image and an enhanced image matching the target image, wherein the enhanced image is used as annotation data in the training sample;

[0009] The convolutional neural network model is trained using the training sample set to obtain an image enhancement model.

[0010] In a second aspect, an embodiment of the present invention provides an image enhancement method, the method comprising:

[0011] Acquire a degraded image to be enhanced;

[0012] Inputting the degraded image into an image enhancement model to obtain a target enhanced image matching the degraded image;

[0013] The target enhanced image is sent to a display device for display.

[0014] In a third aspect, an embodiment of the present invention provides a device for constructing an image enhancement model, the device comprising:

[0015] An original sample acquisition module, used to acquire an original sample set, wherein the original sample includes at least one target image and an enhanced image matching the target image;

[0016] A training sample generation module, used to form a training sample set according to each target image and an enhanced image matching the target image, wherein the enhanced image is used as annotation data in the training sample;

[0017] The model training module is used to train the convolutional neural network model using the training sample set to obtain an image enhancement model.

[0018] In a fourth aspect, an embodiment of the present invention provides an image enhancement device, the device comprising:

[0019] An image acquisition module, used for acquiring a degraded image to be enhanced;

[0020] An image processing module, used for inputting the degraded image into an image enhancement model to obtain a target enhanced image matching the degraded image;

[0021] The display module is used to send the target enhanced image to a display device for display.

[0022] In a fifth aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for constructing an image enhancement model and an image enhancement method described in any embodiment of the present invention.

[0026] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for constructing an image enhancement model and an image enhancement method described in any embodiment of the present invention when executed.

[0027] The technical solution of the embodiment of the present invention first obtains the original sample set, and forms a training sample set according to each target image and the enhanced image matching the target image, the enhanced image is used as the annotation data in the training sample, and then the convolutional neural network model is trained using the training sample set to obtain the image enhancement model, and then the degraded image to be enhanced is obtained, and the degraded image is input into the image enhancement model to obtain the target enhanced image matching the degraded image, and finally the target enhanced image is sent to the display device for display, which solves the problems of the existing image enhancement method model construction difficulty, the low efficiency of image enhancement work and the quality of image enhancement results, realizes the construction of the image enhancement model and the enhancement of the image, reduces the difficulty of building the image enhancement model, and improves the efficiency of the image enhancement work and the quality of the image enhancement results.

[0028] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 is a flowchart of a method for constructing an image enhancement model provided according to Embodiment 1 of the present invention;

[0031] Figure 2 is a flow chart of an image enhancement method provided according to Embodiment 2 of the present invention;

[0032] Figure 3is a structural schematic diagram of a device for constructing an image enhancement model provided according to Embodiment 3 of the present invention;

[0033] Figure 4 is a structural schematic diagram of an image enhancement device provided according to a fourth embodiment of the present invention;

[0034] Figure 5 It is a structural schematic diagram of an electronic device for implementing a method for constructing an image enhancement model and an image enhancement method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variation of the terms "including" and "having" is intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Embodiment 1

[0038] Figure 1 A flowchart of a method for constructing an image enhancement model provided in Example 1 of the present invention. This embodiment is applicable to the situation where an image enhancement model is constructed. The method can be executed by a device for constructing an image enhancement model. The device for constructing an image enhancement model can be implemented in the form of hardware and / or software. The device for constructing an image enhancement model can be configured in a terminal or server with a function of constructing an image enhancement model.

[0039] like Figure 1 As shown, the method includes:

[0040] S110: Obtain an original sample set.

[0041] The original sample includes at least one target image and an enhanced image matching the target image.

[0042] Specifically, the original sample is the initial data set without any processing or processing. In this original sample set, there is at least one target image and an enhanced image matching the target image; further, the target image can be an original image that is expected to be processed or analyzed, such as a natural landscape photo with less than ideal shooting effect, or a portrait with insufficient clarity. The enhanced image is the result of a certain specific processing or optimization of the target image, such as an image obtained by increasing contrast, adjusting brightness, enhancing color saturation, etc.; for example, if the target image is a photo of an indoor scene taken in dim light, then the enhanced image that matches it may be a version with brightness enhancement and color correction, so that the objects in the image are clearer and more identifiable, and the colors are more vivid and realistic.

[0043] S120: Form a training sample set according to each target image and an enhanced image matching the target image, wherein the enhanced image is used as annotation data in the training sample.

[0044] S130. Use the training sample set to train the convolutional neural network model to obtain an image enhancement model.

[0045] The convolutional neural network model is a deep learning model with strong feature learning and generalization capabilities. It can automatically learn effective feature representations from a large amount of data, avoiding the complexity and limitations of manually designed features.

[0046] The convolutional neural network model includes a color restoration layer, a feature extraction layer, a feature fusion layer and an output enhancement layer which are connected in sequence.

[0047] Specifically, a convolutional neural network model is trained using a training sample set to obtain an image enhancement model, including: obtaining a target training sample from the training sample set, and inputting the target training sample into the convolutional neural network model; performing a color restoration operation on a target image in the target training sample through a color restoration layer in the convolutional neural network model to obtain a color compensation map matching the target image; processing the color compensation map through a feature extraction layer in the convolutional neural network model to obtain a multi-scale feature matching the target image; performing feature fusion processing on the multi-scale feature through a feature fusion layer in the convolutional neural network model to obtain a target processed image; adjusting parameters of the convolutional neural network model using a cosine annealing learning rate decay algorithm according to the target processed image and the enhanced image in the target training sample through an output enhancement layer in the convolutional neural network model; returning to execute the operation of obtaining the target training sample in the training sample set until a training termination condition is met, and determining the trained convolutional neural network model as the image enhancement model.

[0048] In a specific implementation scenario of this embodiment, a target training sample is obtained from a training sample set, and the target training sample is input into a convolutional neural network model, where the target training sample is a specific sample carefully selected from the entire training sample set for the current training process; a color restoration operation is performed on a target image in the target training sample through a color restoration layer in the convolutional neural network model to obtain a color compensation map matching the target image, wherein the color restoration layer functions to correct color deviations and distortions that may exist in the target image. For example, if the target image is a landscape photo with dim colors due to poor lighting conditions, the color restoration layer can process it to generate a color compensation map with richer and more natural colors; then, the color compensation map is processed through a feature extraction layer in the convolutional neural network model to obtain multi-scale features matching the target image, wherein the feature extraction layer can extract feature information of different levels and scales from the color compensation map. Taking the recognition of animal images as an example, multi-scale features ranging from subtle hair textures to overall body contours may be extracted; then, the multi-scale features are subjected to feature fusion processing through the feature fusion layer in the convolutional neural network model to obtain a target processing image, wherein the feature fusion layer synthesizes and integrates these multi-scale features to form a more complete and accurate target processing image. For example, for a complex scene image containing multiple objects, the feature fusion layer can fuse together features such as the shape, color, and texture of different objects to construct a comprehensive and clear image representation; further, the convolutional neural network model is parameterized using a cosine annealing learning rate decay algorithm according to the target processing image and the enhanced image in the target training sample through the output enhancement layer in the convolutional neural network model, wherein the cosine annealing learning rate decay algorithm can dynamically adjust the learning rate of the model to optimize the training effect of the model. The output enhancement layer will continuously adjust the parameters of the model based on the difference between the target processed image and the enhanced image in the target training sample, so that the output of the model gradually approaches the ideal enhancement effect; finally, it returns to execute the operation of obtaining the target training sample in the training sample set until the end of training conditions are met, and the trained convolutional neural network model is determined as the image enhancement model. The end of training conditions can be that the accuracy of the model reaches a certain threshold, the number of training rounds reaches a preset value, or the value of the loss function converges to a certain range. After repeated training and optimization, a convolutional neural network model that can effectively perform image enhancement is finally obtained.

[0049] Among them, a color restoration operation is performed on a target image in a target training sample through a color restoration layer in a convolutional neural network model to obtain a color compensation map matching the target image, including: performing a feature extraction operation on the target image through a pre-configured multidimensional feature extraction model to obtain a multidimensional feature map matching the target image; performing channel component extraction processing on the target image to obtain an R channel feature map, a G channel feature map, and a B channel feature map matching the target image; performing an element-level subtraction operation based on the R channel feature map and the G channel feature map to obtain a first feature map, and performing an element-level subtraction operation based on the G channel feature map and the B channel feature map to obtain a second feature map; performing an element-level addition operation based on the R channel feature map, the G channel feature map, the B channel feature map, and the multidimensional feature map to obtain a third feature map; and splicing the first feature map, the second feature map, and the third feature map according to a preset dimensionality rule to obtain a color compensation map matching the target image.

[0050] On the basis of the above steps, specifically, a feature extraction operation is performed on the target image through a pre-configured multidimensional feature extraction model to obtain a multidimensional feature map matching the target image; wherein the multidimensional feature extraction model is a model that can deeply analyze various potential features of the target image, for example, it can capture a variety of complex feature information such as texture details, shape contours, color distribution, etc. in the image; further, the target image is subjected to channel component extraction processing to obtain an R channel feature map, a G channel feature map, and a B channel feature map matching the target image. It is easy to understand that in image processing, RGB channels represent red (Red), green (Green), and blue (Blue) channels, respectively. The R channel feature map mainly reflects the distribution and intensity of the red component in the image, the G channel feature map shows the situation of the green component, and the B channel feature map reflects the information of the blue component. For example, for an image with red flowers as the main body, the value of the flower area in the R channel feature map will be relatively high; then, an element-level subtraction operation is performed based on the R channel feature map and the G channel feature map to obtain a first feature map, and an element-level subtraction operation is performed based on the G channel feature map and the B channel feature map to obtain a second feature map, wherein the element-level subtraction operation is to subtract the elements at corresponding positions in the two channel feature maps one by one, and this operation can highlight the difference characteristics between different channels; for example, if the green component of a certain area in the image is strong and the red component is weak, then the subtraction operation of the R channel feature map and the G channel feature map can more clearly reflect this difference; further, an element-level addition operation is performed based on the R channel feature map, the G channel feature map, the B channel feature map and the multi-dimensional feature map to obtain a third feature map, and the element-level addition operation adds the elements of these different feature maps at corresponding positions, and integrates the information of each channel and multi-dimensional features; finally, the first feature map, the second feature map and the third feature map are spliced ​​according to the preset dimensionality rule to obtain a color compensation map matching the target image. The preset dimensionality rules are pre-set ways of combining these feature maps to form a new image that can effectively compensate for the color deviation or deficiency of the original target image; for example, they may be spliced ​​in channel order or a specific hierarchical structure, and the final generated color compensation map can better restore the true color of the image and improve the image quality and visual effect.

[0051] Further, the target image is subjected to channel component extraction processing to obtain an R channel feature map, a G channel feature map and a B channel feature map matching the target image, including: extracting the R component of each pixel point contained in the target image, and arranging the positions of each R component based on the positions of each pixel point in the target image to obtain an R component feature map; extracting the G component of each pixel point contained in the target image, and arranging the positions of each G component based on the positions of each pixel point in the target image to obtain a G component feature map; extracting the B component of each pixel point contained in the target image, and arranging the positions of each B component based on the positions of each pixel point in the target image to obtain a B component feature map; performing convolution operations on the R component feature map, the G component feature map and the B component feature map respectively through a 1×1 convolution kernel to obtain an R channel feature map, a G channel feature map and a B channel feature map.

[0052] The technical solution of the embodiment of the present invention first obtains the original sample set, and forms a training sample set according to each target image and the enhanced image matching the target image. The enhanced image is used as the annotation data in the training sample. Finally, the convolutional neural network model is trained using the training sample set to obtain the image enhancement model, thereby realizing the construction of the image enhancement model, reducing the difficulty of building the image enhancement model, and improving the efficiency of the image enhancement work and the quality of the image enhancement results.

[0053] Embodiment 2

[0054] Figure 2 A flowchart of an image enhancement method provided in Embodiment 2 of the present invention is provided. This embodiment is applicable to situations where images are to be enhanced. The method can be executed by an image enhancement device. The image enhancement device can be implemented in the form of hardware and / or software. The image enhancement device can be configured in a terminal or server with an image enhancement function.

[0055] S210: Obtain a degraded image to be enhanced.

[0056] S220: Input the degraded image into an image enhancement model to obtain a target enhanced image that matches the degraded image.

[0057] S230: Send the target enhanced image to a display device for display.

[0058] The technical solution of the embodiment of the present invention obtains a degraded image to be enhanced, inputs the degraded image into an image enhancement model, obtains a target enhanced image matching the degraded image, and finally sends the target enhanced image to a display device for display, thereby achieving image enhancement and improving the efficiency of image enhancement work and the quality of image enhancement results.

[0059] Embodiment 3

[0060] Figure 3 This is a schematic diagram of the structure of a device for constructing an image enhancement model provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0061] The original sample acquisition module 310 is used to acquire an original sample set, wherein the original sample includes at least one target image and an enhanced image matching the target image;

[0062] A training sample generation module 320, used to form a training sample set according to each target image and an enhanced image matching the target image, wherein the enhanced image is used as annotation data in the training sample;

[0063] The model training module 330 is used to train the convolutional neural network model using the training sample set to obtain an image enhancement model.

[0064] The technical solution of the embodiment of the present invention first obtains the original sample set, and forms a training sample set according to each target image and the enhanced image matching the target image. The enhanced image is used as the annotation data in the training sample. Finally, the convolutional neural network model is trained using the training sample set to obtain the image enhancement model, thereby realizing the construction of the image enhancement model, reducing the difficulty of building the image enhancement model, and improving the efficiency of the image enhancement work and the quality of the image enhancement results.

[0065] Based on the above embodiment, the model training module 330 includes:

[0066] A convolutional neural unit is used to obtain a target training sample from a training sample set and input the target training sample into a convolutional neural network model;

[0067] A color restoration unit, used to perform a color restoration operation on a target image in a target training sample through a color restoration layer in a convolutional neural network model to obtain a color compensation map matching the target image;

[0068] A feature extraction unit, used to process the color compensation map through a feature extraction layer in a convolutional neural network model to obtain multi-scale features that match the target image;

[0069] A feature fusion unit, used to perform feature fusion processing on the multi-scale features through a feature fusion layer in a convolutional neural network model to obtain a target processed image;

[0070] A parameter adjustment unit, configured to adjust the parameters of the convolutional neural network model by using a cosine annealing learning rate decay algorithm according to the target processed image and the enhanced image in the target training sample through an output enhancement layer in the convolutional neural network model;

[0071] Return to execute the operation of obtaining the target training sample in the training sample set until the training end condition is met, and determine the trained convolutional neural network model as the image enhancement model.

[0072] Based on the above embodiment, the color restoration unit further includes:

[0073] A multidimensional feature extraction unit, used to perform a feature extraction operation on a target image through a pre-configured multidimensional feature extraction model to obtain a multidimensional feature map matching the target image;

[0074] A channel component extraction unit is used to perform channel component extraction processing on the target image to obtain an R channel feature map, a G channel feature map and a B channel feature map matching the target image;

[0075] A subtraction unit, configured to perform an element-level subtraction operation on the R channel feature map and the G channel feature map to obtain a first feature map, and to perform an element-level subtraction operation on the G channel feature map and the B channel feature map to obtain a second feature map;

[0076] An adding unit, used for performing element-level addition operation based on the R channel feature map, the G channel feature map, the B channel feature map and the multi-dimensional feature map to obtain a third feature map;

[0077] The splicing unit is used to splice the first feature map, the second feature map and the third feature map according to a preset dimensional rule to obtain a color compensation map matching the target image.

[0078] Based on the above embodiment, the channel component extraction unit further includes:

[0079] A first component extraction unit, configured to extract an R component of each pixel point contained in the target image, and arrange the positions of the R components based on the positions of the pixels in the target image to obtain an R component feature map;

[0080] A second component extraction unit is used to extract the G component of each pixel point contained in the target image, and arrange the positions of each G component based on the position of each pixel point in the target image to obtain a G component feature map;

[0081] A third component extraction unit is used to extract the B component of each pixel point contained in the target image, and arrange the positions of each B component based on the position of each pixel point in the target image to obtain a B component feature map;

[0082] The convolution unit is used to perform convolution operations on the R component feature map, the G component feature map and the B component feature map through a 1×1 convolution kernel to obtain an R channel feature map, a G channel feature map and a B channel feature map.

[0083] An image enhancement model construction device provided in an embodiment of the present invention can execute an image enhancement model construction method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0084] Embodiment 4

[0085] Figure 4 This is a schematic diagram of the structure of an image enhancement device provided by Embodiment 4 of the present invention. Figure 4 As shown, the device comprises:

[0086] An image acquisition module 410, used to acquire a degraded image to be enhanced;

[0087] An image processing module 420, configured to input the degraded image into an image enhancement model to obtain a target enhanced image that matches the degraded image;

[0088] The display module 430 is used to send the target enhanced image to a display device for display.

[0089] The technical solution of the embodiment of the present invention obtains a degraded image to be enhanced, inputs the degraded image into an image enhancement model, obtains a target enhanced image matching the degraded image, and finally sends the target enhanced image to a display device for display, thereby achieving image enhancement and improving the efficiency of image enhancement work and the quality of image enhancement results.

[0090] An image enhancement device provided in an embodiment of the present invention can execute an image enhancement method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0091] Embodiment 5

[0092] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0093] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11 in communication, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0094] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0095] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for constructing an image enhancement model and a method for enhancing an image.

[0096] Accordingly, a method for constructing an image enhancement model includes:

[0097] Acquire an original sample set, wherein the original sample includes at least one target image and an enhanced image matching the target image;

[0098] Forming a training sample set according to each target image and an enhanced image matching the target image, wherein the enhanced image is used as annotation data in the training sample;

[0099] The convolutional neural network model is trained using the training sample set to obtain an image enhancement model.

[0100] In some embodiments, a method for constructing an image enhancement model and a method for enhancing an image may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for constructing an image enhancement model and a method for enhancing an image described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute a method for constructing an image enhancement model and a method for enhancing an image in any other appropriate manner (e.g., by means of firmware).

[0101] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0103] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0104] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0105] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0106] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0107] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

Claims

1. A method for constructing an image enhancement model, characterized in that: include: Acquire an original sample set, wherein the original sample includes at least one target image and an enhanced image matching the target image; Forming a training sample set according to each target image and an enhanced image matching the target image, wherein the enhanced image is used as annotation data in the training sample; Use the training sample set to train the convolutional neural network model to obtain an image enhancement model; The convolutional neural network model includes a color restoration layer, a feature extraction layer, a feature fusion layer and an output enhancement layer connected in sequence, and the color restoration layer is used to correct the color deviation and distortion in the target image; Among them, the convolutional neural network model is trained using the training sample set to obtain the image enhancement model, including: obtaining the target training sample in the training sample set, and inputting the target training sample into the convolutional neural network model; performing a color restoration operation on the target image in the target training sample through the color restoration layer in the convolutional neural network model to obtain a color compensation map matching the target image; processing the color compensation map through the feature extraction layer in the convolutional neural network model to obtain a multi-scale feature matching the target image; performing feature fusion processing on the multi-scale feature through the feature fusion layer in the convolutional neural network model to obtain a target processed image; adjusting the parameters of the convolutional neural network model according to the target processed image and the enhanced image in the target training sample by using the cosine annealing learning rate decay algorithm through the output enhancement layer in the convolutional neural network model; returning to execute the operation of obtaining the target training sample in the training sample set until the training end condition is met, and determining the trained convolutional neural network model as the image enhancement model.

2. The method according to claim 1, characterized in that Performing a color restoration operation on a target image in a target training sample through a color restoration layer in a convolutional neural network model to obtain a color compensation map matching the target image, including: Performing a feature extraction operation on the target image through a pre-configured multi-dimensional feature extraction model to obtain a multi-dimensional feature map matching the target image; Performing channel component extraction processing on the target image to obtain an R channel feature map, a G channel feature map, and a B channel feature map that match the target image; An element-wise subtraction operation is performed based on the R channel feature map and the G channel feature map to obtain a first feature map, and an element-wise subtraction operation is performed based on the G channel feature map and the B channel feature map to obtain a second feature map; Performing element-level addition operation based on the R channel feature map, the G channel feature map, the B channel feature map, and the multi-dimensional feature map to obtain a third feature map; The first feature map, the second feature map and the third feature map are spliced ​​according to a preset dimensionality rule to obtain a color compensation map that matches the target image.

3. The method according to claim 2, characterized in that: Performing channel component extraction processing on the target image to obtain an R channel feature map, a G channel feature map, and a B channel feature map matching the target image, including: Extracting the R component of each pixel point contained in the target image, and arranging the positions of each R component based on the position of each pixel point in the target image to obtain an R component feature map; Extracting the G component of each pixel point contained in the target image, and arranging the positions of each G component based on the positions of each pixel point in the target image to obtain a G component feature map; Extracting the B component of each pixel point contained in the target image, and arranging the positions of the B components based on the positions of the pixels in the target image to obtain a B component feature map; The R component feature map, the G component feature map and the B component feature map are convolved with a 1×1 convolution kernel to obtain an R channel feature map, a G channel feature map and a B channel feature map.

4. A method for enhancing an image, characterized in that: include: Acquire a degraded image to be enhanced; Inputting the degraded image into an image enhancement model trained by the method according to any one of claims 1 to 3 to obtain a target enhanced image matching the degraded image; The target enhanced image is sent to a display device for display.

5. A device for constructing an image enhancement model, characterized in that: include: An original sample acquisition module, used to acquire an original sample set, wherein the original sample includes at least one target image and an enhanced image matching the target image; A training sample generation module, used to form a training sample set according to each target image and an enhanced image matching the target image, wherein the enhanced image is used as annotation data in the training sample; A model training module is used to train the convolutional neural network model using a training sample set to obtain an image enhancement model; The convolutional neural network model includes a color restoration layer, a feature extraction layer, a feature fusion layer and an output enhancement layer connected in sequence, and the color restoration layer is used to correct the color deviation and distortion in the target image; Among them, the model training module specifically includes: a convolutional neural unit, which is used to obtain a target training sample in a training sample set and input the target training sample into a convolutional neural network model; a color restoration unit, which is used to perform a color restoration operation on a target image in a target training sample through a color restoration layer in a convolutional neural network model to obtain a color compensation map matching the target image; a feature extraction unit, which is used to process the color compensation map through a feature extraction layer in a convolutional neural network model to obtain a multi-scale feature matching the target image; a feature fusion unit, which is used to perform feature fusion processing on the multi-scale features through a feature fusion layer in a convolutional neural network model to obtain a target processed image; a parameter adjustment unit, which is used to adjust the parameters of the convolutional neural network model using a cosine annealing learning rate decay algorithm according to the target processed image and the enhanced image in the target training sample through an output enhancement layer in the convolutional neural network model; and return to execute the operation of obtaining a target training sample in a training sample set until the training termination condition is met, and the trained convolutional neural network model is determined as an image enhancement model.

6. An image enhancement device, characterized in that: include: An image acquisition module, used for acquiring a degraded image to be enhanced; An image processing module, configured to input the degraded image into an image enhancement model trained by the method according to any one of claims 1 to 3, to obtain a target enhanced image matching the degraded image; The display module is used to send the target enhanced image to a display device for display.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for constructing an image enhancement model described in any one of claims 1 to 3 and the method for enhancing an image described in claim 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for constructing an image enhancement model as described in any one of claims 1 to 3 and an image enhancement method as described in claim 4 when executed.

Images