A method, device, equipment and readable storage medium for real-time image enhancement

Abstract

The present invention provides a method, apparatus, device and readable storage medium for real-time image enhancement, which relates to the field of image enhancement technology, including obtaining an original image and a preset target image, preprocessing the original image and the target image to obtain an input image; based on a color correction matrix algorithm, performing image enhancement on the input image to obtain an image enhancement result, wherein the image enhancement channel includes a low-resolution affine matrix extraction channel and a high-resolution image upsampling channel; based on a semantic segmentation algorithm, further reconstructing the image enhancement result to obtain a final image enhancement result. The advantageous effect of the present invention is that, based on semantic segmentation, the model's ability to sharpen images is improved, and the generated image colors are more realistic. Compared with the traditional method of extracting affine matrices through convolutional neural networks, the affine matrix obtained through the attention module has a stronger ability to express the affine between input and output images.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to a method, apparatus, device and readable storage medium for real-time image enhancement. Background Art

[0002] In natural environments, it's difficult for non-professional users to capture high-quality photos using handheld cameras. For example, photos taken in nighttime can be underexposed or overexposed at midday. Photos captured in these environments often require image enhancement, such as white balancing, color correction, and contrast adjustment.

[0003] Traditional image enhancement methods use operations such as histogram transforms, logarithmic transforms, and transform-domain convolution. However, for images, this remains an NP-hard problem, making it difficult for traditional algorithms to efficiently solve it. Pixel-to-pixel encoding and decoding methods require large amounts of data and enormous network parameters, and cannot achieve real-time performance. Summary of the Invention

[0004] The present invention aims to provide a method, apparatus, device, and readable storage medium for real-time image enhancement to improve the above-mentioned problems. To achieve the above-mentioned objectives, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present application provides a method for real-time image enhancement, comprising:

[0006] Acquire an original image and a preset target image, and preprocess the original image and the target image to obtain an input image;

[0007] Based on a color correction matrix algorithm, the input image is enhanced to obtain an image enhancement result, wherein the image enhancement channel includes a low-resolution affine matrix extraction channel and a high-resolution image upsampling channel;

[0008] Based on the semantic segmentation algorithm, the image enhancement result is further reconstructed to obtain the final image enhancement result.

[0009] Preferably, the image enhancement is performed on the input image based on a color correction matrix algorithm to obtain an image enhancement result, which includes:

[0010] A feature extraction method based on an attention mechanism is used to extract global features and local features from the input image and fuse them into an affine matrix;

[0011] Grayscale information of the input image is extracted, and cubic linear interpolation is performed on the affine matrix to generate an image enhancement result.

[0012] Preferably, the step of extracting the grayscale information of the input image includes:

[0013] Sorting the pixels in the image blocks of the input image according to their grayscale values;

[0014] Divide the local image block into a plurality of sub-regions; arrange all pixels of the local image block in non-descending order according to the grayscale value; and divide the local image block into S sub-regions according to the grayscale value sorting;

[0015] Add all the sub-region vectors: obtain a feature descriptor by connecting the descriptors of each sub-region, and calculate the descriptor dimension of the final structure through the feature descriptors to obtain the grayscale information.

[0016] Preferably, the image enhancement result is further reconstructed based on a semantic segmentation algorithm, which includes:

[0017] Obtaining each semantic segmentation image in the image enhancement result;

[0018] Based on a semantic segmentation algorithm, obtaining a difference between a first calculation result and a second calculation result; the first calculation result includes a mean absolute error between each block of the semantic segmentation image and the input image; the second calculation result includes a mean absolute error between each block of the semantic segmentation image and the target image;

[0019] Calculating a weight corresponding to each block of the semantic segmentation image, where the weight is obtained by subtracting the average of the sum of the Euclidean distances between the average pixel values ​​of the input image and the target image from the average pixel value of each block;

[0020] Normalizing all the weights to obtain a normalized result;

[0021] The image enhancement result is enhanced according to the normalization result and the contrast loss function.

[0022] In a second aspect, the present application further provides a real-time image enhancement device, comprising:

[0023] Acquisition module: used to acquire an original image and a preset target image, and pre-process the original image and the target image to obtain an input image;

[0024] Enhancement module: used to perform image enhancement on the input image based on the color correction matrix algorithm to obtain an image enhancement result, wherein the image enhancement channel includes a low-resolution affine matrix extraction channel and a high-resolution image upsampling channel;

[0025] Reconstruction module: used to further reconstruct the image enhancement result based on the semantic segmentation algorithm to obtain the final image enhancement result.

[0026] In a third aspect, the present application further provides a real-time image enhancement device, comprising:

[0027] memory for storing computer programs;

[0028] A processor is configured to implement the steps of the real-time image enhancement method when executing the computer program.

[0029] In a fourth aspect, the present application further provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned method based on real-time image enhancement are implemented.

[0030] The beneficial effects of the present invention are: using the image attention mechanism to extract the affine matrix of low-quality images, and using the attention mechanism to dynamically extract the global features of the image, so that the color affine matrix accurately reflects the relationship between the input image and the target image, thereby enhancing the robustness of the network, and then performing cubic linear interpolation on the affine matrix through its grayscale image, which greatly improves the inference speed as a whole and reduces the model size.

[0031] The present invention introduces contrast loss to fully enhance the mapping ability of the affine matrix, so that the efficiency of converting the input image to the target image is improved. Based on the premise of semantic segmentation, the model's ability to sharpen the image is improved, and the generated image color is more realistic. Compared with the traditional method of extracting the affine matrix through a convolutional neural network, the affine matrix obtained by the attention module has a stronger ability to express the affine between the input and output images. The block contrast loss retains the color fidelity of the image while guiding the generated image to be more inclined to the target image with enhanced quality. By learning an affine matrix and adopting a lightweight network, the present invention reduces the amount of network parameters and can achieve real-time image enhancement of 16-bit images at the millisecond level.

[0032] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the embodiments of the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 Schematic diagram of the process of real-time image enhancement according to an embodiment of the present invention;

[0035] Figure 2 Schematic diagram of the structure of a real-time image enhancement device according to an embodiment of the present invention;

[0036] Figure 3 Schematic diagram of the structure of the real-time image enhancement device described in an embodiment of the present invention.

[0037] In the figure: 701, acquisition module; 702, enhancement module; 7021, fusion unit; 7022, extraction unit; 70221, sorting unit; 70222, segmentation unit; 70223, first calculation unit; 703, reconstruction module; 7031, acquisition unit; 7032, second calculation unit; 7033, third calculation unit; 7034, normalization unit; 7035, enhancement unit; 800, real-time image enhancement device; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. DETAILED DESCRIPTION

[0038] To make the purposes, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures. Furthermore, in the description of the present invention, the terms "first," "second," etc. are used only to distinguish the description and are not to be understood as indicating or implying relative importance.

[0040] Example 1:

[0041] This embodiment provides a real-time image enhancement method.

[0042] See also Figure 1 , the figure shows that the method includes step S100, step S200 and step S300.

[0043] S100: Acquire an original image and a preset target image, and pre-process the original image and the target image to obtain an input image.

[0044] It is understood that in this step, the image is preprocessed. In this embodiment, in order to accurately restore the image's color information, enhance the model's performance, and fully utilize the model's capabilities, 16-bit (RGB channel) images are selected as the source and target images. The target image is a high-quality image that has been manually enhanced (reduced) in contrast and increased (reduced) in exposure by a professional designer. The relationship between the input and target images is expressed as:

[0045]

[0046] Among them, img t is the target image, img s is the original image, is the pixel adjustment function.

[0047] S200 . Perform image enhancement on the input image based on a color correction matrix algorithm to obtain an image enhancement result. The image enhancement channels include a low-resolution affine matrix extraction channel and a high-resolution image upsampling channel.

[0048] It is understandable that, in this step, S200 includes S201 and S202, wherein:

[0049] S201, performing global feature extraction and local feature extraction on the input image based on an attention mechanism, and fusing the extracted features into an affine matrix;

[0050] S202: Extract grayscale information of the input image, perform cubic linear interpolation on the affine matrix, and generate an image enhancement result.

[0051] In this embodiment, the image enhancement method is based on a color affine matrix. Unlike pixel-by-pixel enhancement methods, this method offers improved real-time performance and visual quality for high-resolution images. The network based on this method is divided into a low-resolution affine matrix extraction channel and an upsampling channel that generates a high-resolution image.

[0052] The former channel interpolates the input image to a size of 128*128, then uses an attention-based feature extraction method, performing global and local feature extraction in parallel, to ultimately fuse the resulting image into an affine matrix. The latter channel extracts grayscale information from the input image and performs cubic linear interpolation on the affine matrix to generate a high-resolution generated image.

[0053] Optionally, the feature extraction module, global feature module, and local feature extraction use the attention module. The attention module is a dynamic feature extraction method that can significantly improve the expression ability of the affine matrix. Its simplified formula is:

[0054] X fea =W ι *[softmax(X q ⊙X k )⊙X v ]

[0055] ⊙ is a matrix operation, X q , X k , X v W is the matrix formed by flattening the image into vectors after segmentation and combining them through nonlinear transformation. ι are the perceptron parameters, X fea It is the dynamic feature extracted from the image after passing through the attention network.

[0056] S202 includes S2021, S2022 and S2023, among which:

[0057] S2021. Sort pixels in an image block of the input image according to grayscale values;

[0058] S2022, dividing the local image block into a plurality of sub-regions; arranging all pixels of the local image block in non-descending order according to the grayscale values; and dividing the local image block into S sub-regions according to the grayscale value sorting;

[0059] S2023. Add all the sub-region vectors: obtain a feature descriptor by connecting the descriptors of each sub-region, and calculate the descriptor dimension of the final structure through the feature descriptors to obtain grayscale information.

[0060] It should be noted that in order for the descriptor to contain global information, the local image block is divided into many subregions according to the grayscale value sorting of the pixels in the image block. To achieve this goal, the following two steps are required: (1) All pixels in the local block are sorted in non-descending order according to the grayscale value; (2) The local block is divided into S subregions according to the grayscale value sorting. Each subregion contains the same number of pixels, and the pixels in the subregion are not necessarily adjacent to other pixels. This method ensures that the proposed descriptor is invariant to monotonic changes in grayscale.

[0061] S300: Based on a semantic segmentation algorithm, further reconstruct the image enhancement result to obtain a final image enhancement result.

[0062] It is understandable that step S300 includes S3001, S3002, S3003, S3004 and S3005, wherein:

[0063] S3001, obtaining each semantic segmentation image in the image enhancement result;

[0064] S3002. Obtain a difference between a first calculation result and a second calculation result based on a semantic segmentation algorithm; the first calculation result includes a mean absolute error between each block of the semantic segmentation image and the input image; the second calculation result includes a mean absolute error between each block of the semantic segmentation image and the target image;

[0065] S3003, calculating a weight corresponding to each block of the semantic segmentation image, where the weight is obtained by subtracting the average of the sum of the Euclidean distances between the average pixel values ​​of the input image and the full pixel value of the target image from the average pixel value of each block;

[0066] S3004: Normalize all the weights to obtain a normalized result;

[0067] S3005: Enhance the image enhancement result according to the normalization result and the contrast loss function.

[0068] It should be noted that the steps of combining the contrast loss of different region weights are as follows:

[0069] In order to ensure that the reconstructed image is closer to the target image and further improve the affine ability of the affine matrix, the contrast loss is calculated for each semantic segmentation area.

[0070] In this embodiment, the RGB image is divided into n blocks using the existing semantic segmentation results. The semantic segmentation of the image is mainly aimed at the image foreground and background, as well as the segmentation of different natural landscapes and objects, so as to enhance the main part with obvious color difference and improve the fidelity of the generated model. With the input image The mean absolute error and the difference between each semantic segmentation image and the target image The difference in mean absolute error, where wk is the weight corresponding to each block, which is the average value of the pixel value of each block Subtract the input image respectively and the average value of the pixel values ​​of the entire target image The average value of the sum of the Euclidean distances is obtained, and all weights are normalized. σ is a custom threshold, which is initialized to the mean of the Manhattan distance between the input image and the target image. The formula is is the number of blocks for semantic segmentation, and N is the number of samples in the training set.

[0071]

[0072]

[0073] In addition, X and Y are calculated t L1 loss of the entire image.

[0074]

[0075] This method has a total The calculation is expressed as:

[0076]

[0077] Θ is According to experience, it is more appropriate to set the weight of 1.2.

[0078] In summary, traditional image enhancement methods are generally based on pixel-by-pixel reconstruction, which requires a large number of parameters in the forward network, consuming a lot of energy and time in forward reasoning. The network of the method of the present invention only needs to learn a single affine matrix, with a small number of network parameters and a single inference time of milliseconds, enabling real-time image enhancement.

[0079] According to the above description, the preprocessed images are input into the network and trained according to the above loss, using the Adam optimizer and the learning rate set to 0.0002.

[0080] The advantages of this invention are that it introduces contrast loss, fully enhancing the mapping ability of the affine matrix, thereby improving the efficiency of converting the input image to the target image. Based on the premise of semantic segmentation, it improves the model's ability to sharpen the image, and generates more realistic image colors.

[0081] Example 2:

[0082] like Figure 2 As shown, this embodiment provides a real-time image enhancement device. Referring to FIG2 , the device includes an acquisition module 701, an enhancement module 702, and a reconstruction module 703, wherein:

[0083] Acquisition module 701: used to acquire an original image and a preset target image, and pre-process the original image and the target image to obtain an input image;

[0084] Enhancement module 702: configured to perform image enhancement on the input image based on a color correction matrix algorithm to obtain an image enhancement result, wherein the image enhancement channel includes a low-resolution affine matrix extraction channel and a high-resolution image upsampling channel;

[0085] Reconstruction module 703: used to further reconstruct the image enhancement result based on the semantic segmentation algorithm to obtain the final image enhancement result.

[0086] Specifically, the enhancement module 702 includes a fusion unit 7021 and an extraction unit 7022, wherein:

[0087] Fusion unit 7021: used for performing global feature extraction and local feature extraction on the input image based on an attention mechanism, and fusing the extracted features into an affine matrix;

[0088] Extraction unit 7022: used to extract grayscale information of the input image, perform cubic linear interpolation on the affine matrix, and generate an image enhancement result.

[0089] Specifically, the extraction unit 7022 includes a sorting unit 70221, a segmentation unit 70222, and a first calculation unit 70223, wherein:

[0090] Sorting unit 70221: configured to sort pixels in an image block of the input image according to grayscale values;

[0091] Segmentation unit 70222: configured to segment a local image block into a plurality of sub-regions; all pixels of the local image block are arranged in non-descending order according to the grayscale values; and the local image block is divided into S sub-regions according to the grayscale value sorting;

[0092] The first calculation unit 70223 is used to add all the sub-region vectors: a feature descriptor is obtained by connecting the descriptors of each sub-region, and the descriptor dimension of the final construction is obtained by calculating the feature descriptors to obtain grayscale information.

[0093] Specifically, the reconstruction module 703 includes an acquisition unit 7031, a second calculation unit 7032, a third calculation unit 7033, a normalization unit 7034 and an enhancement unit 7035, wherein:

[0094] An acquisition unit 7031 is configured to acquire each semantic segmentation image in the image enhancement result;

[0095] A second calculation unit 7032 is configured to obtain a difference between a first calculation result and a second calculation result based on a semantic segmentation algorithm; the first calculation result includes a mean absolute error between each block of the semantic segmentation image and the input image; and the second calculation result includes a mean absolute error between each block of the semantic segmentation image and the target image.

[0096] The third calculation unit 7033 is used to calculate the weight corresponding to each block of the semantic segmentation image, where the weight is the average value of the pixel value of each block minus the average value of the sum of the Euclidean distances of the pixel values ​​of the input image and the target image.

[0097] Normalization unit 7034: used to normalize all the weights to obtain a normalized result;

[0098] The enhancement unit 7035 is used to enhance the image enhancement result according to the normalization result and the contrast loss function.

[0099] It should be noted that, regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0100] Example 3:

[0101] Corresponding to the above method embodiment, this embodiment further provides a real-time image enhancement device. The real-time image enhancement device described below and the real-time image enhancement method described above can refer to each other.

[0102] Figure 3 FIG. 8 is a block diagram of a real-time image enhancement device 800 according to an exemplary embodiment. Figure 3As shown, the real-time image enhancement device 800 may include: a processor 801 and a memory 802. The real-time image enhancement device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0103] The processor 801 is used to control the overall operation of the real-time image enhancement device 800 to complete all or part of the steps in the above-mentioned real-time image enhancement method. The memory 802 is used to store various types of data to support the operation of the real-time image enhancement device 800. Such data may include, for example, instructions for any application or method operating on the real-time image enhancement device 800, as well as application-related data, such as contact information, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules. The above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the real-time image enhancement device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, so the corresponding communication component 805 may include: a Wi-Fi module, a Bluetooth module, an NFC module.

[0104] In an exemplary embodiment, the real-time image enhancement device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned real-time image enhancement method.

[0105] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the above-described real-time image enhancement method. For example, the computer-readable storage medium may be the aforementioned memory 802 including the program instructions. The program instructions may be executed by the processor 801 of the real-time image enhancement device 800 to perform the above-described real-time image enhancement method.

[0106] Example 4:

[0107] Corresponding to the above method embodiment, this embodiment further provides a readable storage medium. The readable storage medium described below and the real-time image enhancement method described above can refer to each other.

[0108] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the real-time image enhancement method of the above method embodiment.

[0109] The readable storage medium may specifically be any readable storage medium capable of storing program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0110] In summary, this method uses the attention structure based on the deep learning method to extract the color information of the original image to the maximum extent, and enhances the generated image through the block contrast loss. Compared with the traditional extraction of affine matrix through convolutional neural network, the affine matrix obtained by the attention module has a stronger ability to express the affine between the input and output images. The block contrast loss retains the color fidelity of the image while guiding the generated image to be more towards the target image with enhanced quality. By learning an affine matrix and adopting a lightweight network, the present invention reduces the number of network parameters and can achieve real-time image enhancement of 16-bit images at the millisecond level.

[0111] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0112] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for real-time image enhancement, characterized in that: include: Acquire an original image and a preset target image, and preprocess the original image and the target image to obtain an input image; Based on a color correction matrix algorithm, the input image is enhanced to obtain an image enhancement result, wherein the image enhancement channel includes a low-resolution affine matrix extraction channel and a high-resolution image upsampling channel; Based on the semantic segmentation algorithm, the image enhancement result is further reconstructed to obtain the final image enhancement result; The image enhancement is performed on the input image based on the color correction matrix algorithm to obtain an image enhancement result, which includes: A feature extraction method based on an attention mechanism is used to extract global features and local features from the input image and fuse them into an affine matrix; Extracting grayscale information of the input image, performing cubic linear interpolation on the affine matrix, and generating an image enhancement result; The image enhancement result is further reconstructed based on the semantic segmentation algorithm, including: Obtaining each semantic segmentation image in the image enhancement result; Based on a semantic segmentation algorithm, obtaining a difference between a first calculation result and a second calculation result; the first calculation result includes a mean absolute error between each block of the semantic segmentation image and the input image; the second calculation result includes a mean absolute error between each block of the semantic segmentation image and the target image; Calculating a weight corresponding to each block of the semantic segmentation image, where the weight is obtained by subtracting the average of the sum of the Euclidean distances between the average pixel values ​​of the input image and the target image from the average pixel value of each block; Normalizing all the weights to obtain a normalized result; The image enhancement result is enhanced according to the normalization result and the contrast loss function.

2. The method for real-time image enhancement according to claim 1, characterized in that , said extracting grayscale information of said input image, including: Sorting the pixels in the image blocks of the input image according to their grayscale values; Divide the local image block into a plurality of sub-regions; arrange all pixels of the local image block in non-descending order according to the grayscale value; and divide the local image block into S sub-regions according to the grayscale value sorting; Adding the descriptors of all the sub-regions: obtaining a feature descriptor by connecting the descriptors of each sub-region, and calculating the descriptor dimension of the final structure by the feature descriptors, thereby obtaining the grayscale information.

3. A real-time image enhancement device, characterized in that: include: Acquisition module: used to acquire an original image and a preset target image, and pre-process the original image and the target image to obtain an input image; Enhancement module: used to perform image enhancement on the input image based on the color correction matrix algorithm to obtain an image enhancement result, wherein the image enhancement channel includes a low-resolution affine matrix extraction channel and a high-resolution image upsampling channel; Reconstruction module: used to further reconstruct the image enhancement result based on the semantic segmentation algorithm to obtain the final image enhancement result; The enhancement module includes: Fusion unit: used for performing global feature extraction and local feature extraction on the input image based on an attention mechanism, and fusing the extracted features into an affine matrix; Extraction unit: used for extracting grayscale information of the input image, performing cubic linear interpolation on the affine matrix, and generating an image enhancement result; The reconstruction module includes: An acquisition unit is configured to acquire each semantic segmentation image in the image enhancement result; A second calculation unit is configured to obtain a difference between a first calculation result and a second calculation result based on a semantic segmentation algorithm; the first calculation result includes a mean absolute error between each block of the semantic segmentation image and the input image; and the second calculation result includes a mean absolute error between each block of the semantic segmentation image and the target image. A third calculation unit is used to calculate the weight corresponding to each block of the semantic segmentation image, where the weight is obtained by subtracting the average value of the sum of the Euclidean distances of the average pixel values ​​of the input image and the target image from the average pixel value of each block; Normalization unit: used to normalize all the weights to obtain a normalized result; Enhancement unit: used to enhance the image enhancement result according to the normalization result and the contrast loss function.

4. The real-time image enhancement device according to claim 3, characterized in that: The extraction unit comprises: Sorting unit: used for sorting pixels in the image block of the input image according to grayscale values; A segmentation unit is configured to segment a local image block into a plurality of sub-regions; all pixels of the local image block are arranged in non-descending order according to the grayscale values; and the local image block is divided into S sub-regions according to the grayscale value sorting; The first calculation unit is used to add the descriptors of all the sub-regions: a feature descriptor is obtained by connecting the descriptors of each sub-region, and the descriptor dimension of the final structure is calculated by the feature descriptors to obtain grayscale information.

5. A real-time image enhancement device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the method for real-time image enhancement according to any one of claims 1 or 2 when executing the computer program.

6. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the real-time image enhancement method according to any one of claims 1 or 2.

Images