Method and device for compensating exposure level of image data using neural network

Abstract

A method and apparatus for correcting the exposure level of image data using a neural network model are provided. The method for correcting the exposure level of image data extracts image features from input image data, extracts luminance features from the image data using a pre-trained neural network model, generates a plurality of transformed image features from the image data based on the image features and the luminance features, corrects the exposure level of the image data based on the plurality of transformed image features, and outputs corrected image data.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for correcting the exposure level of image data using a neural network model. Background Technology

[0002] Brightness exposure is a very important element in video, and improper exposure is very limited in visual effects, such as distortion of color and brightness, and areas of the video that are excessively bright or dark can be a factor in degrading video quality.

[0003] In particular, in the fields of photography and image processing, there is a problem in that it is difficult to grasp the overall characteristics of low-light images with insufficient exposure and high-light images with excessive exposure. Therefore, correcting image exposure is emerging as an important and essential element for obtaining high-quality images.

[0004] Histogram equalization is a conventional exposure compensation method. However, while this technique focuses on enhancing the overall contrast of an image to increase its dynamic range and amplify brightness in low-light areas, it suffers from issues such as unnatural image quality due to merely improving contrast, excessive increase in image illumination, or insufficient enhancement.

[0005] Recently, methods for correcting image exposure using deep learning-based exposure correction algorithms are being studied. However, these exposure correction algorithms are limited to correcting image exposure for low-light or high-light images, and there is a problem in that it is difficult to apply them to images with various exposures. Prior art literature

[0006] Korean Registered Patent No. 10-2277005 (July 14, 2021) The problem to be solved

[0007] The present invention aims to provide a method and apparatus for correcting the exposure level of image data that can optimally correct the exposure level of image data using a neural network model. means of solving the problem

[0008] A method for correcting the exposure level of image data according to an embodiment of the present invention comprises: a step of extracting image features from input image data; a step of extracting luminance features from the image data using a previously trained neural network model; a step of generating a plurality of transformed image features from the image data based on the image features and the luminance features; and a step of correcting the exposure level of the image data based on the plurality of transformed image features and outputting corrected image data.

[0009] The step of extracting the luminance feature comprises: a step of estimating the luminance level of the image data; and a step of extracting the luminance feature based on the estimated luminance level.

[0010] The step of generating the plurality of transformed image features comprises: generating a plurality of transformation functions for each RGB channel of the image data based on the image features and the luminance features; estimating a color transformation value for each of the plurality of pixels of each RGB channel based on the plurality of transformation functions; and generating the plurality of transformed image features for each RGB channel of the image data based on the estimated color transformation value.

[0011] The step of outputting the corrected image data comprises: a step of obtaining a plurality of corrected features for the image data by fusing the plurality of transformed image features; and a step of generating the corrected image data by fusing the plurality of transformed image features and the plurality of corrected features based on preset weights, and correcting the exposure level of the image data based on the fused features.

[0012] The step of extracting the image features includes: a step of adjusting the size of the image data based on a preset resolution; and a step of extracting the image features from the size-adjusted image data.

[0013] Here, the neural network model includes a luminance feature extraction unit, an image feature conversion unit, and an exposure level correction unit, wherein the neural network model comprises: the luminance feature extraction unit receiving first image data and second image data having different exposure levels and extracting a first luminance feature for the first image data and a second luminance feature for the second image data; the image feature conversion unit receiving a first image feature extracted from the first image data, a second image feature extracted from the second image data, the first luminance feature, and the second luminance feature as inputs and generating a plurality of converted first image features for the first image feature and a plurality of converted second image features for the second image feature; and the exposure level correction unit receiving the plurality of converted first image features and the plurality of converted second image features as inputs and generating a first corrected image data in which the exposure level is corrected for the first image data and a second corrected image data in which the exposure level is corrected for the second image data. It is learned.

[0014] A method for training a neural network model for correcting the exposure level of image data according to an embodiment of the present invention comprises: a step of receiving first image data and second image data having different exposure levels and training a luminance feature extraction unit of the neural network model to extract a first luminance feature for the first image data and a second luminance feature for the second image data; and a step of receiving a first image feature extracted from the first image data, a second image feature extracted from the second image data, the first luminance feature and the second luminance feature as inputs, and training an image feature conversion unit of the neural network model to generate a plurality of converted first image features for the first image feature and a plurality of converted second image features for the second image feature. and includes the step of training the exposure level correction unit of the neural network model to receive the plurality of converted first image features and the plurality of converted second image features as input, and to generate first corrected image data in which the exposure level is corrected for the first image data and second corrected image data in which the exposure level is corrected for the second image data.

[0015] The step of training the luminance feature extraction unit comprises: a step of estimating the luminance order between the first image data and the second image data; and a step of extracting the first luminance feature for the first image data and extracting the second luminance feature for the second image data based on the estimated luminance order.

[0016] The step of training the luminance feature extraction unit comprises: receiving the actual luminance order between the first image data and the second image data as labeling data; determining an estimated loss value based on a comparison result between the estimated luminance order and the actual luminance order; and adjusting the internal parameters of the luminance feature extraction unit so that the estimated loss value is minimized.

[0017] The step of training the exposure level correction unit comprises: a step of obtaining a plurality of first correction features for the first image data by fusing the plurality of converted first image features; a step of obtaining a plurality of second correction features for the second image data by fusing the plurality of converted second image features; and a step of generating first correction image data in which the exposure level is corrected for the first image data by fusing the plurality of converted first image features and the plurality of first correction features based on preset weights, and generating second correction image data in which the exposure level is corrected for the second image data by fusing the plurality of converted second image features and the plurality of second correction features.

[0018] The step of training the exposure level correction unit comprises: receiving actual correction image data for the first image data and actual correction image data for the second image data, respectively, as labeling data; determining a correction loss value based on a comparison result between the first correction image data and the actual correction image data of the first image data and a comparison result between the second correction image data and the actual correction image data of the second image data; and adjusting internal parameters of the exposure level correction unit so that the correction loss value is minimized.

[0019] An exposure level correction device for image data according to an embodiment of the present invention includes: a memory in which an exposure correction program is stored; and a processor that, when image data is input from the outside, executes the exposure correction program to extract image features from the image data, extracts luminance features from the image data using a previously trained neural network model, generates a plurality of transformed image features from the image data based on the image features and the luminance features, corrects the exposure level of the image data based on the plurality of transformed image features, and outputs corrected image data.

[0020] The processor estimates the luminance level of the image data and extracts the luminance feature based on the estimated luminance level.

[0021] The processor generates a plurality of conversion functions for each RGB channel of the image data based on the image features and the luminance features, estimates color conversion values ​​for a plurality of pixels of each RGB channel based on the plurality of conversion functions, and generates the plurality of converted image features for each RGB channel of the image data based on the estimated color conversion values.

[0022] The processor fuses the plurality of transformed features to obtain a plurality of correction features for the image data, fuses the plurality of transformed features and the plurality of correction features based on preset weights, and corrects the exposure level of the image data based on the fused features to generate the correction image data.

[0023] The processor adjusts the size of the image data based on a preset resolution and extracts image features from the resized image data.

[0024] Here, the neural network model includes a luminance feature extraction unit, an image feature conversion unit, and an exposure level correction unit. Accordingly, the neural network model is trained such that the luminance feature extraction unit receives first image data and second image data having different exposure levels and extracts a first luminance feature for the first image data and a second luminance feature for the second image data, the image feature conversion unit receives a first image feature extracted from the first image data, a second image feature extracted from the second image data, the first luminance feature, and the second luminance feature, and generates a plurality of converted first image features for the first image feature and a plurality of converted second image features for the second image feature, and the exposure level correction unit receives the plurality of converted first image features and the plurality of converted second image features and generates a first corrected image data in which the exposure level is corrected for the first image data and a second corrected image data in which the exposure level is corrected for the second image data. Effects of the invention

[0025] The present invention estimates the luminance characteristics of an image using a pre-trained neural network model and corrects the exposure level of the image to an optimal brightness based on the estimated luminance characteristics, thereby increasing the accuracy of exposure correction for images with various exposures and improving the quality of the corrected image. Brief explanation of the drawing

[0026] FIG. 1 is a drawing showing an exposure level correction device for image data according to an embodiment of the present invention. Figure 2 is a diagram conceptually illustrating the functions of the exposure correction program of Figure 1. FIGS. 3 to 5 are drawings illustrating a method for training an exposure correction model according to an embodiment of the present invention. FIG. 6 is a diagram showing a method for training an exposure correction model according to an embodiment of the present invention. FIG. 7 is a diagram illustrating a method for correcting the exposure level of image data according to an embodiment of the present invention. Specific details for implementing the invention

[0027] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0028] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions in the embodiments of the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0029] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0030] FIG. 1 is a drawing showing an exposure level correction device for image data according to an embodiment of the present invention.

[0031] Referring to FIG. 1, the exposure level correction device (100) of the present embodiment can adjust the exposure level of the image data and generate and output corrected image data when image data having a predetermined brightness, such as an exposure level, is input from the outside.

[0032] This exposure level correction device (100) may include an input / output unit (110), a processor (120), and a memory (130).

[0033] The input / output unit (110) can receive image data from the outside. The image data received by the input / output unit (110) may be an image having a predetermined exposure level, such as a low-illumination or high-illumination exposure level.

[0034] In addition, the input / output unit (110) can output corrected image data generated by the processor (120) to be described later, such as corrected image data in which the exposure level of the image data is optimally corrected, to an external device.

[0035] The processor (120) receives image data through the input / output unit (110), corrects the exposure level of the image data using an exposure correction program (140) stored in the memory (130), and outputs the corrected image data accordingly.

[0036] The memory (130) can store information necessary for the exposure correction program (140) and the execution thereof. The exposure correction program (140) may be software that includes commands capable of extracting image features and luminance features from input image data, and correcting the exposure level of the image data based on the extracted image features and luminance features to generate corrected image data.

[0037] Accordingly, when image data is input to the input / output unit (110), the processor (120) executes the exposure correction program (140) of the memory (130) and can generate and output corrected image data with the exposure level of the image data corrected using the same.

[0038] Figure 2 is a diagram conceptually illustrating the functions of the exposure correction program of Figure 1.

[0039] Referring to FIG. 2, the exposure correction program (140) of the present embodiment may include an image feature extraction unit (141) and a neural network model, such as an exposure correction model (145). Additionally, the exposure correction model (145) may include a luminance feature extraction unit (210), an image feature conversion unit (220), and an exposure level correction unit (230).

[0040] The image feature extraction unit (141), luminance feature extraction unit (210), image feature conversion unit (220), and exposure level correction unit (230) illustrated in FIG. 2 are divided to easily explain the functions of the exposure correction program (140) of the present embodiment, but the present invention is not limited thereto.

[0041] For example, according to an embodiment of the present invention, the image feature extraction unit (141), the luminance feature extraction unit (210), the image feature conversion unit (220), and the exposure level correction unit (230) may have their functions merged or separated, and may be implemented as a series of instructions included in a single program.

[0042] The image feature extraction unit (141) can extract image features from the input image data. The image features may include pixel information for each RGB channel of the image data, for example, RGB gradation levels for each of a plurality of pixels.

[0043] For example, the image feature extraction unit (141) can adjust the size of the image data according to a preset resolution. Here, the image feature extraction unit (141) can adjust the height and width of each of the multiple pixels of the image data.

[0044] Next, the image feature extraction unit (141) can extract image features for each pixel of the image data from the size-adjusted image data.

[0045] The exposure correction model (145) may include a luminance feature extraction unit (210), an image feature conversion unit (220), and an exposure level correction unit (230).

[0046] The luminance feature extraction unit (210) can extract luminance features from image data, such as luminance features including luminance level information for each of a plurality of pixels of image data.

[0047] Here, the luminance feature extraction unit (210) can estimate the luminance level for the image data and extract luminance features for each pixel of the image data based on the estimated luminance level.

[0048] The image feature conversion unit (220) can generate a plurality of conversion features for image data based on previously extracted image features and luminance features, such as image features converted for previously extracted image features.

[0049] Here, the image feature conversion unit (220) generates a conversion function for each RGB channel of the image data and, based on the generated conversion function, can estimate a conversion value that can convert the color conversion value of each pixel, such as the grayscale level of each pixel.

[0050] Next, the image feature conversion unit (220) converts the grayscale level for each pixel of the image data based on the estimated conversion value, and accordingly, can generate a converted image feature including the converted grayscale level.

[0051] The exposure level correction unit (230) can correct the exposure level of image data based on a plurality of converted image features and, accordingly, generate and output corrected image data.

[0052] Here, the exposure level correction unit (230) can obtain multiple correction features for image data by fusing multiple converted image features, and generate corrected image data in which the exposure level of the image data is corrected by fusing multiple converted image features and multiple correction features.

[0053] This exposure correction model (145) may be a neural network model trained to correct the exposure level using training data. In other words, the exposure correction model (145) of the present embodiment trains each of the luminance feature extraction unit (210), image feature conversion unit (220), and exposure level correction unit (230) using a plurality of training data, and can correct the exposure level of image data using the trained luminance feature extraction unit (210), image feature conversion unit (220), and exposure level correction unit (230) to output corrected image data.

[0054] FIGS. 3 to 5 are drawings illustrating a method for training an exposure correction model according to an embodiment of the present invention, and FIG. 6 is a drawing illustrating a method for training an exposure correction model according to an embodiment of the present invention.

[0055] First, referring to FIGS. 3 and FIGS. 6, the exposure correction model (145) can be trained by inputting a plurality of training data, such as first image data and second image data, into a luminance feature extraction unit (210), so that the luminance feature extraction unit (210) extracts and outputs a first luminance feature for the first image data and a second luminance feature for the second image data, respectively (S10).

[0056] Here, the first image data and the second image data may have different luminance levels, for example, different exposure levels. For example, the first image data may be image data at a low illumination level, and the second image data may be image data at a high illumination level.

[0057] Additionally, the luminance feature extraction unit (210) can estimate the luminance order for the first image data and the second image data. Here, the luminance feature extraction unit (210) can determine a single luminance level by combining the two image data and estimate the luminance order of the two image data based on the determined luminance level.

[0058] Next, the luminance feature extraction unit (210) can extract a first luminance feature for the first image data and extract a second luminance feature for the second image data based on the estimated luminance order.

[0059] Here, the first luminance feature may include information regarding the luminance level of the first image data, for example, the luminance value of each pixel. Additionally, the second luminance feature may include the luminance value of each pixel of the second image data.

[0060] In addition, the luminance feature extraction unit (210) can receive the correct answer regarding the luminance order between the first image data and the second image data, such as the actual luminance order, as additional labeling data.

[0061] Here, the labeling data may have a value of 1 if the luminance level of the first image data is higher than the luminance level of the second image data, and a value of 0 if the opposite is true.

[0062] The luminance feature extraction unit (210) can determine an estimated loss value by comparing the luminance order between the output first luminance feature and the second luminance feature with the labeling data, i.e., the actual luminance order. Accordingly, the luminance feature extraction unit (210) can receive the determined estimated loss value further and repeat the aforementioned learning, such as extracting the first luminance feature from the first image data and extracting the second luminance feature from the second image data, while adjusting internal parameters so that the estimated loss value is minimized.

[0063] Next, referring to FIGS. 4 and FIGS. 6, the exposure correction model (145) inputs the image feature extracted through the aforementioned image feature extraction unit (141) and the luminance feature extracted through the luminance feature extraction unit (210) to the image feature conversion unit (220), and trains the image feature conversion unit (220) to output a plurality of converted image features for the image features (S20).

[0064] For example, the image feature conversion unit (220) can be trained to receive the first image feature and the first luminance feature of the first image data and output a plurality of converted first image features for the first image feature.

[0065] Additionally, the image feature conversion unit (220) can be trained to receive the second image feature and the second luminance feature of the second image data and output a plurality of converted second image features for the second image feature.

[0066] Here, the image feature conversion unit (220) can generate a plurality of converted first image features and a plurality of converted second image features in substantially the same manner. Accordingly, for convenience of explanation, an example in which the image feature conversion unit (220) generates a plurality of converted first image features based on the first image feature and the first luminance feature will be described below.

[0067] The image feature conversion unit (220) can generate a luminance-based image feature for the first image data based on the first image feature and the first luminance feature as shown in [Equation 1] below.

[0068] [Mathematical Formula 1]

[0069]

[0070] Here, LN is a layer normalization operation function, MAC is a multi-head cross attention operation function, Z is a first image feature, and L may be a first luminance feature.

[0071] Next, the image feature conversion unit (220) estimates a plurality of conversion functions for the first image data according to [Equation 2] below from the luminance-based image features, and can extend this to each RGB channel of the first image data as in [Equation 3].

[0072] [Mathematical Formula 2]

[0073]

[0074] Here, MLP is a multi-layer perceptron operation function, MSA is a multi-head self-attention function, and F can be a luminance-based image feature.

[0075] [Mathematical Formula 3]

[0076]

[0077] Here, Tr is a plurality of transformation functions for the R channel of the first image data, Tg is a plurality of transformation functions for the G channel of the first image data, and Tb is a plurality of transformation functions for the B channel of the first image data.

[0078] Next, the image feature conversion unit (220) can estimate color conversion values ​​for multiple pixels of each channel, such as grayscale level conversion values ​​for each pixel, based on multiple conversion functions for each channel.

[0079] Next, the image feature conversion unit (220) can generate and output a plurality of converted first image features for each channel of the first image data based on the estimated color conversion value. Here, the converted first image features can be expressed as shown in [Equation 4] below.

[0080] [Mathematical Formula 4]

[0081]

[0082] Here, is a transformed first image feature for the R channel of the first image data, and is a transformed first image feature for the G channel of the first image data, and may be a converted first image feature for channel B of the first image data.

[0083] Next, referring to FIGS. 5 and 6, the exposure correction model (145) can input a plurality of converted first image features and a plurality of converted second image features output from the image feature conversion unit (220) into the exposure level correction unit (230), and train the exposure level correction unit (230) to correct the exposure level of the first image data to generate corrected image data, such as the first corrected image data, and to correct the exposure level of the second image data to generate the second corrected image data (S30).

[0084] Here, the exposure level correction unit (230) can obtain a plurality of first correction features for the first image data by fusing a plurality of converted first image features. Additionally, the exposure level correction unit (230) can obtain a plurality of second correction features for the second image data by fusing a plurality of converted second image features.

[0085] Next, the exposure level correction unit (230) can fuse a plurality of converted first image features and a plurality of first correction features based on a preset weight, and generate first correction image data with an exposure level corrected accordingly.

[0086] Additionally, the exposure level correction unit (230) can fuse a plurality of converted second image features and a plurality of second correction features based on a preset weight, and generate second correction image data with the exposure level corrected accordingly.

[0087] Meanwhile, the exposure level correction unit (230) can receive actual correction image data for the first image data and actual correction image data for the second image data as correction answers, i.e., labeling data.

[0088] Accordingly, the exposure level correction unit (230) can determine a loss value by comparing the output first correction image data and the labeling data, that is, the actual correction image data of the first image data, and can determine a loss value by comparing the second correction image data and the actual correction image data of the second image data.

[0089] Next, the exposure level correction unit (230) can determine a correction loss value for the exposure level correction of the first image data or the second image data based on two loss values. Then, the exposure level correction unit (230) can receive the determined correction loss value further and repeat the aforementioned learning, for example, generating first correction image data for the first image data and generating second correction image data for the second image data, while adjusting internal parameters so that the correction loss value is minimized.

[0090] In this way, the exposure correction model (145) of the present embodiment may be trained to receive a plurality of training data, such as first image data and second image data having different exposure levels, and first image features for the first image data and second image features for the second image data as input, and to generate and output corrected image data with the exposure level corrected for each of the first image data and second image data from these.

[0091] Additionally, the exposure correction model (145) determines the sum of the estimated loss value determined by the aforementioned luminance feature extraction unit (210) and the correction loss value determined by the exposure level correction unit (230) as the final loss value according to the learning of the exposure correction model (145), and can repeatedly train each of the luminance feature extraction unit (210), image feature conversion unit (220), and exposure level correction unit (230) so that the determined final loss value is minimized.

[0092] Referring again to FIG. 1, the processor (120) can output the corrected image data, in which the exposure level is corrected through the learned exposure correction model (145), to the outside through the input / output unit (110).

[0093] In this way, the image data exposure level correction device (100) according to the present embodiment can improve the quality of the corrected image by increasing the accuracy of exposure correction for images with various exposures by using a previously trained neural network model to estimate the luminance characteristics of the image and correcting the exposure level of the image to an optimal brightness based on the estimated luminance characteristics.

[0094] FIG. 7 is a diagram illustrating a method for correcting the exposure level of image data according to an embodiment of the present invention.

[0095] Referring to FIG. 7, when image data is input to the input / output unit (110), the processor (120) can execute the exposure correction program (140) of the memory (130). Accordingly, the image feature extraction unit (141) can extract image features from the input image data (S110).

[0096] Here, the image feature extraction unit (141) can adjust the size of the image data according to a preset resolution and extract image features for each pixel of the image data, such as image features including the RGB grayscale level of each pixel, from the image data with the adjusted size.

[0097] Next, the exposure level of the input image data can be corrected using the learned exposure correction model (145), and the corrected image data can be generated and output accordingly.

[0098] First, the luminance feature extraction unit (210) of the exposure correction model (145) can estimate the luminance level for the image data and extract luminance features for each pixel of the image data based on the estimated luminance level (S120).

[0099] Next, the image feature conversion unit (220) of the exposure correction model (145) can generate a plurality of converted image features for the image data based on the previously extracted image features and luminance features (S130).

[0100] Here, the image feature conversion unit (220) can generate a conversion function for each RGB channel of the image data, estimate the color conversion value of each pixel based on the generated conversion function, and generate a converted image feature by converting the grayscale level for each pixel of the image data based on the estimated conversion value.

[0101] Next, the exposure level correction unit (230) of the exposure correction model (145) can correct the exposure level of the image data based on a plurality of converted image features and output the corrected image data accordingly (S140).

[0102] Here, the exposure correction model (145) can obtain multiple correction features for image data by fusing multiple converted image features, and generate corrected image data in which the exposure level of the image data is corrected by fusing multiple converted image features and multiple correction features.

[0103] The combinations of each block of the block diagram and each step of the flowchart of the present invention described above may be executed by computer program instructions. Since these computer program instructions may be loaded into an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed through the encoding processor of the computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or each step of the flowchart. Since these computer program instructions may also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in the computer-available or computer-readable memory may also produce a manufactured item containing instruction means for performing the function described in each block of the block diagram or each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer may also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

[0104] Additionally, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). Also, it should be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps described in succession may actually be performed substantially simultaneously, or the blocks or steps may sometimes be performed in reverse order according to the corresponding function.

[0105] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0106] 100: Exposure level compensation device 110: Input / Output Section 120: Processor 130: Memory 141: Image Feature Extraction Unit 145: Exposure compensation model 210: Luminance Feature Extraction Unit 220: Image Feature Transformation Unit 230: Exposure Level Correction Section

Claims

Claim 1 A method for correcting the exposure level of image data performed by an exposure level correction device, comprising: a step of extracting image features from input image data; a step of extracting luminance features from the image data using a previously trained neural network model; a step of generating a plurality of transformed image features from the image data based on the image features and the luminance features; and a step of correcting the exposure level of the image data based on the plurality of transformed image features and outputting corrected image data, wherein in the step of generating the plurality of transformed image features, the image features serve as a query in a multi-head cross-attention technique and the luminance features serve as a key and value in the multi-head cross-attention technique. Claim 2 A method for correcting the exposure level of image data according to claim 1, wherein the step of extracting the luminance feature comprises: a step of estimating the luminance level of the image data; and a step of extracting the luminance feature based on the estimated luminance level. Claim 3 A method for correcting the exposure level of image data according to claim 1, wherein the step of generating the plurality of converted image features comprises: generating a plurality of conversion functions for each RGB channel of the image data based on the image features and the luminance features; estimating a color conversion value for each of the plurality of pixels of each RGB channel based on the plurality of conversion functions; and generating the plurality of converted image features for each RGB channel of the image data based on the estimated color conversion value. Claim 4 A method for correcting the exposure level of image data according to claim 1, wherein the step of outputting the corrected image data comprises: a step of obtaining a plurality of correction features for the image data by fusing the plurality of converted image features; and a step of fusing the plurality of converted image features and the plurality of correction features based on a preset weight, and generating the corrected image data by correcting the exposure level of the image data based on the fused features. Claim 5 A method for correcting the exposure level of image data according to claim 1, wherein the step of extracting image features comprises: a step of adjusting the size of the image data based on a preset resolution; and a step of extracting image features from the size-adjusted image data. Claim 6 In claim 1, the neural network model comprises a luminance feature extraction unit, an image feature conversion unit, and an exposure level correction unit, wherein the neural network model comprises: the luminance feature extraction unit receiving first image data and second image data having different exposure levels and extracting a first luminance feature for the first image data and a second luminance feature for the second image data; the image feature conversion unit receiving a first image feature extracted from the first image data, a second image feature extracted from the second image data, the first luminance feature and the second luminance feature as inputs, and generating a plurality of converted first image features for the first image feature and a plurality of converted second image features for the second image feature; and the exposure level correction unit receiving the plurality of converted first image features and the plurality of converted second image features as inputs, and generating a first corrected image data in which the exposure level is corrected for the first image data and a second corrected image data in which the exposure level is corrected for the second image data. Exposure level correction method for image data trained to be generated. Claim 7 A method for training a neural network model to correct the exposure level of image data, wherein the neural network model comprises a luminance feature extraction unit, an image feature conversion unit, and an exposure level correction unit, and a step of training the luminance feature extraction unit to receive inputs of first image data and second image data having different exposure levels and to extract a first luminance feature for the first image data and a second luminance feature for the second image data; and a step of training the image feature conversion unit to receive inputs of a first image feature extracted from the first image data, a second image feature extracted from the second image data, the first luminance feature, and the second luminance feature, and to generate a plurality of converted first image features for the first image feature and a plurality of converted second image features for the second image feature. A method comprising the step of training an exposure level correction unit to receive a plurality of converted first image features and a plurality of converted second image features as inputs, and to generate first corrected image data in which the exposure level is corrected for the first image data and second corrected image data in which the exposure level is corrected for the second image data, wherein in the step of training the image feature conversion unit, the first image feature and the second image feature act as a query in a multi-head cross-attention technique, and the first luminance feature and the second luminance feature act as a key and value in the multi-head cross-attention technique. Claim 8 A method for training a neural network model according to claim 7, wherein the step of training the luminance feature extraction unit comprises: a step of estimating the luminance order between the first image data and the second image data; and a step of extracting the first luminance feature for the first image data and extracting the second luminance feature for the second image data based on the estimated luminance order. Claim 9 In claim 8, the step of training the luminance feature extraction unit comprises: receiving the actual luminance order between the first image data and the second image data as labeling data; determining an estimated loss value based on a comparison result between the estimated luminance order and the actual luminance order; and adjusting the internal parameters of the luminance feature extraction unit so that the estimated loss value is minimized. Claim 10 A method for training a neural network model according to claim 7, wherein the step of training the exposure level correction unit comprises: a step of obtaining a plurality of first correction features for the first image data by fusing the plurality of transformed first image features; a step of obtaining a plurality of second correction features for the second image data by fusing the plurality of transformed second image features; and a step of generating first correction image data in which the exposure level is corrected for the first image data by fusing the plurality of transformed first image features and the plurality of first correction features based on a preset weight, and generating second correction image data in which the exposure level is corrected for the second image data by fusing the plurality of transformed second image features and the plurality of second correction features. Claim 11 In claim 10, the step of training the exposure level correction unit comprises: receiving actual correction image data for the first image data and actual correction image data for the second image data, respectively, as labeling data; determining a correction loss value based on a comparison result between the first correction image data and the actual correction image data of the first image data and a comparison result between the second correction image data and the actual correction image data of the second image data; and adjusting internal parameters of the exposure level correction unit so that the correction loss value is minimized. Claim 12 An exposure level correction device for image data comprising: a memory storing an exposure correction program; and a processor that, when image data is input from the outside, executes the exposure correction program to extract image features from the image data, extracts luminance features from the image data using a pre-trained neural network model, generates a plurality of transformed image features from the image data based on the image features and the luminance features, and corrects the exposure level of the image data based on the plurality of transformed image features to output corrected image data, wherein the image features serve as a query in a multi-head cross-attention technique and the luminance features serve as a key / value in the multi-head cross-attention technique. Claim 13 In claim 12, the processor is an exposure level correction device for image data that estimates the luminance level of the image data and extracts the luminance feature based on the estimated luminance level. Claim 14 In claim 12, the processor generates a plurality of conversion functions for each RGB channel of the image data based on the image features and the luminance features, estimates a color conversion value for each of the plurality of pixels of each RGB channel based on the plurality of conversion functions, and generates the plurality of converted image features for each RGB channel of the image data based on the estimated color conversion value. Claim 15 In claim 12, the above processor fuses the plurality of converted features to obtain a plurality of correction features for the image data, fuses the plurality of converted features and the plurality of correction features based on preset weights, and corrects the exposure level of the image data based on the fused features to generate the corrected image data. Claim 16 In claim 12, the processor is an image data exposure level correction device that adjusts the size of the image data based on a preset resolution and extracts image features from the size-adjusted image data. Claim 17 In claim 12, the neural network model comprises a luminance feature extraction unit, an image feature conversion unit, and an exposure level correction unit, wherein the neural network model comprises: the luminance feature extraction unit receiving first image data and second image data having different exposure levels and extracting a first luminance feature for the first image data and a second luminance feature for the second image data; the image feature conversion unit receiving a first image feature extracted from the first image data, a second image feature extracted from the second image data, the first luminance feature, and the second luminance feature as inputs and generating a plurality of converted first image features for the first image feature and a plurality of converted second image features for the second image feature; and the exposure level correction unit receiving the plurality of converted first image features and the plurality of converted second image features as inputs and generating a first corrected image data in which the exposure level is corrected for the first image data and a second corrected image data in which the exposure level is corrected for the second image data. Exposure level correction device for image data trained to be generated. Claim 18 A computer-readable recording medium storing a computer program, wherein the computer program includes instructions for a processor to perform a method for correcting the exposure level of image data, the step of extracting image features from input image data; the step of extracting luminance features from the image data using a previously trained neural network model; the step of generating a plurality of transformed image features from the image data based on the image features and the luminance features; and the step of correcting the exposure level of the image data based on the plurality of transformed image features and outputting corrected image data, wherein in the step of generating the plurality of transformed image features, the image features serve as a query in a multi-head cross-attention technique and the luminance features serve as a key / value in the multi-head cross-attention technique. Claim 19 A computer program stored on a computer-readable recording medium, wherein the computer program includes instructions for a processor to perform a method for correcting the exposure level of image data, the step of extracting image features from input image data; the step of extracting luminance features from the image data using a previously trained neural network model; the step of generating a plurality of transformed image features from the image data based on the image features and the luminance features; and the step of correcting the exposure level of the image data based on the plurality of transformed image features and outputting corrected image data, wherein in the step of generating the plurality of transformed image features, the image features serve as a query in a multi-head cross-attention technique and the luminance features serve as a key / value in the multi-head cross-attention technique.

Images