Tone mapping method and device, electronic equipment and storage medium

Abstract

The invention relates to a tone mapping method and device, electronic equipment and a storage medium. According to the method, a first logical operation unit, a second logical operation unit, a third logical operation unit and a lookup table of a field programmable logic gate array are matched with one another. Specifically, by inputting the Y channel median value into the lookup table, the Y channel value adjustment coefficient can be directly obtained, and exponential operation between the Y channel median value and the Y channel value adjustment coefficient does not need to be carried out. By inputting the ratio of the Y channel value of each pixel stretched to the full domain to the median value of the Y channel of each pixel stretched to the full domain into the lookup table, the compression value of the Y channel can be directly obtained, and exponential operation of the ratio and the compression value of the Y channel is not needed, so that the data processing efficiency is improved, and the operation resources are saved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a tone mapping method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of computer graphics technology, the demand for image processing is increasing. Tone mapping technology is particularly important in scenarios where high dynamic range (HDR) images are approximated on media with limited dynamic range. HDR images have a wide brightness range, enabling them to reflect real scenes more meticulously; however, they also have a relatively large data volume and are difficult to display directly on common display devices with limited dynamic range, such as cathode ray tube displays, LCD monitors, or projectors.

[0003] Tone mapping technology aims to compress the dynamic range of HDR images below that of the display device, enabling HDR images to adapt to Low Dynamic Range (LDR) displays. Specifically, tone mapping technology significantly reduces contrast to bring the scene brightness within the range that the display device can show, while preserving as much image detail and color information as possible that is crucial for representing the original scene.

[0004] In related technologies, the implementation of tone mapping methods involves a large number of complex calculations on the brightness values ​​of each pixel in the image, resulting in low data processing efficiency and a large consumption of computing resources. Summary of the Invention

[0005] Based on this, the purpose of this application is to provide a tone mapping method, apparatus, electronic device, and storage medium that can improve data processing efficiency and save computing resources.

[0006] A first aspect of this application provides a tone mapping method applied to a field-programmable gate array (FPGA). The FPGA includes a first logic operation unit, a second logic operation unit, a third logic operation unit, and a lookup table. The method includes the following steps:

[0007] The first logic operation unit performs logic operations based on the Y channel values, Y channel median values, and Y channel peak values ​​of each pixel in the image to be processed, to obtain the Y channel values ​​of each pixel after stretching to full domain and the ratio of the stretched Y channel values ​​to the Y channel median values ​​of each pixel after stretching to full domain; the Y channel values ​​of each pixel and the stretched Y channel values ​​of each pixel are sent to the second logic operation unit, and the Y channel median value and the ratio of the image to be processed are sent to the lookup table;

[0008] The lookup table searches for and outputs the Y channel value adjustment coefficient to the second logic operation unit based on the median value of the Y channel of the image to be processed; and searches for and outputs the Y channel compression value corresponding to each pixel to the second logic operation unit based on the ratio.

[0009] The second logic operation unit performs logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color preservation coefficient; and sends the color preservation coefficient to the third logic operation unit.

[0010] The third logic operation unit performs logic operations on the color preservation coefficient and the RGB data of the image to be processed to obtain tone mapping data.

[0011] A second aspect of this application provides a tone mapping apparatus, comprising:

[0012] The Y-channel stretching module is used by the first logic operation unit to perform logical operations on the Y-channel values, Y-channel median values, and Y-channel peak values ​​of each pixel in the image to be processed, to obtain the Y-channel values ​​of each pixel after stretching to full domain and the ratio of the stretched Y-channel values ​​of each pixel to the Y-channel median values ​​of each pixel after stretching to full domain; and to send the Y-channel values ​​of each pixel and the stretched Y-channel values ​​of each pixel to the second logic operation unit, and to send the Y-channel median value and the ratio of the image to be processed to a lookup table.

[0013] The Y-channel compression value output module is used to look up the Y-channel median value of the image to be processed, find and output the Y-channel value adjustment coefficient to the second logic operation unit; and to look up and output the Y-channel compression value corresponding to each pixel to the second logic operation unit according to the ratio.

[0014] The color retention coefficient acquisition module is used by the second logic operation unit to perform logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color retention coefficient; and send the color retention coefficient to the third logic operation unit.

[0015] The tone mapping data acquisition module is used by the third logic operation unit to perform logical operations on the color preservation coefficient and the RGB data of the image to be processed to obtain tone mapping data.

[0016] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it controls the device on which the computer-readable storage medium is located to implement the method described in any of the above.

[0017] A fourth aspect of this application provides a computer device including a processor, a memory, and a computer-readable program stored in the memory, wherein the computer-readable program, when executed by the processor, implements the steps of the method as described above.

[0018] The tone mapping method provided in this application is implemented through the cooperation of the first, second, and third logic operation units of a field-programmable gate array (FPGA) and a lookup table. Specifically, by inputting the Y-channel median value into the lookup table, the Y-channel value adjustment coefficient can be directly obtained without performing exponential calculations between the Y-channel median value and the Y-channel value adjustment coefficient. By inputting the ratio of the Y-channel value of each pixel stretched to full area to the Y-channel median value of each pixel stretched to full area into the lookup table, the Y-channel compressed value can be directly obtained without performing exponential calculations between the ratio and the Y-channel compressed value, thereby improving data processing efficiency and saving computational resources.

[0019] To better understand and implement this application, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0020] Figure 1 A schematic flowchart illustrating the tone mapping method provided in this application embodiment;

[0021] Figure 2 A schematic block diagram of the tone mapping device provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Wherein, when the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0024] It should be understood that the embodiments described below do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0025] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, in the description of this application, unless otherwise stated, “a plurality” means two or more. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items, for example, A and / or B, which can represent: A alone, A and B together, and B alone; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0026] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms, and these terms are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Depending on the context, the word "if" as used in this application can be interpreted as "when," "when," or "in response to determination."

[0027] Tone mapping technology aims to compress the dynamic range of HDR images below that of the display device, enabling HDR images to adapt to Low Dynamic Range (LDR) displays. Specifically, tone mapping technology significantly reduces contrast to bring the scene brightness within the range that the display device can show, while preserving as much image detail and color information as possible that is crucial for representing the original scene.

[0028] In developing this invention, the inventors discovered that tone mapping methods utilize tone mapping functions to compress the brightness values ​​of individual pixels in an image, thereby narrowing the image's brightness range to within the display device's range. These tone mapping functions are typically combinations of various basic functions, such as exponential, logarithmic, and linear functions. Therefore, calculating the compressed brightness values ​​of each pixel in an image using tone mapping functions generates numerous complex calculations involving exponential and logarithmic operations, resulting in low data processing efficiency and high computational resource consumption.

[0029] Therefore, this application implements a tone mapping method based on field-programmable gate arrays (FPGAs). Specific hardware components in the FPGA, such as lookup tables, are used to obtain brightness adjustment coefficients and brightness compression values. For example, adders, subtractors, multipliers, and dividers are used to perform addition, subtraction, multiplication, and division operations on the brightness adjustment coefficients and brightness compression values. This avoids complex operations such as exponential and logarithmic operations, thereby improving data processing efficiency and saving computing resources.

[0030] Please see Figure 1 This is a flowchart illustrating a tone mapping method provided in one embodiment of this application. The tone mapping method provided in this embodiment is applied to a field-programmable gate array (FPGA), which includes a first logic operation unit, a second logic operation unit, a third logic operation unit, and a lookup table. The method includes the following steps:

[0031] S10: The first logic operation unit performs logic operations based on the Y channel value, Y channel median value, and Y channel peak value of each pixel in the image to be processed, to obtain the Y channel value of each pixel after stretching to full domain and the ratio of the stretched Y channel value to the stretched Y channel median value of each pixel; the Y channel value of each pixel and the stretched Y channel value of each pixel are sent to the second logic operation unit, and the Y channel median value and ratio of the image to be processed are sent to the lookup table;

[0032] S20: The lookup table searches for and outputs the Y channel value adjustment coefficient to the second logic operation unit based on the median value of the Y channel of the image to be processed; and searches for and outputs the Y channel compression value corresponding to each pixel to the second logic operation unit based on the ratio.

[0033] S30: The second logic operation unit performs logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color preservation coefficient; and sends the color preservation coefficient to the third logic operation unit.

[0034] S40: The third logic operation unit performs logic operations on the color retention coefficient and the RGB data of the image to be processed to obtain tone mapping data.

[0035] The tone mapping method provided in this application is implemented through the cooperation of a first logic operation unit, a second logic operation unit, a third logic operation unit, and a lookup table within a field-programmable gate array (FPGA). Specifically, by inputting the Y-channel median value into the lookup table, the Y-channel value adjustment coefficient can be directly obtained without performing exponential calculations between the Y-channel median value and the Y-channel value adjustment coefficient. By inputting the ratio of the Y-channel value of each pixel stretched to full area to the Y-channel median value of each pixel stretched to full area into the lookup table, the Y-channel compressed value can be directly obtained without performing exponential calculations between the ratio and the Y-channel compressed value, thereby improving data processing efficiency and saving computational resources.

[0036] The following embodiments of this application use field-programmable gate arrays as the execution subject to illustrate the various steps of the tone mapping method.

[0037] For step S10, the first logic operation unit performs logic operations based on the Y channel value, Y channel median value, and Y channel peak value of each pixel in the image to be processed to obtain the Y channel value of each pixel after stretching to full domain and the ratio of the stretched Y channel value to the stretched Y channel median value of each pixel; the Y channel value of each pixel and the stretched Y channel value of each pixel are sent to the second logic operation unit, and the Y channel median value and ratio of the image to be processed are sent to the lookup table;

[0038] Among them, Field Programmable Gate Array (FPGA) is a hardware device that includes, but is not limited to, programmable input / output units, programmable logic units, clock management modules, embedded block RAM, routing resources, and low-level functional units.

[0039] The image to be processed refers to an image whose brightness dynamic range is greater than that of the display device. Therefore, tone mapping is required for the image to be processed.

[0040] It's important to note that for a single frame of an image, each pixel typically has three primary color components: red (R), green (G), and blue (B), forming the pixel's RGB color space. In the RGB color space, the display effect of a pixel can be determined by the R, G, and B values. Of course, pixels can also be divided into other primary color components, thus forming other types of color spaces, such as the CMY (cyan, magenta, yellow) color space.

[0041] In addition to the color spaces based on the three primary color components mentioned above, existing technologies also include color spaces composed of luminance and chrominance. Luminance can also be called grayscale value, and chrominance includes hue and saturation. Specifically, these color spaces include the YUV color space, the YCbCr color space (a scaled and offset version of YUV, often used in continuous image processing in films or color spaces in digital photography systems), the Hue, Saturation, Intensity (HSI) color space, and the HSV color space, etc.

[0042] In the YUV color space, Y represents luminance information, while U and V represent chrominance information.

[0043] The median value of the Y channel refers to the Y channel value corresponding to a cumulative probability of 0.5 in the Y channel histogram of the image to be processed.

[0044] The Y-channel peak value refers to the largest Y-channel value in the YUV data of the image to be processed.

[0045] In this embodiment, the image to be processed is RGB data. Through color space conversion, the RGB data of the image can be converted to YUV data, thereby obtaining the YUV data of the image to be processed in the YUV color space. Alternatively, the image to be processed may not be RGB data, but data from other color spaces; in this case, color space conversion can be used to obtain the YUV data of the image to be processed.

[0046] Specifically, the field-programmable gate array (FPGA) includes a programmable input unit and a data conversion unit. The programmable input unit receives an image to be processed, determines the data format of the image, and if the image is not YUV data, sends it to the data conversion unit. The data conversion unit converts the image to YUV data and sends the YUV data to the first logic operation unit. If the image is YUV data, it sends it to the first logic operation unit.

[0047] The first logic operation unit receives the Y channel values ​​of each pixel in the YUV data of the image to be processed, stretches the Y channel values ​​of each pixel to full range, and obtains the Y channel values ​​of each pixel after stretching to full range. The first logic operation unit obtains the median value of the Y channel of the image to be processed, stretches the median value of the Y channel to full range, and obtains the median value of the Y channel of each pixel after stretching to full range. After obtaining the Y channel values ​​of each pixel after stretching to full range and the median value of the Y channel of each pixel after stretching to full range, the ratio of the Y channel values ​​of each pixel after stretching to full range to the median value of the Y channel of each pixel after stretching to full range can be obtained.

[0048] In an optional embodiment, the field-programmable gate array further includes a read address controller, a write address controller, a dual-port memory, a counter, and an arithmetic unit. Before step S10, steps S101 to S106 are included, as follows:

[0049] S101: The address read controller uses the Y channel value of each pixel in the image to be processed as an address and reads the count value of the corresponding address from the dual-port memory; the count value is the number of pixels corresponding to the Y channel value.

[0050] The read address controller is used to read data from the dual-port memory. The dual-port memory is a true dual-port memory, supporting simultaneous reading and writing to the same address.

[0051] In this embodiment, the dual-port memory stores several arrays, the number of which is the same as the Y-channel value level of the image to be processed. Each array stores the number of pixels for one Y-channel value, and each array has an index value, which is the Y-channel level corresponding to the array. For example, if the Y-channel value level of the image to be processed is 0 to 255, then the number of arrays is 256. If an array stores Y-channel level 10, then the index value of that array is 10.

[0052] Each time the address controller receives the Y channel value of a pixel, it uses the Y channel value as an address and matches it with the index values ​​of various arrays in the dual-port memory to read the number of pixels stored in the matching array.

[0053] S102: The counter increments the count value at the corresponding address by 1 to obtain a new count value.

[0054] In this embodiment, the counter is used to update the number of pixels with the same Y channel value. For example, if the existing number of pixels with a Y channel value of 10 is 5, and the current pixel has a Y channel value of 10, then the number of pixels with a Y channel value of 10 is updated to 6.

[0055] S103: The write address controller writes the new count value to the dual-port memory.

[0056] The write address controller is used to write data to the dual-port memory.

[0057] In this embodiment, the write address controller writes the new count value to the dual-port memory, updating the pixel count in the array of the dual-port memory. For example, if the current pixel's Y channel value is 10, then the array element storing Y channel level 10 is incremented by 1.

[0058] S104: The arithmetic unit selects a Y channel value from the Y channel values ​​of each pixel as the current Y channel value, retrieves the number of pixels corresponding to the minimum Y channel value to the current Y channel value from the dual-port memory, sums the number of pixels, and obtains the total number of pixels.

[0059] In this embodiment of the application, the dual-port memory stores the number of pixels corresponding to each Y channel value. For example, if the current Y channel value is 5 and the minimum Y channel value is 0, the number of pixels corresponding to Y channel values ​​from 0 to 5 are 1, 2, 3, 4, and 5 respectively, and the total number of pixels is 15.

[0060] S105: When the total number of pixels is less than the preset number, the arithmetic unit updates the next Y channel value of the current Y channel value to the current Y channel value, and obtains the total number of pixels again until the total number of pixels is greater than or equal to the preset number, and then determines the current Y channel value as the median value of the Y channel.

[0061] The preset quantity is determined based on the total number of pixels in the image to be processed and the cumulative probability. For example, if the total number of pixels in the image to be processed is 1000 and the cumulative probability is 0.5, then the preset quantity is 500.

[0062] In this embodiment of the application, the arithmetic unit accumulates the number of pixels for each Y channel value to obtain the total number of pixels, and takes the current Y channel value when the total number of pixels is greater than or equal to a preset number as the median value of the Y channel.

[0063] S106: The arithmetic unit compares the Y channel values ​​of each pixel and takes the largest Y channel value as the peak value of the Y channel.

[0064] In this embodiment, the arithmetic unit selects two Y channel values ​​from the Y channel values ​​of each pixel for comparison, saves the larger Y channel value, and continues to compare the Y channel values ​​of all pixels to obtain the largest Y channel value.

[0065] In this embodiment of the application, since the dual-port memory supports simultaneous reading and writing to the same address, the mutual cooperation of the read address controller, write address controller, dual-port memory, counter and arithmetic unit can quickly count the number of pixels with the same Y channel value and accumulate the number of pixels, thereby improving the generation efficiency of the Y channel median value.

[0066] In an optional embodiment, the first logic operation unit includes a first shift register, a first subtractor, a first divider, and a second divider. Step S10 includes steps S11 to S14, as follows:

[0067] S11: The first shift register shifts the Y channel value and the median value of the Y channel of each pixel in the image to be processed to the left by a preset number of bits to obtain the shifted Y channel value and the shifted median value of the Y channel of each pixel.

[0068] S12: The first subtractor subtracts the corresponding Y channel value before the shift from the Y channel value of each pixel after the shift, and subtracts the median value of the Y channel before the shift from the median value of the Y channel after the shift, so as to obtain the gray level difference and the median value difference of the Y channel for each pixel respectively.

[0069] S13: The first divider divides the grayscale difference of each pixel by the peak value of the Y channel and divides the median difference of the Y channel by the peak value of the Y channel to obtain the Y channel value of each pixel after stretching to the full domain and the median value of the Y channel after stretching to the full domain, respectively.

[0070] S14: The second divider divides the Y channel value of each stretched pixel by the median value of the stretched Y channel to obtain the ratio.

[0071] The preset bit depth can be the bit width of the Y channel values ​​in the image to be processed. For example, the preset bit depth is 10.

[0072] In this embodiment, the stretching formula for the Y channel value is as follows:

[0073]

[0074] E1 represents the Y channel value of each pixel after stretching to full area, E0 represents the Y channel value of each pixel, n represents the preset bit depth, and Cmax represents the peak value of the Y channel.

[0075] The formula for stretching the median value of the Y channel is as follows:

[0076]

[0077] Thre1 represents the median value of the Y channel after stretching to full range, and Thre represents the median value of the Y channel.

[0078] Specifically, the first shift register shifts the Y-channel value of each pixel to the left by a preset number of bits n, so that the Y-channel value of each pixel is multiplied by 2 to the nth power to obtain the shifted Y-channel value of each pixel. The first shift register shifts the Y channel median value to the left by a preset number of bits n. Then, the Y channel median value is multiplied by 2 to the nth power to obtain the shifted Y channel median value Thre*2. n The first subtractor subtracts the corresponding Y-channel value before shifting from the Y-channel value of each pixel after shifting, thus obtaining the grayscale difference for each pixel. Subtracting the original Y-channel median from the shifted Y-channel median yields the Y-channel median difference, Thre*2. n -Thre. The first divider divides the grayscale difference of each pixel by the peak value of the Y channel to obtain the Y channel value E1 of each pixel after stretching to full area. It then divides the difference in the median values ​​of the Y channels by the peak value of the Y channels to obtain the median value Thre1 of the Y channels after stretching to full area. The second divider divides the Y channel value of each pixel after stretching to full area by the median value of the Y channels after stretching to full area to obtain the ratio.

[0079] The embodiments of this application, through the cooperation of the first shift register, the first subtractor, the first divider, and the second divider, can obtain the Y channel values ​​of each pixel after stretching to full domain, so that the maximum Y channel value after subsequent compression can reach the maximum brightness display capability of the display device, and make full use of the display resources of the display device.

[0080] For step S20, the lookup table searches for and outputs the Y channel value adjustment coefficient to the second logic operation unit based on the median value of the Y channel of the image to be processed; and searches for and outputs the Y channel compression value corresponding to each pixel to the second logic operation unit based on the ratio.

[0081] The lookup table comprises multiple input terminals, one output terminal, and multiple storage units. Each input terminal can take either a 0 or a 1 state, and each storage unit stores the value of a Boolean function. These values ​​correspond to specific combinations of input signals, and the states of these input signals determine the output of the lookup table. Specifically, the lookup table stores the Y-channel value adjustment coefficients corresponding to the bit values ​​in each Y-channel and the Y-channel compression values ​​corresponding to each ratio.

[0082] In this embodiment, the median value of the Y channel is used as the input to a lookup table, and the lookup table outputs the Y channel value adjustment coefficient corresponding to the Y channel median value to the second logic operation unit. The ratio is used as the input to a lookup table, and the lookup table outputs the Y channel compression value corresponding to the ratio to the second logic operation unit.

[0083] In an optional embodiment, the lookup table includes a first memory that stores a plurality of Y-channel value adjustment coefficients corresponding to the Y-channel median values. In step S20, the lookup table searches for and outputs the Y-channel value adjustment coefficients to the second logic operation unit based on the Y-channel median values ​​of the image to be processed, including step S21, as follows:

[0084] S21: The lookup table retrieves the Y-channel value adjustment coefficient corresponding to the Y-channel median value from the first memory based on the Y-channel median value, and outputs the Y-channel value adjustment coefficient to the second logic operation unit.

[0085] In this embodiment, the first memory in the lookup table stores the Y-channel value adjustment coefficients corresponding to the midpoint values ​​of each Y-channel. Specifically, the correspondence between the Y-channel midpoint values ​​and the Y-channel value adjustment coefficients is as follows:

[0086]

[0087] Where KTh represents the Y channel value adjustment coefficient, Thre represents the median value of the Y channel, and n represents the bit width of the Y channel of the image to be processed.

[0088] The microprocessor controller can calculate the Y-channel value adjustment coefficient corresponding to the bit value of each Y-channel according to the above correspondence, and then write the Y-channel value adjustment coefficient corresponding to the bit value of each Y-channel into the first memory of the lookup table.

[0089] Performing exponential operations on the Y-channel median to obtain the Y-channel value adjustment coefficient consumes significant computational resources. This embodiment of the application uses a lookup table to obtain the Y-channel value adjustment coefficient based on the Y-channel median, thus saving computational resources.

[0090] In an optional embodiment, the lookup table includes a second memory that stores multiple Y-channel compressed values ​​corresponding to ratios. Step S20 involves searching for and outputting the Y-channel compressed value corresponding to each pixel to the second logic operation unit based on the ratio, including step S22, as follows:

[0091] S22: The lookup table searches for the Y-channel compressed value corresponding to the ratio in the second memory, and outputs the Y-channel compressed value to the second logic operation unit.

[0092] In this embodiment, the second memory in the lookup table stores the Y-channel compression values ​​corresponding to each ratio. Specifically, the correspondence between the ratios and the Y-channel compression values ​​is as follows:

[0093]

[0094] Where Thre1 / E1 represents the ratio, E2 represents the Y channel compression value, and α represents a fixed constant with a specific value of 6.

[0095] The microprocessor controller can calculate the Y-channel compression value corresponding to each ratio based on the above correspondence, and then write the Y-channel compression value corresponding to each ratio into the second memory of the lookup table.

[0096] Performing exponential operations on the comparison values ​​to obtain the compressed Y-channel value consumes significant computational resources. This embodiment of the application uses a lookup table to obtain the compressed Y-channel value based on the ratio, thus saving computational resources.

[0097] For step S30, the second logic operation unit performs logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color preservation coefficient; and sends the color preservation coefficient to the third logic operation unit.

[0098] In this embodiment, the second logic operation unit performs logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, and the stretched Y channel value of each pixel to obtain the compressed Y channel value of each pixel. The second logic operation unit performs logical operations on the compressed Y channel value of each pixel and the Y channel value of each pixel to obtain the color preservation coefficient, and sends the color preservation coefficient to the third logic operation unit.

[0099] In an optional embodiment, the second logic operation unit includes a first multiplier, a first adder, and a third divider. In step S30, the second logic operation unit performs logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color retention coefficient, including steps S31 to S33, as follows:

[0100] S31: The first multiplier multiplies the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, and the first weight coefficient to obtain the first product result; it multiplies the stretched Y channel value of each pixel with the second weight coefficient to obtain the second product result; wherein, the first weight coefficient is determined based on the peak brightness of the image to be processed and the peak brightness of the display device; the second weight coefficient is the sum of the negative of the first weight coefficient and 1.

[0101] In this embodiment, the first weighting coefficient is the product of the difference between the peak brightness of the image to be processed and the peak brightness of the display device, and a preset proportional coefficient. The preset proportional coefficient can be set manually according to actual needs, or it can be determined based on the brightness and perceptual quantization curve of the display device.

[0102] Specifically, the expression for the first weighting coefficient is as follows:

[0103] W = (Cmax - Dmax) / (1 - OE(100))

[0104] Where W represents the first weighting coefficient, Cmax represents the peak brightness of the image to be processed, Dmax represents the peak brightness of the display device, and OE(100) represents the value of the perceptual quantization curve when the brightness of the display device is 100 nits. Specifically, OE(100) is 0.5081.

[0105] S32: The first adder sums the first product result with the second product result to obtain the compressed Y channel value of each pixel.

[0106] In this embodiment, the expression for the compressed Y channel value of each pixel is as follows:

[0107] E4=W*KTh*E2+(1-W)*E1

[0108] E4 represents the compressed Y channel value of each pixel, W represents the first weighting coefficient, and 1-W represents the second weighting coefficient.

[0109] S33: The third divider divides the compressed Y channel value of each pixel by the Y channel value of each pixel to obtain the color preservation coefficient.

[0110] In this embodiment of the application, the expression for the color retention coefficient is:

[0111]

[0112] Where K represents the color retention coefficient.

[0113] The embodiments of this application can automatically and quickly obtain the color preservation coefficient through the cooperation of the first multiplier, the first adder and the third divider, which facilitates the subsequent color preservation processing of the image to be processed.

[0114] In step S40, the third logic operation unit performs logic operations on the color preservation coefficient and the RGB data of the image to be processed to obtain tone mapping data.

[0115] In this embodiment of the application, after the Y channel value is compressed, in order to ensure that the color does not have a large deviation, for each pixel, the RGB data of the image to be processed is processed proportionally according to the change of the Y channel value to obtain tone mapping data.

[0116] In an optional embodiment, the third logic operation unit includes a second multiplier and a fourth divider. Step S40 includes steps S41 to S42, as follows:

[0117] S41: The fourth divider divides the peak brightness of the image to be processed by the peak brightness of the display device to obtain the maximum brightness coefficient;

[0118] S42: The second multiplier multiplies the color preservation coefficient, the maximum brightness coefficient, and the RGB values ​​of each pixel in the RGB data of the image to be processed to obtain tone mapping data.

[0119] In this embodiment of the application, the expression for tone mapping data is as follows:

[0120] Rt = K * L * R,

[0121] Gt = K * L * G,

[0122] Bt = K * L * B,

[0123] L = Cmax / Dmax

[0124] Where R, G, and B represent the RGB values ​​of each pixel in the RGB data, K is the color retention coefficient, L is the maximum brightness coefficient, and Rt, Gt, and Bt represent the RGB values ​​of each pixel in the tone mapping data.

[0125] This application embodiment uses the cooperation of a second multiplier and a fourth divider to convert RGB data into tone mapping data, which can eliminate color deviation caused by brightness changes and improve the color display effect of tone mapping images.

[0126] The following are embodiments of the apparatus described in this application, which can be used to execute the methods described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the methods described in the embodiments of this application.

[0127] Please see Figure 2 This illustration shows a schematic diagram of the tone mapping device provided in an embodiment of this application. The tone mapping device 5 provided in an embodiment of this application includes:

[0128] Y-channel stretching module 51 is used by the first logic operation unit to perform logical operations on the Y-channel values, Y-channel median values, and Y-channel peak values ​​of each pixel in the image to be processed, to obtain the Y-channel values ​​of each pixel after stretching to full domain and the ratio of the Y-channel values ​​of each pixel after stretching to full domain to the Y-channel median values ​​of each pixel after stretching to full domain; and to send the Y-channel values ​​of each pixel and the Y-channel values ​​of each pixel after stretching to the second logic operation unit, and to send the Y-channel median values ​​and the ratio of the image to be processed to a lookup table.

[0129] Y-channel compression value output module 52 is used to look up the Y-channel value adjustment coefficient to the second logic operation unit based on the median value of the Y-channel of the image to be processed; and to look up and output the Y-channel compression value corresponding to each pixel to the second logic operation unit based on the ratio.

[0130] The color retention coefficient acquisition module 53 is used by the second logic operation unit to perform logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color retention coefficient; and send the color retention coefficient to the third logic operation unit.

[0131] The tone mapping data acquisition module 54 is used by the third logic operation unit to perform logical operations on the color preservation coefficient and the RGB data of the image to be processed to obtain tone mapping data.

[0132] The tone mapping method provided in this application utilizes the cooperation of a first logic operation unit, a second logic operation unit, a third logic operation unit, and a lookup table within a field-programmable gate array (FPGA). Specifically, by inputting the Y-channel median value into the lookup table, the Y-channel value adjustment coefficient can be directly obtained without performing exponential operations between the Y-channel median value and the Y-channel value adjustment coefficient. Similarly, by inputting the ratio of the Y-channel value of each pixel stretched to full area to the Y-channel median value of each pixel stretched to full area into the lookup table, the Y-channel compressed value can be directly obtained without performing exponential operations between the ratio and the Y-channel compressed value, thereby improving data processing efficiency and saving computational resources.

[0133] The following are embodiments of the device described in this application, which can be used to execute the methods described in the embodiments of this application. For details not disclosed in the embodiments of the device described in this application, please refer to the methods described in the embodiments of this application.

[0134] Please see Figure 3 This application also provides an electronic device 300, which may specifically be a computer, mobile phone, tablet computer, tone mapping device, etc. In an exemplary embodiment of this application, the electronic device 300 is an image tone mapping processing device, which may include: at least one processor 301, at least one memory 302, at least one display, at least one network interface 303, user interface 304, and at least one communication bus 305.

[0135] The user interface 304 is primarily used to provide an input interface for the user and to acquire user input data. Optionally, the user interface may also include a standard wired interface or a wireless interface.

[0136] The network interface 303 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0137] The communication bus 305 is used to enable communication between these components.

[0138] The processor 301 may include one or more processing cores. The processor connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0139] The memory 302 may include random access memory (RAM) or read-only memory. Optionally, the memory may include a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. The memory may also optionally be at least one storage device located remotely from the aforementioned processor. Figure 3 As shown, a memory, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and operating applications.

[0140] The processor can be used to call the application program that stores the tone mapping method in the memory and specifically execute the method steps of the above embodiment. For the specific execution process, please refer to the detailed description shown in the embodiment, which will not be repeated here.

[0141] This application also provides a computer-readable storage medium storing a computer program thereon, the instructions of which are adapted to be loaded by a processor and executed by the method steps of the embodiments shown above. For details of the execution process, please refer to the specific descriptions shown in the embodiments, which will not be repeated here. The device containing the storage medium can be an electronic device such as a personal computer, laptop computer, smartphone, or tablet computer.

[0142] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0143] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function selected in one or more boxes.

[0145] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function selected in one or more boxes.

[0146] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0147] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0148] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape storage, disk storage, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0149] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0150] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A tone mapping method, characterized in that, The method is applied to field-programmable gate arrays (FPGAs), which include a first logic operation unit, a second logic operation unit, a third logic operation unit, and a lookup table; the method includes the following steps: The first logic operation unit performs logic operations based on the Y channel values, Y channel median values, and Y channel peak values ​​of each pixel in the image to be processed, to obtain the Y channel values ​​of each pixel after stretching to full domain and the ratio of the stretched Y channel values ​​of each pixel to the Y channel median values ​​of each pixel after stretching to full domain; the Y channel values ​​of each pixel and the stretched Y channel values ​​of each pixel are sent to the second logic operation unit, and the Y channel median values ​​of the image to be processed and the ratio are sent to the lookup table; The lookup table searches for and outputs the Y channel value adjustment coefficient to the second logic operation unit based on the median value of the Y channel of the image to be processed. Based on the ratio, the Y channel compression value corresponding to each pixel is found and output to the second logic operation unit; The second logic operation unit performs logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color preservation coefficient; and sends the color preservation coefficient to the third logic operation unit. The third logic operation unit performs logic operations on the color retention coefficient and the RGB data of the image to be processed to obtain tone mapping data.

2. The tone mapping method according to claim 1, characterized in that: The first logic operation unit includes a first shift register, a first subtractor, a first divider, and a second divider; The first logic operation unit performs logic operations based on the Y-channel values, Y-channel median values, and Y-channel peak values ​​of each pixel in the image to be processed to obtain the Y-channel values ​​of each pixel after stretching to full domain and the ratio of the stretched Y-channel values ​​of each pixel to the Y-channel median values ​​of each pixel after stretching to full domain. The steps include: The first shift register shifts the Y channel value and the median value of the Y channel of each pixel in the image to be processed to the left by a preset number of bits to obtain the shifted Y channel value and the shifted median value of the Y channel of each pixel. The first subtractor subtracts the corresponding Y channel value before the shift from the Y channel value of each pixel after the shift, and subtracts the median value of the Y channel before the shift from the median value of the Y channel after the shift, so as to obtain the grayscale difference and the Y channel median difference for each pixel respectively. The first divider divides the grayscale difference corresponding to each pixel by the peak value of the Y channel, and divides the median difference of the Y channel by the peak value of the Y channel, to obtain the Y channel value of each pixel after stretching to full domain and the median value of the Y channel after stretching to full domain, respectively. The second divider divides the Y channel value of each stretched pixel by the median value of the stretched Y channel to obtain the ratio.

3. The tone mapping method according to claim 1, characterized in that: The second logic operation unit includes a first multiplier, a first adder, and a third divider; The second logic operation unit performs logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color retention coefficient, including: The first multiplier multiplies the Y-channel value adjustment coefficient, the Y-channel compression value corresponding to each pixel, and a preset first weighting coefficient to obtain a first product result; it multiplies the stretched Y-channel value of each pixel with a preset second weighting coefficient to obtain a second product result; wherein, the first weighting coefficient is determined based on the peak brightness of the image to be processed and the peak brightness of the display device; the second weighting coefficient is the sum of the negative of the first weighting coefficient and 1; The first adder sums the first product result with the second product result to obtain the compressed Y channel value of each pixel; The third divider divides the compressed Y channel value of each pixel by the Y channel value of each pixel to obtain the color retention coefficient.

4. The tone mapping method according to claim 1, characterized in that: The lookup table includes a first memory, which stores a plurality of Y-channel value adjustment coefficients corresponding to the bit values ​​of the Y-channels; The step of finding and outputting the Y-channel value adjustment coefficient to the second logic operation unit based on the Y-channel median value of the image to be processed includes: The lookup table retrieves the Y-channel value adjustment coefficient corresponding to the Y-channel median value from the first memory based on the Y-channel median value, and outputs the Y-channel value adjustment coefficient to the second logic operation unit.

5. The tone mapping method according to claim 1, characterized in that: The lookup table includes a second memory, which stores multiple Y-channel compression values ​​corresponding to ratios; The step of finding and outputting the Y-channel compressed value corresponding to each pixel to the second logic operation unit according to the ratio includes: The lookup table retrieves the Y-channel compressed value corresponding to the ratio from the second memory based on the ratio, and outputs the Y-channel compressed value to the second logic operation unit.

6. The tone mapping method according to claim 1, characterized in that: The third logic operation unit includes a second multiplier and a fourth divider; The step of the third logic operation unit performing logical operations on the color retention coefficient and the RGB data of the image to be processed to obtain tone mapping data includes: The fourth divider divides the peak brightness of the image to be processed by the peak brightness of the display device to obtain the maximum brightness coefficient; The second multiplier multiplies the color preservation coefficient, the maximum brightness coefficient, and the RGB values ​​of each pixel in the RGB data of the image to be processed to obtain tone mapping data.

7. The tone mapping method according to claim 1, characterized in that: The field-programmable gate array also includes a read address controller, a write address controller, a dual-port memory, a counter, and an arithmetic unit; Before the first logic operation unit performs logic operations based on the Y-channel values, Y-channel median values, and Y-channel peak values ​​of each pixel in the image to be processed to obtain the Y-channel values ​​of each pixel after stretching to full domain and the ratio of the stretched Y-channel values ​​of each pixel to the Y-channel median values ​​of each pixel after stretching to full domain, the following steps are included: The read address controller uses the Y channel value of each pixel in the image to be processed as an address and reads the count value of the corresponding address from the dual-port memory; the count value is the number of pixels corresponding to the Y channel value. The counter increments the count value at the corresponding address by 1 to obtain a new count value; The write address controller writes the new count value into the dual-port memory; The arithmetic unit selects a Y channel value from the Y channel values ​​of each pixel as the current Y channel value, retrieves the number of pixels from the minimum Y channel value to the current Y channel value from the dual-port memory, sums the number of pixels, and obtains the total number of pixels. When the total number of pixels is less than a preset number, the arithmetic unit updates the next Y channel value of the current Y channel value to the current Y channel value, and obtains the total number of pixels again until the total number of pixels is greater than or equal to the preset number, and then determines the current Y channel value as the median value of the Y channel. The arithmetic unit compares the Y channel values ​​of each pixel and takes the largest Y channel value as the peak value of the Y channel.

8. A tone mapping device, characterized in that, include: The Y-channel stretching module is used by the first logic operation unit to perform logical operations on the Y-channel values, Y-channel median values, and Y-channel peak values ​​of each pixel in the image to be processed, to obtain the Y-channel values ​​of each pixel after stretching to full domain and the ratio of the stretched Y-channel values ​​of each pixel to the Y-channel median values ​​of each pixel after stretching to full domain; to send the Y-channel values ​​of each pixel and the stretched Y-channel values ​​of each pixel to the second logic operation unit, and to send the Y-channel median value of the image to be processed and the ratio to the lookup table; The Y-channel compression value output module is used to find and output the Y-channel value adjustment coefficient to the second logic operation unit based on the Y-channel median value of the image to be processed by the lookup table. Based on the ratio, the Y channel compression value corresponding to each pixel is found and output to the second logic operation unit; The color retention coefficient acquisition module is used by the second logic operation unit to perform logical operations on the Y channel value adjustment coefficient, the Y channel compression value corresponding to each pixel, the stretched Y channel value of each pixel, and the Y channel value of each pixel to obtain the color retention coefficient; and to send the color retention coefficient to the third logic operation unit. The tone mapping data acquisition module is used by the third logic operation unit to perform logical operations on the color preservation coefficient and the RGB data of the image to be processed to obtain tone mapping data.

9. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed, controls the device containing the computer-readable storage medium to implement the method as described in any one of claims 1-7.

10. An electronic device, characterized in that, It includes a processor, a memory, and a computer-readable program stored in the memory, which, when executed by the processor, implements the steps of the method as described in any one of claims 1-7.

Images