Image processing method and device, electronic equipment, storage medium and chip

Abstract

The invention provides an image processing method, and the method comprises the steps: carrying out the conversion processing of a to-be-processed YUV image in a first color gamut range, obtaining an RGB image in a second color gamut range, generating a target mapping function, and enabling the target mapping function to be a function obtained through the fusion processing of a first mapping function and a second mapping function, the first mapping function is a function obtained by performing histogram statistics according to a Y channel component of the to-be-processed YUV image, and the second mapping function is a preset function, and performing mapping processing on an image in the to-be-played video based on the target mapping function to obtain a target RGB image. According to the scheme, the RGB pixels are combined with the brightness, the brightness of the image after tone mapping is increased, and the image quality of the image is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to an image processing method and apparatus, an electronic device, a storage medium, and a chip. Background Art

[0002] With the development of audio-visual technology, people's demand for high-quality videos is increasing. HDR (high dynamic range) videos are gradually entering the consumer field. The main difference between HDR videos and SDR (standard dynamic range) videos is that HDR videos can record and display a wider brightness range, bringing a more realistic visual experience. After the brightness range of HDR videos and SDR videos changes, a new mapping curve is required for the conversion of optical signals to electrical signals.

[0003] When generating the mapping function, the statistical histograms of the RGB three channels are calculated. According to the areas below the upper limit of dark pixels and above the lower limit of bright pixels that have been calculated, the original statistical histogram is cropped, and the remaining histogram is re-normalized to obtain an effective histogram. According to the proportions of high-brightness, medium-brightness, and low-brightness pixels in the image, two inflection points are calculated. Based on the two calculated inflection points, and (0, 0), (0, 1), the coefficients of the cubic interpolation function are determined to obtain the final mapping function.

[0004] When calculating the statistical histogram in this solution, the histograms of the RGB three channels are directly superimposed as the final effective histogram, resulting in inaccurate mapping of dark area pixels and problems such as being too dark or too black. Summary of the Invention

[0005] The present disclosure provides an image processing method and apparatus, an electronic device, a storage medium, and a chip to solve the problems in the related art, combine RGB pixels with brightness, increase the brightness of the image after tone mapping, and improve the image quality.

[0006] The first aspect embodiment of the present disclosure provides an image processing method, which includes:

[0007] Performing conversion processing on a to-be-processed YUV image in a first color gamut range to obtain an RGB image in a second color gamut range;

[0008] Generating a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function, the first mapping function is a function obtained by performing histogram statistics on the Y channel component of the to-be-processed YUV image, and the second mapping function is a preset function;

[0009] Perform mapping processing on the images in the video to be played based on the target mapping function to obtain the target RGB image.

[0010] In some embodiments of the present disclosure, the generating the target mapping function includes:

[0011] Decode the video stream to be processed into the YUV image to be processed, and determine the maximum pixel value in the YUV image to be processed;

[0012] Perform normalization processing on each pixel value in the YUV image to be processed based on the maximum pixel value to obtain the normalized pixel value;

[0013] Calculate the second mapping function based on the normalized pixel value;

[0014] Perform histogram statistics on the Y-channel component of the YUV image to be processed, and preprocess the histogram to obtain the first mapping function;

[0015] Fuse the first mapping function and the second mapping function to obtain the target mapping function.

[0016] In some embodiments of the present disclosure, the preprocessing the histogram to obtain the first mapping function includes:

[0017] According to the preset correspondence between the Y-channel component and the RGB component, determine the conversion coefficients of the R component, the G component, and the B component respectively;

[0018] Adjust the number of each pixel in the histogram according to the conversion coefficients of the R component, the G component, and the B component to obtain the adjusted histogram;

[0019] Accumulate the adjusted histogram according to the pixel value size to obtain the cumulative histogram function;

[0020] Perform normalization processing on the cumulative histogram function to obtain the first mapping function.

[0021] In some embodiments of the present disclosure, the performing normalization processing on the cumulative histogram function to obtain the first mapping function includes:

[0022] Perform smoothing processing on the normalized histogram function to obtain the smoothed histogram mapping function;

[0023] Obtain the historical mapping function, where the historical mapping function is the mapping function corresponding to the previous adjacent YUV image of the YUV image to be processed;

[0024] Fuse the historical mapping function with the smoothed histogram mapping function to obtain a final first mapping function.

[0025] In some embodiments of the present disclosure, the fusing the first mapping function and the second mapping function to obtain the target mapping function includes:

[0026] Fuse the first mapping function and the second mapping function according to a preset first fusion coefficient and a preset second fusion coefficient to obtain the target mapping function, where the first fusion coefficient is the fusion coefficient of the first mapping function and the second fusion coefficient is the fusion coefficient of the second mapping function.

[0027] In some embodiments of the present disclosure, after performing mapping processing on the images in the video to be played based on the target mapping function to obtain a target RGB image, the method further includes:

[0028] Convert the target RGB image into an HDR image adapted to the display attribute information of the display device to display the HDR image.

[0029] A second aspect embodiment of the present disclosure provides an image processing apparatus, the apparatus includes:

[0030] A first conversion unit, configured to perform conversion processing on a to-be-processed YUV image in a first color gamut range to obtain an RGB image in a second color gamut range;

[0031] A generation unit, configured to generate a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function, the first mapping function is a function obtained by performing histogram statistics on the Y channel component of the to-be-processed YUV image, and the second mapping function is a preset function;

[0032] A processing unit, configured to perform mapping processing on the images in the video to be played based on the target mapping function to obtain a target RGB image.

[0033] In some embodiments of the present disclosure, the generation unit includes:

[0034] A decoding module, configured to decode a to-be-processed video stream into the to-be-processed YUV image;

[0035] A determination module, configured to determine the maximum pixel value in the to-be-processed YUV image;

[0036] A first processing module, configured to perform normalization processing on each pixel value in the to-be-processed YUV image based on the maximum pixel value to obtain a normalized pixel value;

[0037] The calculation module, Yang Hongyu calculates the second mapping function based on the normalized pixel values;

[0038] The statistics module is used to perform histogram statistics on the Y-channel component of the to-be-processed YUV image;

[0039] The second processing module is used to preprocess the histogram to obtain the first mapping function;

[0040] The third processing module is used to fuse the first mapping function and the second mapping function to obtain the target mapping function.

[0041] In some embodiments of the present disclosure, the second processing module is further used for:

[0042] According to the preset correspondence between the Y-channel component and the RGB components, respectively determine the conversion coefficients of the R component, the G component, and the B component;

[0043] Adjust the number of each pixel of the histogram according to the conversion coefficients of the R component, the G component, and the B component to obtain an adjusted histogram;

[0044] Accumulate the adjusted histogram according to the pixel value size to obtain an accumulated histogram function;

[0045] Normalize the accumulated histogram function to obtain the first mapping function.

[0046] In some embodiments of the present disclosure, the second processing module is further used for:

[0047] Smooth the histogram function after normalization to obtain a smoothed histogram mapping function;

[0048] Obtain a historical mapping function, where the historical mapping function is the mapping function corresponding to the previous adjacent YUV image of the to-be-processed YUV image;

[0049] Fuse the historical mapping function and the smoothed histogram mapping function to obtain a final first mapping function.

[0050] In some embodiments of the present disclosure, the third processing module is further used to fuse the first mapping function and the second mapping function according to a preset first fusion coefficient and a preset second fusion coefficient to obtain the target mapping function, where the first fusion coefficient is the fusion coefficient of the first mapping function, and the second fusion coefficient is the fusion coefficient of the second mapping function.

[0051] In some embodiments of the present disclosure, the device further includes:

[0052] A second conversion unit, configured to, after performing mapping processing on an image in the video to be played based on the target mapping function to obtain a target RGB image, convert the target RGB image into an HDR image adapted to the display attribute information of a display device, so as to display the HDR image.

[0053] An embodiment of the third aspect of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method described in the embodiment of the first aspect of the present disclosure.

[0054] An embodiment of the fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method described in the embodiment of the first aspect of the present disclosure.

[0055] An embodiment of the fifth aspect of the present disclosure provides a chip, the chip includes one or more interfaces and one or more processors; the interfaces are configured to receive signals from a memory of an electronic device and send signals to the processors, the signals include computer instructions stored in the memory, and when the processors execute the computer instructions, the electronic device is caused to execute the method described in the embodiment of the first aspect of the present disclosure.

[0056] In summary, according to the image processing method proposed by the present disclosure, the method includes performing conversion processing on a to-be-processed YUV image in a first color gamut range to obtain an RGB image in a second color gamut range, generating a target mapping function, the target mapping function being a function obtained by fusing a first mapping function and a second mapping function, the first mapping function being a function obtained by performing histogram statistics on the Y channel component of the to-be-processed YUV image, the second mapping function being a preset function, and performing mapping processing on an image in the video to be played based on the target mapping function to obtain a target RGB image. The solution of the present disclosure combines RGB pixels with brightness, increases the brightness of the image after tone mapping, and improves the image quality.

[0057] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure and do not constitute an improper limitation to the present disclosure.

[0059] Figure 1Flow chart of an image processing method provided by an embodiment of the present disclosure;

[0060] Figure 2 Flow chart of an image processing method provided by an embodiment of the present disclosure;

[0061] Figure 3 Flow chart of an image processing method provided by an embodiment of the present disclosure;

[0062] Figure 4 Flow chart of an image processing method provided by an embodiment of the present disclosure;

[0063] Figure 5 Schematic diagram of image processing provided by an embodiment of the present disclosure;

[0064] Figure 6 Schematic diagram of the structure of an image processing apparatus provided by an embodiment of the present disclosure;

[0065] Figure 7 Schematic diagram of the structure of an image processing apparatus provided by an embodiment of the present disclosure;

[0066] Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0067] Figure 9 Schematic diagram of the structure of a chip provided by an embodiment of the present disclosure. Detailed implementation manners

[0068] The embodiments of the present disclosure will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present disclosure, but should not be construed as a limitation to the present disclosure.

[0069] With the development of audio-visual technology, people's demand for high-quality videos is increasing. HDR (high dynamic range) videos are gradually entering the consumer field. The main difference between HDR videos and SDR (standard dynamic range) videos is that HDR videos can record and display a wider brightness range, bringing a more realistic visual experience. After the brightness range of HDR videos and SDR videos changes, a new mapping curve is required for the conversion of optical signals to electrical signals.

[0070] When generating the mapping function, calculate the statistical histograms of the three RGB channels. According to the calculated regions below the upper limit of dark pixels and above the lower limit of bright pixels, crop the original statistical histogram, re-normalize the remaining histogram to obtain an effective histogram. Calculate two inflection points based on the proportions of high-brightness, medium-brightness, and low-brightness pixels in the image. Based on the two calculated inflection points, as well as (0, 0) and (0, 1), determine the coefficients of the cubic interpolation function to obtain the final mapping function.

[0071] When calculating the statistical histogram in this solution, directly superimpose the histograms of the three RGB channels as the final effective histogram, resulting in inaccurate mapping of dark-region pixels and problems such as being too dark or too black.

[0072] Therefore, to solve the problems existing in the related art, the present disclosure proposes an image processing method. The YUV image to be processed in the first color gamut range is converted to obtain an RGB image in the second color gamut range, and a target mapping function is generated. The target mapping function is a function obtained by fusing a first mapping function and a second mapping function. The first mapping function is a function obtained by performing histogram statistics on the Y channel component of the YUV image to be processed, and the second mapping function is a preset function. Based on the target mapping function, map the images in the video to be played to obtain target RGB images. During the image processing, the brightness dimension is introduced when generating the target mapping function, and processing is performed based on this target mapping function.

[0073] This solution introduces the brightness dimension when generating the target mapping function and performs processing based on this target mapping function, that is, combines RGB pixels with brightness, increases the brightness of the image after tone mapping, and improves the image quality.

[0074] The embodiments of the present disclosure are not exhaustive, but only illustrate some embodiments and do not constitute specific limitations on the protection scope of the present disclosure. Without contradiction, each step in an embodiment can be implemented as an independent embodiment, and the steps can be combined arbitrarily. For example, the solution after removing some steps in an embodiment can also be implemented as an independent embodiment, and the order of the steps in an embodiment can be arbitrarily exchanged. Additionally, the optional implementation methods in an embodiment can be combined arbitrarily; furthermore, the embodiments can be combined arbitrarily. For example, some or all of the steps of different embodiments can be combined arbitrarily, and an embodiment can be combined arbitrarily with the optional implementation methods of other embodiments.

[0075] In each embodiment of the present disclosure, without special instructions and logical conflicts, the terms and / or descriptions between the embodiments are consistent and can be referenced to each other. The technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0076] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure.

[0077] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular form, such as "a", "an", "the", "above", "the foregoing", "the aforementioned", "this", etc., may mean "one and only one", or may also mean "one or more", "at least one", etc. For example, in the case of using articles such as "a", "an", "the" in English translation, the noun after the article can be understood as a singular expression or a plural expression.

[0078] In some embodiments, terms such as "in response to...", "in response to determining...", "in the case of...", "when...", "while...", "if...", etc. may be interchangeable.

[0079] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not lower than", "above", etc. may be interchangeable, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", "below", etc. may be interchangeable.

[0080] Prefix words such as "first", "second", etc. in the embodiments of the present disclosure are only used to distinguish different described objects, and do not constitute a limitation on the position, order, priority, quantity, content, etc. of the described objects. The description of the described objects refers to the description in the claims or the context of the embodiments, and should not constitute an unnecessary limitation due to the use of prefix words.

[0081] In the embodiments of the present disclosure, "a plurality of" means two or more.

[0082] In the embodiments of the present disclosure, terms such as "import", "input", "read in", etc. may be interchangeable.

[0083] In some embodiments, a device, etc. can be interpreted as physical or virtual, and its name is not limited to the name recorded in the embodiments. Terms such as "device", "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", "subject", etc. may be interchangeable.

[0084] In some embodiments, terms such as "terminal", "terminal device", "user equipment (UE)", "user terminal", "mobile station (MS)", "mobile terminal (MT)", subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, etc. may be used interchangeably.

[0085] Figure 1 The figure is a flowchart of an image processing method provided by an embodiment of the present disclosure. This method can be applicable to application scenarios such as smart terminals. For example, it can be executed by a terminal integrated with an image processing function or an image processor in the terminal, or by a server suitable for performing image processing. The present disclosure does not limit this. As Figure 1 shown, this image processing method includes steps 101-103.

[0086] Step 101: Perform conversion processing on the YUV image to be processed in the first color gamut range to obtain an RGB image in the second color gamut range.

[0087] In some embodiments, when the display does not support a high-definition format (such as the HDR10+ video format), conversion needs to be performed through the method described in the embodiments of the present disclosure to achieve the effect of correctly displaying the HDR10+ video format.

[0088] In some embodiments, when playing a video to be processed on a display device, HDR-to-SDR content mapping is performed on the device side or the server side. For example, the HDR video uses the smpte2086 mapping curve. For a display screen that does not support this mapping curve, a data processor (DisPlay Unit, DPU) is required to convert this mapping so that the display screen can correctly display the HDR video content.

[0089] Before playing the video to be processed, the video to be processed is decoded by a decoder to obtain a YUV image to be processed. The YUV image to be processed is YUV data in a non-linear first color gamut range. In some embodiments, the first color gamut range includes but is not limited to the BT2020 color gamut.

[0090] The YUV data in the non-linear first color gamut range is converted by a preset conversion matrix to obtain an RGB image in the non-linear first color gamut range; the preset conversion matrix described in the embodiments of the present disclosure includes but is not limited to the YUV2RGB conversion matrix.

[0091] The RGB image in the non-linear first color gamut range is processed by an electro-optical transfer function (Electro-Optical Transfer Function, EOTF) to obtain an RGB image in the linear first color gamut range; in the linear space, the RGB image is subjected to color gamut conversion using a color gamut conversion matrix to convert the first color gamut range to a second color gamut range, obtaining an RGB image in the linear second color gamut range; in the embodiments of the present disclosure, the second color gamut range includes but is not limited to the BT709 color gamut.

[0092] Step 102, generate a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function. The first mapping function is a function obtained by performing histogram statistics on the Y channel component of the YUV image to be processed, and the second mapping function is a preset function.

[0093] For the YUV image to be processed decoded in step 101, histogram statistics are performed on the Y channel component of the YUV image to be processed, and a second mapping function is calculated. For the RGB image in the second color gamut range, after fusing the first mapping function and the second mapping function according to the calculated second mapping function, a target mapping function is obtained.

[0094] Step 103, perform mapping processing on the images in the video to be played based on the target mapping function to obtain a target RGB image.

[0095] Perform mapping on each frame of the video to be processed based on the calculated target mapping function to obtain a target RGB image in the linear second color gamut range (BT709 color gamut).

[0096] In summary, according to the image processing method proposed by the present disclosure, the method includes performing conversion processing on a to-be-processed YUV image in a first color gamut range to obtain an RGB image in a second color gamut range, and generating a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function. The first mapping function is a function obtained by performing histogram statistics on the Y channel component of the to-be-processed YUV image, and the second mapping function is a preset function. Based on the target mapping function, mapping processing is performed on the images in the to-be-played video to obtain a target RGB image. The solution of the present disclosure combines RGB pixels with luminance, increases the luminance of the image after tone mapping, and improves the image quality.

[0097] Figure 2 Further, a flowchart of an image processing method proposed by the present disclosure is shown. Based on Figure 1 the shown embodiment, step 102 is further explained. Figure 2 It may include the following steps:

[0098] Step 201, decode the to-be-processed video stream into the to-be-processed YUV image, and determine the maximum pixel value in the to-be-processed YUV image.

[0099] In some embodiments, the to-be-processed YUV image output by the decoder is represented as I, the pixel value in the to-be-processed YUV image is represented as i, and the maximum pixel value is represented as M.

[0100] Step 202, perform normalization processing on each pixel value in the to-be-processed YUV image based on the maximum pixel value to obtain a normalized pixel value.

[0101] Based on the determined maximum pixel value for normalization processing, t is the normalized pixel value, t = i / M.

[0102] Step 203, calculate the second mapping function based on the normalized pixel value.

[0103] Based on the normalized pixel value t, calculate the mapping curve of the second mapping function. This mapping curve can be any of the following mapping curves for conversion algorithms, such as the Reinhard tone mapping, Filmic tone mapping, ACE Stone mapping, etc. mapping curves.

[0104] In the embodiments of the present disclosure, taking the ACES mapping curve as an example, the calculation formula is as follows:

[0105]

[0106] Among them, A = 2.51, B = 0.03, C = 2.43, D = 0.59, E = 0.14. Among them, ABCD are all pre-set parameters. By inputting the normalized pixel value t into formula (1), the mapping curve of the image in the video to be played can be obtained.

[0107] It should be noted that the above ACES mapping curve is only an exemplary illustration, rather than a limitation on the second mapping function.

[0108] Apply the second mapping curve to the adjustment of each pixel value of the j-th frame image (i.e., the current image) to obtain the output result Result of the j-th frame image. Base 。

[0109] Step 204: Perform histogram statistics on the Y-channel component of the YUV image to be processed, and preprocess the histogram to obtain the first mapping function.

[0110] According to the YUV image to be processed output by the decoder, perform histogram statistics on the Y-channel component (luminance component).

[0111] In some embodiments, for an image I with a length of W and a height of H, the total number of its pixels is N = W × H. Let n i represent the number of pixels with a pixel value of i in the image, then where M is the maximum pixel value of the pixels in the image.

[0112] The histogram is represented by the following formula (2):

[0113] Hist = n i , i ∈ [0, M] (Formula 2)

[0114] When performing preprocessing on the histogram to obtain the first mapping function, it can be implemented in, but not limited to, the following ways, such as Figure 3 shown, including:

[0115] Step 2041: According to the correspondence between the preset Y-channel component and the RGB component, determine the conversion coefficients of the R component, the G component, and the B component respectively.

[0116] According to the correspondence between the Y-channel component and the RGB component in the color gamut conversion, adjust the histogram (histogram reshaping).

[0117] In some embodiments, since most HDR videos are mainly in the first color gamut range, the BT2020 color gamut, for example, the BT2020 color gamut is taken as an example here, and it is not limited to the BT2020 color gamut in actual applications.

[0118] The relationship between the Y component and RGB in the BT2020 color gamut is shown in Formula 3:

[0119] Y = W r ×R + W g ×G + W b ×B (Formula 3)

[0120] For the BT2020 color gamut, W r = 0.2726, W g = 0.6780, W b = 0.0593, where W r , W g and W b are the conversion coefficients of the R component, G component, and B component respectively, and their specific data are all preset. The corresponding W r , W g and W b have differences.

[0121] Step 2042, adjust the number of each pixel of the histogram according to the conversion coefficients of the R component, G component, and B component to obtain an adjusted histogram.

[0122] n i = n i / 4 (Formula 4)

[0123]

[0124] Adjust the number of each pixel of the histogram according to Formulas (4) and (5) to obtain histogram Hist new

[0125] Step 2043, perform an accumulation process on the adjusted histogram according to the magnitude of pixel values to obtain an accumulated histogram function.

[0126] Accumulate Hist new to obtain a histogram accumulation function, as shown in Formula (6):

[0127]

[0128] In some embodiments, for Hist newAccumulation, for example, pixel values include 1 - 5. Count the number of different pixel values to obtain a histogram. Assume that the number of pixel value 1 is 4, the number of pixel value 2 is 10, the number of pixel value 3 is 30, the number of pixel value 4 is 6, and the number of pixel value 5 is 9. The accumulated pixel values are as follows: the number of pixel value 1 is 4, the number of pixel value 2 is 5 (1 + 4), the number of pixel value 3 is 35 (5 + 30, where 5 is the accumulated result of pixel value 1 and pixel value 2, and 30 is the number of pixel value 3), the number of pixel value 4 is 41 (35 + 6, where 35 is the accumulated result of pixel value 1, pixel value 2, and pixel value 3, and 6 is the number of pixel value 4), the number of pixel value 5 is 50 (41 + 9, 41 is the accumulated result of pixel value 1, pixel value 2, pixel value 3, and pixel value 4, and 9 is the number of pixel value 5), and so on until the entire histogram is accumulated.

[0129] It should be noted that the above example is only for illustrative purposes and does not limit the pixel values and their numbers in the histogram. The embodiments of the present disclosure do not make any limitations in this regard.

[0130] Step 2044: Perform normalization processing on the accumulated histogram function to obtain the first mapping function.

[0131] Normalize the histogram accumulation function to obtain the first mapping function, as shown in formula (7)

[0132]

[0133] where Hist M is the accumulated result of the maximum pixel value in the histogram.

[0134] Step 205: Perform fusion processing on the first mapping function and the second mapping function to obtain the target mapping function.

[0135] When performing the fusion processing on the first mapping function and the second mapping function to obtain the target mapping function, the following methods can be used but are not limited to them, as Figure 4 shown including:

[0136] Step 2051: Perform smoothing processing on the histogram function after normalization to obtain the smoothed histogram mapping function.

[0137] Since the histogram distribution may be uneven, the difference in the number of pixels of adjacent pixel values may be relatively large, resulting in jumps in the histogram mapping function after the normalization processing in step 2044, which may cause color discontinuity.

[0138] In some embodiments, to avoid this situation, a Bezier function is used to smooth the histogram function after normalization to obtain an improved mapping function for the current image frame in the video to be played.

[0139]

[0140]

[0141] Step 2052: Obtain a historical mapping function, where the historical mapping function is the mapping function corresponding to the previous adjacent YUV image of the YUV image to be processed.

[0142] To avoid jitter between frames, the processing result of the historical mapping function can be used for the fusion processing of the current frame.

[0143] In some embodiments, the historical mapping function is the mapping function corresponding to the previous adjacent YUV image of the YUV image to be processed. After the historical mapping function completes the image processing, the final processing result will be cached.

[0144] Step 2053: Perform a fusion process on the historical mapping function and the histogram mapping function after smoothing to obtain a final first mapping function.

[0145] The mapping function of the current image frame will be fused with the historical mapping function to obtain a final first mapping function.

[0146]

[0147] Among them, α is a fusion coefficient. In the embodiments of the present disclosure, α is an empirical value parameter and can be set according to service requirements. Specifically, the embodiments of the present disclosure do not make any limitations.

[0148] Apply the final first mapping function to image j to obtain the result Result of histogram mapping Bezier .

[0149] Fuse the output result Result Base (the output result of step 203) and the mapping result Result Bezier of the first mapping function to obtain a final mapping result.

[0150] Result = β × Result Base + (1 - β) × Result Bezier (11)

[0151] Among them, β is the fusion coefficient, and β ∈ [0, 1]. β is an empirical value parameter that can be set according to business requirements. Specifically, the embodiments of the present disclosure do not make limitations in this regard.

[0152] In some embodiments, after performing mapping processing on the images in the video to be played based on the target mapping function to obtain a target RGB image, the target RGB image is converted into an HDR image adapted to the display attribute information of the display device to display the HDR image. The target RGB image output in step 103 (the RGB image in the linear BT709 color gamut) is converted to obtain data adapted to screen display and output to the screen for display. For the implementation manner of obtaining an HDR image adapted to the display attribute information of the display device, any implementation manner in the related art can be referred to. Specifically, the embodiments of the present disclosure do not make limitations in this regard.

[0153] The above embodiments have described in detail the method of image processing, as Figure 5 shown, Figure 5 shows a schematic diagram of image processing provided by the embodiments of the present disclosure. For the specific implementation process of each node, reference can be made to the detailed description of the above embodiments, and the embodiments of the present disclosure will not elaborate herein one by one.

[0154] Corresponding to the above image processing method, the present invention also proposes an image processing device. Since the device embodiments of the present invention correspond to the above method embodiments, for the details not disclosed in the device embodiments, reference can be made to the above method embodiments, and the present invention will not elaborate herein.

[0155] Figure 6 is a schematic structural diagram of an image processing device 300 provided by the embodiments of the present disclosure. The image processing device includes:

[0156] A first conversion unit 31, configured to perform conversion processing on a YUV image to be processed in a first color gamut range to obtain an RGB image in a second color gamut range;

[0157] A generation unit 32, configured to generate a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function. The first mapping function is a function obtained by performing histogram statistics on the Y channel component of the YUV image to be processed, and the second mapping function is a preset function;

[0158] A processing unit 33, configured to perform mapping processing on the images in the video to be played based on the target mapping function to obtain a target RGB image.

[0159] In summary, according to the image processing device proposed by the present disclosure, the device includes converting a to-be-processed YUV image in a first color gamut range to obtain an RGB image in a second color gamut range, and generating a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function. The first mapping function is a function obtained by performing a histogram statistics on the Y channel component of the to-be-processed YUV image, and the second mapping function is a preset function. Based on the target mapping function, mapping processing is performed on the images in the to-be-played video to obtain a target RGB image. The solution of the present disclosure combines RGB pixels with luminance, increases the luminance of the image after tone mapping, and improves the image quality.

[0160] Further, in a possible implementation manner of the embodiment of the present disclosure, as Figure 7 shown, the generating unit 32 includes:

[0161] A decoding module 321, configured to decode the to-be-processed video stream into the to-be-processed YUV image;

[0162] A determining module 322, configured to determine the maximum pixel value in the to-be-processed YUV image;

[0163] A first processing module 323, configured to perform normalization processing on each pixel value in the to-be-processed YUV image based on the maximum pixel value to obtain a normalized pixel value;

[0164] A calculation module 324, Yang Hongyu, configured to calculate the second mapping function based on the normalized pixel value;

[0165] A statistics module 325, configured to perform histogram statistics on the Y channel component of the to-be-processed YUV image;

[0166] A second processing module 326, configured to preprocess the histogram to obtain the first mapping function;

[0167] A third processing module 327, configured to fuse the first mapping function and the second mapping function to obtain the target mapping function.

[0168] Further, in a possible implementation manner of the embodiment of the present disclosure, as Figure 7 shown, the second processing module 326 is further configured to:

[0169] According to the preset correspondence between the Y channel component and the RGB component, respectively determine the conversion coefficients of the R component, the G component, and the B component;

[0170] Adjust the number of each pixel in the histogram according to the conversion coefficients of the R component, the G component, and the B component to obtain an adjusted histogram;

[0171] Accumulate the adjusted histogram according to the magnitude of pixel values to obtain an accumulated histogram function;

[0172] Normalize the accumulated histogram function to obtain the first mapping function.

[0173] Further, in a possible implementation manner of the embodiment of the present disclosure, as Figure 7 shown, the second processing module 326 is further configured to:

[0174] Smooth the histogram function after normalization processing to obtain a smoothed histogram mapping function;

[0175] Obtain a historical mapping function, where the historical mapping function is the mapping function corresponding to the previous adjacent YUV image of the YUV image to be processed;

[0176] Fuse the historical mapping function and the smoothed histogram mapping function to obtain a final first mapping function.

[0177] Further, in a possible implementation manner of the embodiment of the present disclosure, as Figure 7 shown, the third processing module 327 is further configured to fuse the first mapping function and the second mapping function according to a preset first fusion coefficient and a preset second fusion coefficient to obtain the target mapping function, where the first fusion coefficient is the fusion coefficient of the first mapping function, and the second fusion coefficient is the fusion coefficient of the second mapping function.

[0178] Further, in a possible implementation manner of the embodiment of the present disclosure, as Figure 7 shown, the device further includes:

[0179] A second conversion unit 34, configured to convert the target RGB image into an HDR image adapted to the display attribute information of the display device after performing mapping processing on the image in the video to be played based on the target mapping function, so as to display the HDR image.

[0180] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above several embodiments, the implementation manners of the methods are also applicable to the device provided in this embodiment, and will not be described in detail in this embodiment.

[0181] In the embodiments provided by the present application above, the methods and apparatuses provided by the embodiments of the present application are introduced. To implement the various functions in the methods provided by the embodiments of the present application above, an electronic device may include a hardware structure, software modules, and implement the various functions above in the form of a hardware structure, software modules, or a combination of a hardware structure and software modules. A certain function among the various functions above may be executed in the manner of a hardware structure, software modules, or a combination of a hardware structure and software modules.

[0182] Figure 8 FIG. 4 is a block diagram of an electronic device 400 for implementing the above image processing method according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0183] Referring to Figure 8 , the electronic device 400 may include one or more of the following components: a processing component 402, a memory 404, a power component 406, a multimedia component 408, an audio component 410, an input / output (I / O) interface 412, a sensor component 414, and a communication component 416.

[0184] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0185] The memory 404 is configured to store various types of data to support the operation of the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0186] The power supply component 406 provides power for various components of the electronic device 400. The power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 400.

[0187] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0188] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 further includes a speaker for outputting audio signals.

[0189] The I / O interface 412 provides an interface between the processing component 402 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0190] The sensor assembly 414 includes one or more sensors for providing an assessment of the status of various aspects of the electronic device 400. For example, the sensor assembly 414 can detect the on / off state of the electronic device 400, the relative positioning of components, such as components for the display and keypad of the electronic device 400. The sensor assembly 414 can also detect a change in the position of the electronic device 400 or a component of the electronic device 400, the presence or absence of user contact with the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and a change in the temperature of the electronic device 400. The sensor assembly 414 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0191] The communication component 416 is configured to facilitate communication, in a wired or wireless manner, between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio), or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0192] In an exemplary embodiment, the electronic device 400 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0193] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of the electronic device 400 to complete the above-described methods through image processing. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0194] Embodiments of the present disclosure also propose a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute the methods described in the above embodiments of the present disclosure.

[0195] For the case where the electronic device may be a chip or a chip system, reference may be made to Figure 9 the structural schematic diagram of the chip shown. Figure 9 The chip shown includes a processor 501 and an interface 502. Among them, the number of processors 501 may be one or more, and the number of interfaces 502 may be multiple.

[0196] Optionally, the chip further includes a memory 503, and the memory 503 is used to store necessary computer programs and data.

[0197] Those skilled in the art can also understand that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. Whether such a function is implemented by hardware or software depends on the specific application and the design requirements of the entire system. For each specific application, those skilled in the art can use various methods to implement the function, but such implementation should not be construed as exceeding the scope protected by the embodiments of the present application.

[0198] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order different from those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0199] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0200] Any process or method description represented in the flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present invention includes additional implementations where functions may be executed not in the order shown or discussed, including in a substantially simultaneous manner according to the functions involved or in a reverse order, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0201] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing a logical function and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion with one or more wirings (control method), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which a program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0202] It should be understood that various parts of the embodiments of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0203] Those of ordinary skill in the art can understand that all or part of the steps carried out in implementing the above-described embodiment methods can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0204] In addition, each functional unit in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, or the like.

[0205] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. An image processing method, characterized in that, The method includes: Performing conversion processing on a to-be-processed YUV image in a first color gamut range to obtain an RGB image in a second color gamut range; wherein, the to-be-processed YUV image is obtained by decoding an image in a to-be-played video; Generating a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function, the first mapping function is a function obtained by performing histogram statistics on the Y channel component of the to-be-processed YUV image, and the second mapping function is a preset function; Performing mapping processing on the images in the to-be-played video based on the target mapping function to obtain a target RGB image.

2. The method according to claim 1, wherein The generating the target mapping function includes: Decoding a to-be-processed video stream into the to-be-processed YUV image and determining the maximum pixel value in the to-be-processed YUV image; Performing normalization processing on each pixel value in the to-be-processed YUV image based on the maximum pixel value to obtain normalized pixel values; Calculating the second mapping function based on the normalized pixel values; Performing histogram statistics on the Y channel component of the to-be-processed YUV image and preprocessing the histogram to obtain the first mapping function; Fusing the first mapping function and the second mapping function to obtain the target mapping function.

3. The method according to claim 2, wherein The preprocessing the histogram to obtain the first mapping function includes: Respectively determining conversion coefficients of the R component, the G component, and the B component according to the corresponding relationship between the preset Y channel component and the RGB components; Adjusting the number of each pixel in the histogram according to the conversion coefficients of the R component, the G component, and the B component to obtain an adjusted histogram; Performing cumulative processing on the adjusted histogram according to the pixel value size to obtain a cumulative histogram function; Performing normalization processing on the cumulative histogram function to obtain the first mapping function.

4. The method according to claim 3, characterized in that The performing normalization processing on the cumulative histogram function to obtain the first mapping function includes: Performing smoothing processing on the histogram function after normalization processing to obtain a smoothed histogram mapping function; Obtaining a historical mapping function, where the historical mapping function is the mapping function corresponding to the previous adjacent YUV image of the to-be-processed YUV image; Fusing the historical mapping function and the smoothed histogram mapping function to obtain the final first mapping function.

5. The method according to claim 2, wherein The fusing the first mapping function and the second mapping function to obtain the target mapping function includes: Fusing the first mapping function and the second mapping function according to a preset first fusion coefficient and a preset second fusion coefficient to obtain the target mapping function, where the first fusion coefficient is the fusion coefficient of the first mapping function and the second fusion coefficient is the fusion coefficient of the second mapping function.

6. The method according to any one of claims 1-5, characterized in that, After performing mapping processing on the images in the to-be-played video based on the target mapping function to obtain a target RGB image, the method further includes: Convert the target RGB image into an HDR image adapted to the display attribute information of the display device to display the HDR image.

7. An image processing apparatus, characterized in that, The device includes: A first conversion unit for performing conversion processing on a to-be-processed YUV image in a first color gamut range to obtain an RGB image in a second color gamut range; A generation unit for generating a target mapping function, where the target mapping function is a function obtained by fusing a first mapping function and a second mapping function, the first mapping function is a function obtained by performing histogram statistics on the Y channel component of the to-be-processed YUV image, and the second mapping function is a preset function; A processing unit for performing mapping processing on the images in the to-be-played video based on the target mapping function to obtain a target RGB image.

8. An electronic device, characterized in that, It includes: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

10. A chip, characterized in that, It includes one or more interfaces and one or more processors; the interfaces are used to receive signals from the memory of the electronic device and send the signals to the processors, and the signals include computer instructions stored in the memory. When the processors execute the computer instructions, the electronic device executes the method according to any one of claims 1-6.

Images