Image processing method and device

Abstract

The invention provides an image processing method. The method comprises the following steps: acquiring a target picture and an initial tone mapping curve of the target picture; determining a plurality of interpolation points according to the brightness information and / or the dynamic metadata of the target picture; determining a spline function passing through the plurality of interpolation points, wherein the spline function is smoothly connected with the initial tone mapping curve at at least one interpolation point in the plurality of interpolation points; correcting the initial tone mapping curve according to a spline function to obtain a corrected tone mapping curve of the target picture; and performing tone mapping on the target picture according to the corrected tone mapping curve. The invention further provides an image processing device. The method provided by the invention aims at enabling the display equipment to vividly and truly reproduce the scene in the real world.

Description

[0001] This application is a divisional application. The application number of the original application is 201910626171.8, and the original application date is July 11, 2019. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of image processing, and more specifically, to an image processing method and device. Background Art

[0003] Dynamic range (DR) can be used to represent the ratio of the maximum grayscale value to the minimum grayscale value of an image. Dynamic range can also be used to represent the ratio of the maximum brightness value to the minimum brightness value of an image. The real-world dynamic range is between 10 -3 -10 6 The range between the two is called high dynamic range (HDR). In other words, the brightness values ​​of things in the real world may be very large or very small. However, the display capabilities of display devices are usually limited. For example, the brightness of thing A in the real world is brighter, such as the brightness of thing A is 2000nit, and the maximum brightness that the display device can display is lower than the brightness of thing A, such as the maximum brightness that the display device can display is only 500nit. Without any processing, the display device will not be able to reproduce thing A in the real world. In another case, when the brightness of thing B in the real world is darker, such as the brightness value of thing B is 10nit, and the minimum brightness that the display device can display is higher than the brightness of thing B, such as the minimum brightness that the display device can display is only 100nit. Without any processing, the display device will not be able to reproduce thing B in the real world. The present application provides a method for enabling a display device to vividly and realistically reproduce scenes in the real world. Summary of the invention

[0004] The present application provides an image processing method and apparatus, the purpose of which is to enable a display device to vividly and realistically reproduce scenes in the real world.

[0005] In a first aspect, a method for image processing is provided, comprising: obtaining a target image and an initial tone mapping curve of the target image; determining a plurality of interpolation points based on brightness information and / or dynamic metadata of the target image; determining a spline function passing through the plurality of interpolation points, wherein the spline function is smoothly connected to the initial tone mapping curve at at least one interpolation point among the plurality of interpolation points; correcting the initial tone mapping curve according to the spline function to obtain a corrected tone mapping curve of the target image; and tone mapping the target image according to the corrected tone mapping curve.

[0006] Among them, the determining of multiple interpolation points based on the brightness information and / or dynamic metadata of the target image may be the determining of a correction interval for correcting the initial tone mapping curve based on the brightness information and / or dynamic metadata of the target image; the determining of the spline function passing through the multiple interpolation points may be the determining of a spline function for correcting the initial tone mapping curve within the correction interval; the initial tone mapping curve is corrected according to the spline function to obtain a corrected tone mapping curve of the target image; the correcting of the initial tone mapping curve according to the spline function to obtain a corrected tone mapping curve of the target image may be the correcting of the initial tone mapping curve to the spline function within the correction interval to obtain a corrected tone mapping curve of the target image.

[0007] The present application provides a method for correcting an already obtained initial tone mapping curve, so that the method of obtaining the tone mapping curve is more flexible. By analyzing the brightness of the target image and reasonably correcting the tone mapping curve, the image display effect can be adjusted for different brightness areas, and the brightness change details of the image can be enhanced to bring a better visual experience to the user. By performing various forms of fine-tuning on the tone mapping curve, a variety of display effects such as soft, vivid, bright, dusk, etc. can be obtained. For example, the form of the tone mapping curve can be flexibly determined according to the device properties of the display device and the shooting device, so that the image display is more vivid and natural. In addition, in the process of displaying a video, the tone mapping curve of a frame can be determined, and for the frames adjacent to the frame, the parameters of the corrected tone mapping curve are determined, so that the video display effect can be improved without adding too much data.

[0008] In combination with the first aspect, in certain implementations of the first aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of dark area pixels included in the target image is greater than or equal to a second preset threshold, the brightness value of the dark area pixels is less than the first preset threshold, the multiple interpolation points include a first interpolation point, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining that the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

[0009] In the embodiment of the present application, when the image is dark, the electronic device can determine that there is an interpolation point at the first endpoint or a position close to the first endpoint to improve the display effect of the dark area. Therefore, the correction position of the initial tone mapping curve can be determined according to the brightness of the image, so that the pixel points with brightness lower than the display capability of the display device can be displayed on the display device.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points also include a second interpolation point, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining the first coordinate value of the second interpolation point based on the number of pixels in the dark area.

[0011] In an embodiment of the present application, when the image is dark, the electronic device can determine the correction range of the initial tone mapping curve based on the area of ​​the dark area in the target image, so that pixels with brightness lower than the display capability of the display device can have a better display effect on the display device.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points also include a third interpolation point, the target image also includes a dark area transition pixel point, the brightness value of the dark area transition pixel point is greater than or equal to the first preset threshold, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining the first coordinate value of the third interpolation point based on the number of the dark area transition pixel points.

[0013] The third interpolation point may be a transition point, so that the spline function and the initial tone mapping curve smoothly transition at the third interpolation point. In the embodiment of the present application, when the image is dark, correction may be made at one end of the initial tone mapping curve. In order to ensure that the corrected tone mapping curve is smooth, that is, to ensure the image display effect, an interpolation point that plays a role of smooth connection may be determined so that the spline function is smoothly connected to the initial tone mapping curve.

[0014] In combination with the first aspect, in certain implementations of the first aspect, the initial tone mapping curve passes through the third interpolation point, and determining the spline function passing through the multiple interpolation points includes: obtaining a first spline function passing through the first interpolation point and the second interpolation point; determining a second spline function passing through the second interpolation point and the third interpolation point, the second spline function being smoothly connected to the first spline function at the second interpolation point, and the second spline function being smoothly connected to the initial tone mapping curve at the third interpolation point.

[0015] In combination with the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function, and the second spline function is a cubic spline function.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the spline function includes a third spline function passing through the first interpolation point, the second interpolation point, and the third interpolation point. Exemplarily, the third spline function is a cubic spline function.

[0017] In combination with the first aspect, in certain implementations of the first aspect, a first offset corresponding to a second interpolation point is obtained, the first offset being used to represent a correction amplitude of the initial tone mapping curve at a first correction position, the first correction position being a first coordinate value of the second interpolation point; the second coordinate value of the second interpolation point is determined based on the function value of the initial tone mapping curve at the first correction position and the first offset; the spline function passing through the multiple interpolation points is determined, including: determining the third spline function based on the first coordinate value of the second interpolation point and the second coordinate value of the second interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at a third interpolation point.

[0018] In combination with the first aspect, in certain implementations of the first aspect, the third spline function includes a fourth spline function passing through a first interpolation point and a second interpolation point, and a fifth spline function passing through the second interpolation point and the third interpolation point, and the second-order derivative value of the fourth spline function at the second interpolation point is the same as the second-order derivative value of the fifth spline function at the second interpolation point.

[0019] Exemplarily, the third spline function is a cubic spline function.

[0020] Exemplarily, the derivative of each spline function in the above scenario is always greater than 0.

[0021] In combination with the first aspect, in some implementations of the first aspect, the multiple interpolation points further include an eleventh interpolation point, and the second-order derivative of the spline function at the eleventh interpolation point is 0.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of bright area pixels included in the target image is greater than or equal to a fourth preset threshold, the brightness value of the bright area pixels is greater than a third preset threshold, the multiple interpolation points include a fourth interpolation point, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining that the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

[0023] In the embodiment of the present application, when the image is bright, the electronic device can determine that there is an interpolation point at or near the second endpoint to improve the display effect of the bright area. Therefore, the correction position of the initial tone mapping curve can be determined according to the brightness of the image, so that pixels with brightness higher than the display capability of the display device can be displayed on the display device.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points also include a fifth interpolation point, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining the first coordinate value of the fifth interpolation point based on the number of pixels in the bright area.

[0025] In an embodiment of the present application, when the image is bright, the electronic device can determine the correction range of the initial tone mapping curve based on the area of ​​the bright area in the target image, so that pixels with brightness higher than the display capability of the display device can have a better display effect on the display device.

[0026] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points also include a sixth interpolation point, the target image also includes a bright area transition pixel point, the brightness value of the bright area transition pixel point is less than or equal to the third preset threshold, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining the first coordinate value of the sixth interpolation point based on the number of the bright area transition pixel points.

[0027] The sixth interpolation point may be a transition point, so that the spline function and the initial tone mapping curve smoothly transition at the sixth interpolation point. In the embodiment of the present application, when the image is bright, correction may be made at one end of the initial tone mapping curve. In order to ensure that the corrected tone mapping curve is smooth, that is, to ensure the image display effect, an interpolation point that plays a role of smooth connection may be determined so that the spline function is smoothly connected to the initial tone mapping curve.

[0028] In combination with the first aspect, in certain implementations of the first aspect, the initial tone mapping curve passes through the sixth interpolation point, and determining the spline function passing through the multiple interpolation points includes: obtaining a first spline function passing through the fourth interpolation point and the fifth interpolation point; determining a second spline function passing through the fifth interpolation point and the sixth interpolation point, the second spline function being smoothly connected to the first spline function at the fifth interpolation point, and the second spline function being smoothly connected to the initial tone mapping curve at the sixth interpolation point.

[0029] In combination with the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function, and the second spline function is a cubic spline function.

[0030] In combination with the first aspect, in certain implementations of the first aspect, the spline function includes a third spline function passing through the fourth interpolation point, the fifth interpolation point, and the sixth interpolation point. Exemplarily, the third spline function is a cubic spline function.

[0031] In combination with the first aspect, in certain implementations of the first aspect, a first offset corresponding to a fifth interpolation point is obtained, the first offset being used to represent a correction amplitude of the initial tone mapping curve at a first correction position, the first correction position being a first coordinate value of the fifth interpolation point; the second coordinate value of the fifth interpolation point is determined according to the function value of the initial tone mapping curve at the first correction position and the first offset; the spline function passing through the multiple interpolation points is determined, including: determining the third spline function according to the first coordinate value of the fifth interpolation point and the second coordinate value of the fifth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at a sixth interpolation point.

[0032] In combination with the first aspect, in some implementations of the first aspect, the third spline function includes a fourth spline function passing through a fourth interpolation point and a fifth interpolation point, and a fifth spline function passing through a fifth interpolation point and a sixth interpolation point, and the second-order derivative value of the fourth spline function at the fifth interpolation point is the same as the second-order derivative value of the fifth spline function at the fifth interpolation point. In combination with the first aspect, in some implementations of the first aspect, the dynamic metadata includes at least one of the following: coordinate information of each interpolation point in the multiple interpolation points, regional brightness indication information, or coordinate information of multiple key pixel points.

[0033] In combination with the first aspect, in some implementations of the first aspect, the dynamic metadata includes coordinate information of each interpolation point in the multiple interpolation points.

[0034] In an embodiment of the present application, information for protecting interpolation points can be pre-set in dynamic metadata, so that the image processing device does not need to analyze the pixels of the image, and can determine the multiple interpolation points only based on the dynamic metadata, thereby reducing the amount of computational effort in image processing.

[0035] In combination with the first aspect, in certain implementations of the first aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the dynamic metadata includes regional brightness indication information, and when the regional brightness indication information indicates a dark area, the multiple interpolation points include a first interpolation point, and a first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint; and / or, when the regional brightness indication information indicates a bright area, the multiple interpolation points include a second interpolation point, and the first coordinate value of the second interpolation point is the same as the first coordinate value of the second endpoint.

[0036] In an embodiment of the present application, when the image is dark, the electronic device can determine that there is an interpolation point at the first endpoint or a position close to the first endpoint. Therefore, according to the brightness of the image, the correction position for correcting the initial tone mapping curve can be determined, so that pixels with brightness lower than the display capability of the display device can be displayed on the display device. When the image is bright, the electronic device can determine that there is an interpolation point at the second endpoint or a position close to the second endpoint. Therefore, according to the brightness of the image, the correction position for correcting the initial tone mapping curve can be determined, so that pixels with brightness higher than the display capability of the display device can be displayed on the display device. Since dark or bright is specified by dynamic metadata, the image processing device does not need to analyze the pixels of the image, and the multiple interpolation points can be determined only based on the dynamic metadata, reducing the amount of calculation for image processing.

[0037] In combination with the first aspect, in some implementations of the first aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the dynamic metadata includes P coordinate information corresponding to P key pixels one by one, P is a positive integer, and the multiple interpolation points are determined according to the brightness information of the target image and the dynamic metadata, including: according to the P coordinate information and the brightness information of the target image, determining that the P key pixels are dark area pixels or bright area pixels, the brightness value of the dark area pixel is less than a first preset threshold, and the brightness value of the bright area pixel is greater than a third preset threshold; when the P key pixels are dark area pixels, determining that the multiple interpolation points include a first interpolation point, and the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint; or, when the P key pixels are bright area pixels, determining that the multiple interpolation points include a second interpolation point, and the first coordinate value of the second interpolation point is the same as the first coordinate value of the second endpoint.

[0038] In an embodiment of the present application, when the image is dark, the electronic device can determine that there is an interpolation point at the first endpoint or a position close to the first endpoint. Therefore, the correction position for correcting the initial tone mapping curve can be determined according to the brightness of the image, so that pixels with brightness lower than the display capability of the display device can be displayed on the display device. When the image is bright, the electronic device can determine that there is an interpolation point at the second endpoint or a position close to the second endpoint. Therefore, the correction position for correcting the initial tone mapping curve can be determined according to the brightness of the image, so that pixels with brightness higher than the display capability of the display device can be displayed on the display device. Since the pixel points are specified by dynamic metadata, the image processing device does not need to determine the target pixel points when performing scene analysis on the target image, and directly determines the dark area pixel points and the bright area pixel points of the image according to the target pixel points specified by the dynamic metadata, so that the key protected areas and the corresponding multiple interpolation points can be determined more flexibly, reducing the amount of calculation for image processing.

[0039] In combination with the first aspect, in certain implementations of the first aspect, the target image includes a target pixel point having a brightness value between a first brightness value and a second brightness value, the proportion of the target pixel point to all pixel points of the target image is greater than an eighth preset threshold, the difference between the first brightness value and the second brightness value is less than a seventh preset threshold, the multiple interpolation points include a twelfth interpolation point, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining the twelfth interpolation point based on the average brightness of the target pixel point, the minimum brightness value of the target image, and the maximum brightness value of the target image.

[0040] The first coordinate value of the twelfth interpolation point may be greater than the first coordinate value of the first interpolation point mentioned above.

[0041] The first coordinate value of the twelfth interpolation point may be greater than the first coordinate value of the second interpolation point mentioned above.

[0042] The first coordinate value of the twelfth interpolation point may be greater than the first coordinate value of the third interpolation point mentioned above.

[0043] The first coordinate value of the twelfth interpolation point may be smaller than the first coordinate value of the fourth interpolation point mentioned above.

[0044] The first coordinate value of the twelfth interpolation point may be smaller than the first coordinate value of the fifth interpolation point mentioned above.

[0045] The first coordinate value of the twelfth interpolation point may be smaller than the first coordinate value of the sixth interpolation point mentioned above.

[0046] In the embodiment of the present application, the correction range can be determined on the initial tone mapping function according to the brightness range of the pixel to be corrected. In other words, correction can be performed in any section of the initial tone mapping function, and the correction flexibility is high. When the brightness range is narrow, the light and dark contrast of the image is not obvious. By correcting the initial tone mapping curve, the image with unclear light and dark contrast can be displayed more vividly on the electronic device.

[0047] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points also include a thirteenth interpolation point, and the multiple interpolation points are determined according to the brightness information of the target image, including: determining the thirteenth interpolation point according to the number of target pixel points.

[0048] In the embodiment of the present application, the correction range can be determined according to the number of pixels to be corrected, so that an image with unclear light and dark contrast can be displayed more vividly on the electronic device.

[0049] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points also include a fourteenth interpolation point, the target image also includes a target transition pixel point, the brightness value of the target transition pixel point is less than or equal to the first brightness value, or the brightness value of the target transition pixel point is greater than or equal to the second brightness value, and the first brightness value is less than the second brightness value, and the multiple interpolation points are determined based on the brightness information of the target image, including: determining the first coordinate value of the fourteenth interpolation point based on the number of the target transition pixel points.

[0050] In one example, the first coordinate value of the fourteenth interpolation point is greater than the first coordinate value of the twelfth interpolation point, and greater than the first coordinate value of the thirteenth interpolation point.

[0051] In one example, the first coordinate value of the fourteenth interpolation point is smaller than the first coordinate value of the twelfth interpolation point, and smaller than the first coordinate value of the thirteenth interpolation point.

[0052] The fourteenth interpolation point may be a transition point, so that the spline function and the initial tone mapping curve smoothly transition at the fourteenth interpolation point. In the embodiment of the present application, in order to ensure that the corrected tone mapping curve is smooth, that is, to ensure the image display effect, an interpolation point that plays a smooth connection role may be determined so that the spline function is smoothly connected to the initial tone mapping curve.

[0053] In combination with the first aspect, in certain implementations of the first aspect, the initial tone mapping curve passes through the fourteenth interpolation point, and determining the spline function passing through the multiple interpolation points includes: obtaining a first spline function passing through the twelfth interpolation point and the thirteenth interpolation point; determining a second spline function passing through the thirteenth interpolation point and the fourteenth interpolation point, the second spline function being smoothly connected to the first spline function at the thirteenth interpolation point, and the second spline function being smoothly connected to the initial tone mapping curve at the fourteenth interpolation point.

[0054] In combination with the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function, and the second spline function is a cubic spline function.

[0055] In combination with the first aspect, in some implementations of the first aspect, the spline function includes a third spline function passing through the twelfth interpolation point, the thirteenth interpolation point, and the fourteenth interpolation point, and the first coordinate value of the thirteenth interpolation point is between the first coordinate value of the twelfth interpolation point and the first coordinate value of the fourteenth interpolation point; before determining the multiple interpolation points, the method also includes: obtaining a first offset, the first offset is used to represent the correction amplitude of the initial tone mapping curve at a first correction position, and the first correction position is the first coordinate value of the thirteenth interpolation point; the determining of the multiple interpolation points includes: determining the second coordinate value of the thirteenth interpolation point according to the function value of the initial tone mapping curve at the first correction position and the first offset; the determining of the spline function passing through the multiple interpolation points includes: determining the third spline function according to the first coordinate value of the thirteenth interpolation point and the second coordinate value of the thirteenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the twelfth interpolation point and / or the fourteenth interpolation point.

[0056] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, and the determination of the spline function passing through the multiple interpolation points includes: obtaining a first spline function passing through the seventh interpolation point and the eighth interpolation point; determining a second spline function passing through the eighth interpolation point and the ninth interpolation point, the second spline function being smoothly connected to the first spline function at the eighth interpolation point, and the second spline function being smoothly connected to the initial tone mapping curve at the ninth interpolation point.

[0057] In an example, the seventh interpolation point may be the first interpolation point mentioned above, the eighth interpolation point may be the second interpolation point mentioned above, and the ninth interpolation point may be the third interpolation point mentioned above.

[0058] In an example, the seventh interpolation point may be the fourth interpolation point mentioned above, the eighth interpolation point may be the fifth interpolation point mentioned above, and the ninth interpolation point may be the sixth interpolation point mentioned above.

[0059] In one example, the eighth interpolation point may be the twelfth interpolation point mentioned above, the seventh interpolation point may be the fourteenth interpolation point mentioned above whose first coordinate value is less than the first coordinate value of the twelfth interpolation point and less than the first coordinate value of the thirteenth interpolation point, and the ninth interpolation point may be the fourteenth interpolation point mentioned above whose first coordinate value is greater than the first coordinate value of the twelfth interpolation point and greater than the first coordinate value of the thirteenth interpolation point.

[0060] In the embodiment of the present application, by obtaining the empirical value of the first spline function in advance, the correction effect within the key correction range can be ensured; and then the second spline function is used to achieve the effect of weak correction and smooth connection within the weak correction range.

[0061] In combination with the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function, and the second spline function is a cubic spline function.

[0062] In the embodiment of the present application, since the first spline function is an empirical value, and the first-order spline function requires fewer empirical values ​​to be known in advance, it is easier to determine the specific form of the first spline function. When the degree of the second spline function is small, the smooth connection effect of the second spline function is poor. When the degree of the second spline function is large, the prerequisite and calculation amount of the second spline function are large, so it is more appropriate to select a cubic spline function as the second spline function.

[0063] In combination with the first aspect, in some implementations of the first aspect, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and the first coordinate value of the eighth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; before determining the multiple interpolation points, the method further includes: obtaining a first offset, the first offset is used to indicate a correction amplitude of the initial tone mapping curve at a first correction position, and the first correction position is the first coordinate value of the eighth interpolation point; the determining the multiple interpolation points includes: determining the second coordinate value of the eighth interpolation point according to the function value of the initial tone mapping curve at the first correction position and the first offset; the determining the spline function passing through the multiple interpolation points includes: determining the third spline function according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

[0064] In an example, the seventh interpolation point may be the first interpolation point mentioned above, the eighth interpolation point may be the second interpolation point mentioned above, and the ninth interpolation point may be the third interpolation point mentioned above.

[0065] In an example, the seventh interpolation point may be the fourth interpolation point mentioned above, the eighth interpolation point may be the fifth interpolation point mentioned above, and the ninth interpolation point may be the sixth interpolation point mentioned above.

[0066] In an example, the seventh interpolation point may be the twelfth interpolation point mentioned above, the eighth interpolation point may be the thirteenth interpolation point mentioned above, and the ninth interpolation point may be the fourteenth interpolation point mentioned above.

[0067] In the embodiment of the present application, by obtaining the empirical value of the first offset in advance, the correction effect within the key correction range can be ensured; and the third spline function can achieve the effect of weak correction and smooth connection within the weak correction range.

[0068] In combination with the first aspect, in certain implementations of the first aspect, the third spline function includes a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point, and the second-order derivative value of the fourth spline function at the eighth interpolation point is the same as the second-order derivative value of the fifth spline function at the eighth interpolation point.

[0069] In combination with the first aspect, in some implementations of the first aspect, the multiple interpolation points further include a tenth interpolation point, the third spline function passes through the tenth interpolation point, and the first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point. Before determining the multiple interpolation points, the method further includes: obtaining a second offset, the second offset is used to represent the correction amplitude of the initial tone mapping curve at a second correction position, and the second correction position is the first coordinate value of the tenth interpolation point; the determining the multiple interpolation points also includes: determining the second coordinate value of the tenth interpolation point according to the function value of the initial tone mapping curve at the second correction position and the second offset; the determining the third spline function according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point includes: determining the third spline function according to the first coordinate value and the second coordinate value of the eighth interpolation point and the first coordinate value and the second coordinate value of the tenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

[0070] In the embodiment of the present application, by obtaining the empirical values ​​of the first offset and the second offset in advance, the correction effect within the key correction range is controlled; and the third spline function can achieve the effect of weak correction and smooth connection within the weak correction range.

[0071] In combination with the first aspect, in some implementations of the first aspect, the third spline function is a cubic spline function.

[0072] In the embodiment of the present application, when the degree of the third spline function is small, the smooth connection effect of the third spline function is poor. When the degree of the third spline function is large, the prerequisites and calculation amount of the third spline function are large. Therefore, it is more appropriate to select a cubic spline function as the third spline function.

[0073] In combination with the first aspect, in certain implementations of the first aspect, a derivative of the spline function is always greater than 0.

[0074] In the embodiment of the present application, by restricting the derivative at any point to be greater than 0, it is possible to avoid a large gap between the final spline function and the initial tone mapping curve, such as an oscillating spline function, thereby avoiding over-correction.

[0075] In combination with the first aspect, in some implementations of the first aspect, the multiple interpolation points further include an eleventh interpolation point, and the second-order derivative of the spline function at the eleventh interpolation point is 0.

[0076] In the embodiment of the present application, by limiting the second-order derivative of the eleventh interpolation point to 0, the spline function can be controlled to increase rapidly on one side of the eleventh interpolation point and increase slowly on the other side of the eleventh interpolation point.

[0077] In combination with the first aspect, in certain implementations of the first aspect, the multiple interpolation points also include a fifteenth interpolation point, the spline function is higher than the initial tone mapping function on one side of the fifteenth interpolation point, and the spline function is lower than the initial tone mapping function on the other side of the fifteenth interpolation point.

[0078] In the embodiment of the present application, the spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0079] In combination with the first aspect, in some implementations of the first aspect, the initial tone mapping curve includes one of the following formulas:

[0080]

[0081] L′=a×L+b;

[0082]

[0083] Among them, L represents the linear signal value, L' represents the nonlinear signal value, p, a, m, b, c1, c2, c3, n, N, ρ are all dynamic range mapping coefficients, 0≤L≤1.

[0084] In the embodiments of the present application, the method provided in the first aspect can be applied to a variety of initial tone mapping curves and has strong flexibility.

[0085] In a second aspect, an image processing device is provided, including: an acquisition module, used to acquire a target image and an initial tone mapping curve of the target image; a processing module, used to determine a plurality of interpolation points based on brightness information and / or dynamic metadata of the target image; the processing module, further used to determine a spline function passing through the plurality of interpolation points, the spline function being smoothly connected to the initial tone mapping curve at at least one interpolation point among the plurality of interpolation points; the processing module, further used to correct the initial tone mapping curve according to the spline function to obtain a corrected tone mapping curve of the target image; and a tone mapping module, used to perform tone mapping on the target image according to the corrected tone mapping curve.

[0086] In combination with the second aspect, in certain implementations of the second aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of dark area pixels included in the target image is greater than or equal to the second preset threshold, the brightness value of the dark area pixel is less than the first preset threshold, the multiple interpolation points include a first interpolation point, and the processing module is specifically used to determine that the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

[0087] In combination with the second aspect, in some implementations of the second aspect, the multiple interpolation points also include a second interpolation point, and the processing module is specifically used to determine the first coordinate value of the second interpolation point according to the number of pixels in the dark area.

[0088] In combination with the second aspect, in certain implementations of the second aspect, the multiple interpolation points also include a third interpolation point, the target image also includes a dark area transition pixel point, the brightness value of the dark area transition pixel point is greater than or equal to the first preset threshold, and the processing module is specifically used to determine the first coordinate value of the third interpolation point based on the number of the dark area transition pixel points.

[0089] In combination with the second aspect, in certain implementations of the second aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of bright area pixels included in the target image is greater than or equal to a fourth preset threshold, the brightness value of the bright area pixel is greater than a third preset threshold, the multiple interpolation points include a fourth interpolation point, and the processing module is specifically used to determine that the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

[0090] In combination with the second aspect, in some implementations of the second aspect, the multiple interpolation points also include a fifth interpolation point, and the processing module is specifically configured to determine a first coordinate value of the fifth interpolation point according to the number of pixels in the bright area.

[0091] In combination with the second aspect, in certain implementations of the second aspect, the multiple interpolation points also include a sixth interpolation point, the target image also includes a bright area transition pixel point, the brightness value of the bright area transition pixel point is less than or equal to the third preset threshold, and the processing module is specifically used to determine the first coordinate value of the sixth interpolation point based on the number of the bright area transition pixel points.

[0092] In combination with the second aspect, in some implementations of the second aspect, the dynamic metadata includes coordinate information of each interpolation point in the multiple interpolation points.

[0093] In combination with the second aspect, in certain implementations of the second aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the dynamic metadata includes regional brightness indication information, and when the regional brightness indication information indicates a dark area, the multiple interpolation points include a first interpolation point, and the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint; and / or, when the regional brightness indication information indicates a bright area, the multiple interpolation points include a second interpolation point, and the first coordinate value of the second interpolation point is the same as the first coordinate value of the second endpoint.

[0094] In combination with the second aspect, in some implementations of the second aspect, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the dynamic metadata includes P coordinate information corresponding to P key pixels one by one, P is a positive integer, and the processing module is specifically used to determine, based on the P coordinate information and the brightness information of the target image, that the P key pixels are dark area pixels or bright area pixels, the brightness value of the dark area pixel is less than a first preset threshold, and the brightness value of the bright area pixel is greater than a third preset threshold; when the P key pixels are dark area pixels, the processing module is specifically used to determine that the multiple interpolation points include a first interpolation point, and the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint; or, when the P key pixels are bright area pixels, the processing module is specifically used to determine that the multiple interpolation points include a second interpolation point, and the first coordinate value of the second interpolation point is the same as the first coordinate value of the second endpoint.

[0095] In combination with the second aspect, in certain implementations of the second aspect, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, and the processing module is specifically used to obtain a first spline function passing through the seventh interpolation point and the eighth interpolation point; the processing module is specifically used to determine a second spline function passing through the eighth interpolation point and the ninth interpolation point, the second spline function is smoothly connected to the first spline function at the eighth interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point.

[0096] In combination with the second aspect, in some implementations of the second aspect, the first spline function is a linear spline function, and the second spline function is a cubic spline function.

[0097] In combination with the second aspect, in some implementations of the second aspect, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and the first coordinate value of the eighth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; before the processing module determines the multiple interpolation points, the acquisition module is further used to acquire a first offset, and the first offset is used to represent the correction amplitude of the initial tone mapping curve at a first correction position, and the first correction position is the first coordinate value of the eighth interpolation point; the processing module is specifically used to determine the second coordinate value of the eighth interpolation point according to the function value of the initial tone mapping curve at the first correction position and the first offset; the processing module is specifically used to determine the third spline function according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

[0098] In combination with the second aspect, in certain implementations of the second aspect, the third spline function includes a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point, and the second-order derivative value of the fourth spline function at the eighth interpolation point is the same as the second-order derivative value of the fifth spline function at the eighth interpolation point.

[0099] In combination with the second aspect, in some implementations of the second aspect, the multiple interpolation points also include a tenth interpolation point, the third spline function passes through the tenth interpolation point, and the first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point. Before the processing module determines the multiple interpolation points, the acquisition module is also used to acquire a second offset, and the second offset is used to represent the correction amplitude of the initial tone mapping curve at a second correction position, and the second correction position is the first coordinate value of the tenth interpolation point; the processing module is specifically used to determine the second coordinate value of the tenth interpolation point according to the function value of the initial tone mapping curve at the second correction position and the second offset; the processing module is specifically used to determine the third spline function according to the first coordinate value and the second coordinate value of the eighth interpolation point and the first coordinate value and the second coordinate value of the tenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

[0100] In combination with the second aspect, in some implementations of the second aspect, the third spline function is a cubic spline function.

[0101] In combination with the second aspect, in certain implementations of the second aspect, a derivative of the spline function is always greater than 0.

[0102] In combination with the second aspect, in some implementations of the second aspect, the multiple interpolation points further include an eleventh interpolation point, and the second-order derivative of the spline function at the eleventh interpolation point is 0.

[0103] In conjunction with the second aspect, in some implementations of the second aspect, the initial tone mapping curve includes one of the following formulas:

[0104]

[0105] L′=a×L+b;

[0106]

[0107] Among them, L represents the linear signal value, L' represents the nonlinear signal value, p, a, m, b, c1, c2, c3, n, N, ρ are all dynamic range mapping coefficients, 0≤L≤1.

[0108] In a third aspect, the present application provides an image processing device, comprising a module for executing the first aspect or any one of the implementations of the first aspect.

[0109] In a fourth aspect, the present application provides an image processing device, comprising a processor and a transmission interface, wherein the transmission interface is used to receive or send image data; the processor is used to call program instructions stored in a memory and execute the program instructions to execute the method described in the first aspect or any one of the implementation methods of the first aspect.

[0110] In a possible implementation manner, the image processing device may further include a memory.

[0111] Exemplarily, the image processing device may be a chip.

[0112] In a fifth aspect, the present application provides an electronic device, comprising a processor and a memory, for executing the method described in the first aspect or any one of the implementations of the first aspect.

[0113] In a sixth aspect, the present application provides a computer-readable storage medium storing program instructions, which, when the program instructions are executed on a computer or a processor, cause the computer or the processor to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0114] In a seventh aspect, the present application provides a computer program product, including a computer program. When the computer program is executed on a computer or a processor, it will cause the computer or the processor to execute the method described in the first aspect or any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] Figure 1 It is a block diagram of an exemplary image or video transmission system for implementing an embodiment of the present application.

[0116] Figure 2 It is a schematic flowchart of an exemplary image processing method provided in an embodiment of the present application.

[0117] Figure 3 It is a schematic diagram of multiple initial tone mapping curves provided in an embodiment of the present application.

[0118] Figure 4 It is a schematic diagram of multiple spline functions provided in the embodiments of the present application.

[0119] Figure 5 It is a schematic diagram of an exemplary spline function for correcting an initial tone mapping curve for a dark area provided in an embodiment of the present application.

[0120] Figure 6 It is a schematic diagram of an exemplary spline function for correcting an initial tone mapping curve for a bright area provided in an embodiment of the present application.

[0121] Figure 7 It is a schematic diagram of an exemplary spline function for correcting an initial tone mapping curve for an area with weak light-dark contrast provided in an embodiment of the present application.

[0122] Figure 8 It is a structural diagram of an exemplary image processing device provided in an embodiment of the present application.

[0123] Fig. 9 It is a structural diagram of an exemplary image processing device provided in another embodiment of the present application.

[0124] Fig.10 It is a structural diagram of an exemplary electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0125] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0126] In an embodiment of the present application, the electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through a process, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system. The application layer includes applications such as a browser, an address book, a word processing software, and an instant messaging software. In addition, the embodiment of the present application does not specifically limit the specific structure of the execution subject of the method provided in the embodiment of the present application, as long as it can be performed by running a program that records the code of the method provided in the embodiment of the present application, so as to perform image processing according to the method provided in the embodiment of the present application, for example, the execution subject of the method provided in the embodiment of the present application can be an electronic device, or a functional module in an electronic device that can call a program and execute the program.

[0127] In addition, various aspects or features of the present application can be implemented as methods, devices or products using standard programming and / or engineering techniques. The term "product" used in this application covers computer programs that can be accessed from any computer-readable device, carrier or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or tapes, etc.), optical disks (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.). In addition, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0128] The following describes the system architecture used in the embodiments of the present invention. Figure 1 , Figure 1 A schematic block diagram of an image display system 10 used in an embodiment of the present invention is given as an example. Figure 1As shown, the image display system 10 may include an image acquisition device 12 and an image processing device 14, and the image acquisition device 12 generates a high dynamic range image. The image processing device 14 can process the high dynamic range image generated by the image acquisition device 12. The image acquisition device 12, the image processing device 14, or the image processing device shown in various embodiments of the present application may include one or more processors and a memory coupled to the one or more processors. The memory may include but is not limited to a random access memory (RAM), a read-only memory (ROM), an EEPROM, a flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures that can be accessed by a computer, as described herein. The actual form of the image acquisition device 12 and the image processing device 14 may include various devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, vehicle-mounted computers, wireless communication devices, or the like.

[0129] Although Figure 1 The image acquisition device 12 and the image processing device 14 are shown as separate devices, but the device embodiment may also include both the image acquisition device 12 and the image processing device 14 or the functions of both, that is, the image acquisition device 12 and the image processing device 14 are two different parts of the same device, for example, the image acquisition device may be a camera in a mobile smartphone, and the image processing device may be a processor in the mobile smartphone. In such embodiments, the image acquisition device 12 or the corresponding functions and the image processing device 14 or the corresponding functions may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0130] The image acquisition device 12 and the image processing device 14 may be communicatively connected via a link 13, and the image processing device 14 may receive the high dynamic range image from the image acquisition device 12 via the link 13. The link 13 may include one or more media or devices capable of moving the high dynamic range image from the image acquisition device 12 to the image processing device 14. In one example, the link 13 may include one or more communication media that enable the image acquisition device 12 to transmit the high dynamic range image directly to the image processing device 14 in real time. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the image acquisition device 12 to the image processing device 14.

[0131] Exemplarily, the image acquisition device 12 includes an image source 16 and a communication interface 22 or a transmission interface. Optionally, the image acquisition device 12 may also include an image processor 20. In a specific implementation form, the image processor 20, the image source 16, and the communication interface 22 may be hardware components in the image acquisition device 12, or may be software programs in the image acquisition device 12. They are described as follows:

[0132] The picture source 16 may include or may be any type of picture capture device, for example, for capturing real-world pictures, and / or any type of pictures or comments (for screen content encoding, some text on the screen is also considered to be part of the picture or image to be encoded) generating device, for example, a computer graphics processor for generating computer animation pictures, or any type of device for acquiring and / or providing real-world pictures, computer animation pictures (for example, screen content, virtual reality (VR) pictures), and / or any combination thereof (for example, augmented reality (AR) pictures). The picture source 16 may be a camera for capturing pictures or a memory for storing pictures, and the picture source 16 may also include any type of (internal or external) interface for storing previously captured or generated pictures and / or acquiring or receiving pictures. When the picture source 16 is a camera, the picture source 16 may be, for example, a local or integrated camera integrated in the image acquisition device; when the picture source 16 is a memory, the picture source 16 may be a local or integrated memory integrated in the image acquisition device. When the picture source 16 includes an interface, the interface may be, for example, an external interface for receiving pictures from an external video source, the external video source being, for example, an external picture capturing device, such as a camera, an external memory, or an external picture generating device, the external picture generating device being, for example, an external computer graphics processor, a computer, or a server. The interface may be any class of interfaces according to any proprietary or standardized interface protocol, such as a high definition multimedia interface (HDMI), a mobile industry processor interface (MIPI), a display serial interface (DSI) standardized by MIPI, an embedded display port (eDP) standardized by the Video Electronics Standards Association (VESA), or a V-By-One interface, which is a digital interface standard developed for image transmission, as well as various wired or wireless interfaces, optical interfaces, and the like.

[0133] Among them, the picture can be regarded as a two-dimensional array or matrix of pixels (picture element). The pixels in the array can also be called sampling points. The number of sampling points in the array or picture in the horizontal and vertical directions (or axes) defines the size and / or resolution of the picture. In order to represent the color, three color components are usually used, that is, the picture can be represented as or include three sampling arrays. For example, in the RGB format or color space, the picture includes corresponding red, green and blue sampling arrays. However, in video coding, each pixel is usually represented in a brightness / chroma format or color space, such as for a picture in the YUV format, including a brightness component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The brightness (luma) component Y represents the brightness or grayscale level intensity (for example, the two are the same in a grayscale picture), and the two chroma (chroma) components U and V represent the chroma or color information components. Accordingly, the picture in the YUV format includes a brightness sampling array of brightness sampling values ​​(Y), and two chroma sampling arrays of chroma values ​​(U and V). An image in RGB format can be converted or transformed into YUV format, and vice versa, which is also called color conversion or color format conversion. If the image is black and white, the image may only include a brightness sampling array. In the embodiment of the present invention, the image transmitted from the image source 16 to the image processor may also be referred to as raw image data 17.

[0134] The image processor 20 is used to perform image processing, such as retouch, color format conversion (eg, from RGB format to YUV format), color adjustment or denoising.

[0135] The communication interface 22 may be used to receive the image-processed picture data 19 and transmit the image-processed picture data 19 to the image processing device 14 or any other device (such as a memory) via the link 13 for storage or direct reconstruction. The other device may be any device for storage. The communication interface 22 may, for example, be used to encapsulate the image-processed picture data 19 into a suitable format, such as a data packet, for transmission over the link 13.

[0136] The image processing device 14 includes a communication interface 28, an image processor 32 and a display terminal device 34. They are described as follows:

[0137] The communication interface 28 can be used to receive the image data 19 after image processing from the image acquisition device 12 or any other source, such as a storage device, and the storage device is a storage device for the image data after image processing. Exemplarily, the specific examples of the communication interface 22 and the communication interface 28 can refer to the aforementioned description of the interface, which will not be repeated here. The communication interface 28 can be used to transmit or receive the image data 19 after image processing via the link 13 between the image acquisition device 12 and the image processing device 14 or via any type of network, such as a direct wired or wireless connection, any type of network such as a wired or wireless network or any combination thereof, or any type of private network and public network, or any combination thereof. The communication interface 28 can be used, for example, to decapsulate the data packet transmitted by the communication interface 22 to obtain the image data 19 after image processing.

[0138] Both the communication interface 28 and the communication interface 22 can be configured as a unidirectional communication interface or a bidirectional communication interface, and can be used, for example, to send and receive messages to establish a connection, confirm and exchange any other information related to the communication link and / or, for example, image data after image processing and / or data transmission.

[0139] The image processor 32 is used to perform tone mapping processing on the image-processed picture data 19 to obtain tone-mapped picture data 33. The processing performed by the image processor 32 may also include: color format conversion (for example, from YUV format to RGB format), color adjustment, repair or resampling, or any other processing, and may also be used to transmit the tone-mapped picture data 33 to the display terminal device 34.

[0140] A display end device 34 is used to receive the tone-mapped image data 33 to display the image to, for example, a user or viewer. The display end device 34 can be or include any type of display for presenting the reconstructed image, such as an integrated or external display or monitor. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0141] Although, Figure 1The image acquisition device 12 and the image processing device 14 are depicted as separate devices, but device embodiments may also include both the image acquisition device 12 and the image processing device 14 or the functionality of both, i.e., the image acquisition device 12 or the corresponding functionality and the image processing device 14 or the corresponding functionality. In such embodiments, the image acquisition device 12 or the corresponding functionality and the image processing device 14 or the corresponding functionality may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0142] Although, Figure 1 The image processing device 14 and the display end device 34 are illustrated as separate devices, but the device embodiment may also include both the image processing device 14 and the display end device 34 or the functionality of both, that is, the image processing device 14 or the corresponding functionality and the display end device 34 or the corresponding functionality. In such embodiments, the image processing device 14 or the corresponding functionality and the display end device 34 or the corresponding functionality may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0143] It is obvious to a person skilled in the art based on the description that the functionality or Figure 1 The presence and (accurate) division of the functionality of the image acquisition device 12 and / or image processing device 14 shown may vary depending on the actual device and application. The image acquisition device 12 and the image processing device 14 may include any of a variety of devices, including any type of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a video camera, a desktop computer, a set-top box, a television, a camera, a car device, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use or use any type of operating system.

[0144] The image processor 20 and the image processor 32 may be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology is partially implemented in software, the device may store the instructions of the software in a suitable non-transitory computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors.

[0145] The image acquisition device can capture pictures or videos containing natural scenes by receiving light signals from natural scenes. In order to facilitate the transmission of images or videos, it is necessary to convert the light signal into an electrical signal, and record the image information of each pixel through a fixed value range of brightness value or chromaticity value (for example, grayscale value between 0 and 255). The curve that converts the light signal into an electrical signal is the photoelectric transfer function. Common photoelectric transfer functions include PQ (perceptual quantizer) photoelectric transfer function, HLG (hybrid log-gamma) photoelectric transfer function, and scene luminance fidelity (SLF) photoelectric transfer function. The display end device can reproduce the natural scene by displaying pictures or videos. Usually, the image acquisition device and the display end device are located in two different locations. Therefore, the image acquisition device is often called the front-end device, and the display end device is often called the back-end device. The display device can determine the brightness of an object when shooting based on the photoelectric transfer function and the brightness value or grayscale value of each pixel in the image, that is, the electronic device can convert YUV or RGB information into brightness in units of nit. The brightness of the object may exceed the display capability of the display end device. Since the brightness information acquired by the image acquisition device does not match the brightness display capability of the display device, there is a situation where a high-brightness image is displayed using a display device with low-brightness display capability, and a situation where a low-brightness image is displayed using a display device with high-brightness display capability. Among them, the former situation is also called a high-to-low tone mapping process, and the latter situation is also called a low-to-high tone mapping process. Tone mapping refers to a computer graphics technology that approximately displays a high dynamic range image on a display device with limited dynamic range display capability. One way is to map the tone of a high dynamic range image to a display device with low dynamic range display capability based on a tone mapping curve. It should be noted that the tone mapping curve can be determined by the display device or by a processing device external to the display device.

[0146] However, in some cases, when the tone mapping curve is not accurately determined, the display effect of the high dynamic range image is poor. For example, during video playback, if only one tone mapping curve is determined for the video, the image content displayed between different video frames may be quite different. For example, some video frames display very dark images, while other video frames display very bright images. If a fixed tone mapping curve is used for the entire video, it is impossible to perfectly reproduce each frame of the video on the display device. In addition, the tone mapping curve carries a large amount of data. If a tone mapping curve is determined for each image or frame, data redundancy may be introduced.

[0147] The present application provides an image processing method and device, which can avoid the above problems.

[0148] Figure 2 Shown is a schematic flow chart of an image processing method provided by the present application. Figure 2 The method shown may be performed by a processor for tone mapping, such as Figure 1 The image processing device 14 shown is either an image acquisition device including a tone mapping processor, or a display terminal device including a tone mapping processor.

[0149] 201. Obtain a target image and an initial tone mapping curve of the target image.

[0150] The target image can be Figure 1 The target image can be a frame in a video, a static image, or a certain area in an image, such as several image blocks in an image.

[0151] The initial tone mapping curve may be any form of curve that can be used in the tone mapping process. Specifically, Figure 3 Several possible forms of the initial tone mapping curve are shown.

[0152] Figure 3 The initial tone mapping curve shown by the solid line is also called the SLF tone mapping curve, which satisfies:

[0153]

[0154] Wherein, L represents the linear signal value, L' represents the nonlinear signal value, p, a, m, b are all dynamic range mapping coefficients, 0≤L≤1, 0≤L'≤1. a is proportional to the dynamic range display capability of the display device and inversely proportional to the dynamic range of the target image. b is proportional to the minimum brightness display capability of the display device.

[0155] Figure 3 The initial tone mapping curve shown by the single dot-dashed line is a first-order polynomial tone mapping function, which satisfies:

[0156] L′=a×L+b,

[0157] Wherein, L represents the linear signal value, L' represents the nonlinear signal value, a and b are dynamic range mapping coefficients, 0≤L≤1, 0≤L'≤1. a is proportional to the dynamic range display capability of the display device and inversely proportional to the dynamic range of the target image. b is proportional to the minimum brightness display capability of the display device.

[0158] Figure 3The initial tone mapping curve shown by the dashed line is also called the sigmoidal tone mapping curve, which satisfies:

[0159]

[0160] Among them, L represents the linear signal value, L' represents the nonlinear signal value, c1, c2, c3, and n are all dynamic range mapping coefficients, 0≤L≤1, 0≤L'≤1. c1, c2, c3, and n are all related to the average brightness, maximum brightness, and minimum brightness of the target image, and are related to the minimum display brightness and maximum display brightness of the display device.

[0161] Figure 3 The initial tone mapping curve shown by the double-dotted line is also called the Bezier tone mapping curve, which satisfies:

[0162]

[0163] Wherein, L represents the linear signal value, L' represents the nonlinear signal value, n, N, ρ are all dynamic range mapping coefficients, 0≤L≤1, 0≤L'≤1. n, N, ρ are related to the brightness distribution of the target image and also to the average brightness of the target image.

[0164] The electronic device may determine parameters in the initial tone mapping curve according to brightness information and / or dynamic metadata of the target image.

[0165] It should be understood that both the brightness value in the YUV format and the grayscale value in the RGB format can reflect the brightness of the image. Therefore, the brightness information in the present application can be, for example, the brightness value in the YUV format or the grayscale value in the RGB format.

[0166] It should be understood that the dynamic metadata may include information about the brightness range and the proportion of each brightness range, for example, the number of pixels in the brightness range of 0 to 10 nit is 200. The dynamic metadata also includes the average brightness of the pixels in the brightness range, for example, the average brightness of the pixels in the brightness range of 0 to 10 nit is 6.2 nit.

[0167] Taking the SLF formula as an example, p and m can be determined by the overall brightness of the image. When the image is brighter overall, the value of p is smaller and the value of m is larger; when the image is darker overall, the value of p is larger and the value of m is smaller. The value range of p can be 3 to 6. The value range of m can be 2 to 4, and m is usually 2.4. The electronic device can statistically analyze the brightness information to determine the overall brightness of the image. Based on the information contained in the dynamic metadata, the electronic device can determine that the overall brightness of the image is brighter when the proportion of brighter pixels is higher, and determine that the overall brightness of the image is darker when the proportion of darker pixels is higher.

[0168] Optionally, the initial tone mapping curve may be a normalized curve, that is, the coordinates of any point in the initial tone mapping curve are (first coordinate value 1, second coordinate value 1), the first coordinate value 1 is between 0 and 1, and the second coordinate value 1 is between 0 and 1. The following is an explanation based on the normalized initial tone mapping curve, and it should be understood that the present application is not limited to this.

[0169] 202. Determine a plurality of interpolation points according to brightness information and / or dynamic metadata of the target image.

[0170] The purpose of determining the multiple interpolation points is to determine the spline function used to correct the initial tone mapping curve. By determining the multiple interpolation points, the spline function can be limited to pass through the multiple interpolation points, thereby obtaining an image display effect with a better tone mapping effect.

[0171] The interpolation point may be determined by determining the coordinates of the interpolation point, or the first coordinate value or the second coordinate value of the interpolation point. The coordinates of the interpolation point may be expressed as (x, y), where x may represent the first coordinate value and y may represent the second coordinate value. The electronic device may determine multiple interpolation points based on the brightness information of the image, or based on the dynamic metadata of the image, or based on the brightness information of the image and the dynamic metadata of the image. For example, the first coordinate value of interpolation point 1, the second coordinate value of interpolation point 2, and the coordinates of interpolation point 3 are determined.

[0172] In the present application, the purpose of determining the interpolation point is to determine the modification position of the initial tone mapping curve. That is, there may be a correction range between two interpolation points included in the plurality of interpolation points, and the electronic device may modify the initial tone mapping curve within the correction range.

[0173] In one example, the electronic device determines that the first coordinate value of interpolation point 1 is first coordinate value 1, and the first coordinate value of interpolation point 2 is first coordinate value 2, first coordinate value 1<first coordinate value 2, and it can be determined that the initial tone mapping curve is modified within the interval from first coordinate value 1 to first coordinate value 2.

[0174] Similarly, the electronic device determines that the second coordinate value of interpolation point 1 is second coordinate value 1, and the second coordinate value of interpolation point 2 is second coordinate value 2, and second coordinate value 1<second coordinate value 2. It can be determined that the initial tone mapping curve is modified within the interval from second coordinate value 1 to second coordinate value 2.

[0175] In an example, the electronic device determines that the first coordinate value of the interpolation point 1 is the first coordinate value 1, and the second coordinate value of the interpolation point 2 is the second coordinate value 2. The electronic device can determine, based on the initial tone mapping curve, that the first coordinate value corresponding to the second coordinate value 2 is the first coordinate value 2. The electronic device determines that the first coordinate value 1<first coordinate value 2, and can determine to modify the initial tone mapping curve within the interval of the second coordinate value 1 to the second coordinate value 2.

[0176] Similarly, the electronic device determines that the second coordinate value of the interpolation point 1 is the first coordinate value 1, and the second coordinate value of the interpolation point 2 is the second coordinate value 2. The electronic device can determine, based on the initial tone mapping curve, that the second coordinate value corresponding to the first coordinate value 1 is the second coordinate value 1. The electronic device judges that the second coordinate value 1 is less than the second coordinate value 2, and can determine to modify the initial tone mapping curve within the interval from the second coordinate value 1 to the second coordinate value 2.

[0177] In one example, the electronic device determines that the coordinates of interpolation point 1 are (first coordinate value 1, second coordinate value 1), and the coordinates of interpolation point 2 are (first coordinate value 2, second coordinate value 2), first coordinate value 1 < first coordinate value 2 (since the initial tone function is usually a monotonically increasing function, second coordinate value 1 < second coordinate value 2), and it can be determined that the initial tone mapping curve is modified within the interval from the first coordinate value 1 to the first coordinate value 2 (or the interval from the second coordinate value 1 to the second coordinate value 2).

[0178] The initial tone mapping curve includes two endpoints, namely the first endpoint and the second endpoint. Figure 4 Points 1 and 5 are shown. Among them, the brightness value corresponding to the first endpoint is lower and is located in the dark area of ​​the image, and the brightness value corresponding to the second endpoint is higher and is located in the bright area of ​​the image. When the image is dark, the electronic device can determine that there is an interpolation point at the first endpoint or a position close to the first endpoint. When the image is bright, the electronic device can determine that there is an interpolation point at the second endpoint or a position close to the second endpoint. When the image contains a part of pixels with small brightness differences, the position of the interpolation point can be determined based on the brightness of the part of pixels and the overall brightness of the image. For example, when the brightness range of the picture is between 100 and 300 nit, and 50% of the pixels are within the range of 195 to 205 nit, it can be determined that there is an interpolation point near the middle of the initial tone mapping curve.

[0179] In scenario 1, when the image is dark, the electronic device can determine that there is an interpolation point at the first endpoint or a position close to the first endpoint. The schematic diagram of the interpolation point and the spline function in this scenario is as follows: Figure 5 shown.

[0180] Optionally, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of dark area pixels contained in the target image is greater than or equal to a second preset threshold, the brightness value of the dark area pixels is less than the first preset threshold, the multiple interpolation points include a first interpolation point, and determining the multiple interpolation points based on the brightness information of the target image includes: determining that the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

[0181] In the present application, the brightness value corresponding to the first endpoint may correspond to the first coordinate value of the first endpoint and / or to the second coordinate value of the first endpoint. The brightness value corresponding to the second endpoint may correspond to the first coordinate value of the second endpoint and / or to the second coordinate value of the second endpoint.

[0182] For example, the first preset threshold is 20, and the pixel whose Y component value in the YUV format is less than 20 can be considered as a dark pixel, or the pixel whose R / G / B component value in the RGB format is less than 20 can be considered as a dark pixel. For another example, the first preset threshold is 20nit, and the pixel whose brightness is less than 20nit can be considered as a dark pixel.

[0183] Therefore, when the target image contains a sufficient number of dark area pixels, the first endpoint can be selected as the first interpolation point, and the first coordinate value of the first endpoint is determined to be the same as the first coordinate value of the first interpolation point. In other words, when the target image contains a sufficient number of dark area pixels, a section of the initial tone factor curve close to the first endpoint can be corrected.

[0184] Optionally, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the dynamic metadata includes P coordinate information corresponding to P key pixel points one by one, P is a positive integer, and the multiple interpolation points are determined according to the brightness information of the target image and the dynamic metadata, including: determining that the P key pixel points are dark area pixel points according to the P coordinate information and the brightness information of the target image, and the brightness value of the dark area pixel points is less than a first preset threshold; when the P key pixel points are dark area pixel points, determining that the multiple interpolation points include a first interpolation point, and the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

[0185] For example, if the first preset threshold is 20, it can be considered that the pixel whose value or average value of the Y component in the YUV format is less than 20 is a dark pixel, or the pixel whose value or average value of the R / G / B component in the RGB format is less than 20 is a dark pixel. For another example, if the first preset threshold is 20nit, it can be considered that the pixel whose brightness or average brightness is less than 20nit is a dark pixel.

[0186] Therefore, by indicating a number of key pixels through dynamic metadata, it is possible to determine whether the key pixels belong to dark area pixels based on the average brightness or maximum brightness of the key pixels. In the case where the key pixels are dark area pixels, the first endpoint of the brightness mapping curve is used as the first interpolation point, and the first coordinate value of the first endpoint is determined to be the same as the first coordinate value of the first interpolation point. In other words, in the case where the key pixels are dark area pixels, a section close to the first endpoint on the initial tone factor curve can be corrected.

[0187] In the present application, dark area pixels can be understood as pixels with low brightness. Dark area pixels can be key pixels indicated in dynamic metadata. Dark area pixels can also be determined by the electronic device based on the brightness distribution of the pixels. In other words, whether the target image contains dark area pixels can be determined in different ways.

[0188] Optionally, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the dynamic metadata includes regional brightness indication information, and when the regional brightness indication information indicates a dark area, the multiple interpolation points include a first interpolation point, and a first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

[0189] When the dynamic metadata includes indication information indicating a dark area, the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint. Figure 4 For example, when the indication information is 0, or includes the indication of "dark", it is determined that the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

[0190] like Figure 4 The figure shows an initial tone mapping curve. The two ends of the initial tone mapping curve are 1 point and 5 point respectively. The first endpoint can be Figure 4 1 point shown, the second endpoint can be Figure 4 As shown in the figure, the brightness value corresponding to the first endpoint is low, and the brightness value corresponding to the second endpoint is high. Figure 4As shown, the first interpolation point may be point 1', the first endpoint may be point 1, and the first coordinate value of point 1 is the same as the first coordinate value of point 1'. In some cases, point 1 coincides with point 1', that is, the first coordinate value of point 1 is the same as the first coordinate value of point 1', and the second coordinate value of point 1 is the same as the second coordinate value of point 1'. The second coordinate value of the first interpolation point can be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0191] Next, we need to determine the correction range of the initial tone mapping curve. That is, we can determine a second interpolation point to correct the initial tone mapping curve between the first interpolation point and the second interpolation point to ensure that high dynamic range images can be displayed on display devices with low dynamic range display capabilities or to ensure that the image details of dark scenes are better presented on the display device. Figure 4 As shown, the fifth interpolation point may be point 2'.

[0192] Optionally, the multiple interpolation points also include a second interpolation point, and determining the multiple interpolation points according to the brightness information of the target image includes: determining a first coordinate value of the second interpolation point according to the number of pixels in the dark area.

[0193] When the number of dark area pixels is large, or the proportion of dark area pixels to all pixels is high, the first coordinate value of the second interpolation point is close to the first coordinate value of the first interpolation point. When the number of dark area pixels is small, or the proportion of dark area pixels to all pixels is low, the first coordinate value of the second interpolation point is far away from the first coordinate value of the first interpolation point. That is to say, when the number of dark area pixels is large, the correction range of the initial tone mapping curve close to the first endpoint is small, and when the number of dark area pixels is small and greater than the second preset threshold, the correction range of the initial tone mapping curve close to the first endpoint is large. The second coordinate value of the second interpolation point can be obtained according to an empirical value, a user input value or an initial tone mapping curve.

[0194] In addition, the modified tone mapping curve needs to be smooth and differentiable, so a third interpolation point can be determined so that the spline function used to modify the initial tone mapping curve can be smoothly connected to the initial tone mapping curve at the third interpolation point.

[0195] The correction range for correcting the initial tone mapping curve may include a strong correction range and a weak correction range. The correction range between the first interpolation point and the second interpolation point may be understood as a strong correction range, and the correction range between the second interpolation point and the third interpolation point may be understood as a weak correction range. The weak correction range is a transition correction range between the strong correction range and the uncorrected range. Similarly, the spline function for correcting the initial tone mapping curve may include a strong correction spline function and a weak correction spline function. The spline function between the first interpolation point and the second interpolation point may be understood as a strong correction spline function, and the spline function between the second interpolation point and the third interpolation point may be understood as a weak correction spline function. The weak correction spline function is a transition correction spline function between the strong correction spline function and the initial tone mapping curve. In addition, since the spline function between the second interpolation point and the third interpolation point is different from the initial tone mapping curve, the spline function between the second interpolation point and the third interpolation point has the function of a correction curve in addition to the function of a transition curve. As Figure 4 As shown, the third interpolation point may be point 4'.

[0196] Optionally, the multiple interpolation points also include a third interpolation point, the target image also includes a dark area transition pixel point, the brightness value of the dark area transition pixel point is greater than or equal to the first preset threshold, and the multiple interpolation points are determined according to the brightness information of the target image, including: determining the first coordinate value of the third interpolation point according to the number of the dark area transition pixel points.

[0197] Among them, the dark area transition pixel is a pixel whose brightness is greater than the first preset threshold mentioned above. For example, the first preset threshold is 20, and the pixel whose Y component value in the YUV format is greater than 20 can be considered as a dark area transition pixel, or the pixel whose R / G / B component value in the RGB format is greater than 20 is considered as a dark area transition pixel. For another example, the third preset threshold is 100nit, and the pixel whose brightness is greater than 100nit can be considered as a dark area transition pixel. In some cases, the brightness of the dark area transition pixel will be less than the fifth preset threshold, and the fifth preset threshold is greater than the first preset threshold.

[0198] Optionally, the multiple interpolation points also include a third interpolation point, the dynamic metadata includes Q coordinate information corresponding one-to-one to the Q key pixel points, Q is a positive integer, and when the multiple interpolation points include the first interpolation point, the multiple interpolation points are determined according to the brightness information of the target image and the dynamic metadata, including: determining that the Q key pixel points are dark area transition pixel points according to the Q coordinate information and the brightness information of the target image, and the brightness value of the dark area pixel points is greater than or equal to the first preset threshold; when the Q key pixel points are dark area pixel points, determining that the multiple interpolation points include a third interpolation point, and the first coordinate value of the third interpolation point is the same as the first coordinate value of the first endpoint.

[0199] Among them, the dark area transition pixel is a pixel whose brightness or average brightness is greater than the first preset threshold mentioned above. For example, the first preset threshold is 20, and the pixel whose value or average value of the Y component in the YUV format is greater than 20 can be considered as a dark area transition pixel, or the pixel whose value or average value of the R / G / B component in the RGB format is greater than 20 is a dark area transition pixel. For another example, the third preset threshold is 100nit, and the pixel whose brightness or average brightness is greater than 100nit can be considered as a dark area transition pixel. In some cases, the brightness of the dark area transition pixel will be less than the fifth preset threshold, and the fifth preset threshold is greater than the first preset threshold.

[0200] When the number of dark area transition pixel points is large, or the proportion of dark area transition pixel points to all pixel points is high, the first coordinate value of the third interpolation point is close to the first coordinate value of the second interpolation point. When the number of dark area transition pixel points is small, or the proportion of dark area transition pixel points to all pixel points is low, the first coordinate value of the third interpolation point is far away from the first coordinate value of the second interpolation point. In other words, when the number of dark area transition pixel points is large, the weak correction spline function is shorter, and when the number of dark area transition pixel points is small, the weak correction spline function is longer.

[0201] In the present application, dark area transition pixels can be understood as pixels with dim brightness and brightness greater than dark area pixels. Dark area transition pixels can be key pixels indicated in dynamic metadata. Dark area transition pixels can also be obtained by the electronic device according to the brightness distribution of the pixels. In other words, the number of dark area transition pixels in the target image can be determined in different ways. The second coordinate value of the third interpolation point can be obtained based on empirical values, user input values, and the initial tone mapping curve.

[0202] In scenario 2, when the image is bright, the electronic device can determine that there is an interpolation point at or near the second endpoint. The schematic diagram of the interpolation point and the spline function in this scenario is as follows: Figure 6 shown.

[0203] Optionally, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of bright area pixels contained in the target image is greater than or equal to a fourth preset threshold, the brightness value of the bright area pixels is greater than a third preset threshold, the multiple interpolation points include a fourth interpolation point, and determining the multiple interpolation points based on the brightness information of the target image includes: determining that the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

[0204] In the present application, the brightness value corresponding to the first endpoint may correspond to the first coordinate value of the first endpoint and / or to the second coordinate value of the first endpoint. The brightness value corresponding to the second endpoint may correspond to the first coordinate value of the second endpoint and / or to the second coordinate value of the second endpoint.

[0205] For example, the third preset threshold is 200, and the pixel points whose Y component value in the YUV format is greater than 200 can be considered as bright area pixels, or the pixel points whose R / G / B component values ​​in the RGB format are greater than 200 can be considered as bright area pixels. For another example, the third preset threshold is 400nit, and the pixel points whose brightness is greater than 400nit can be considered as bright area pixels.

[0206] Therefore, when the target image contains a sufficient number of bright area pixels, the second endpoint can be determined as an interpolation point for correcting the bright area (corresponding to the fourth interpolation point in this embodiment), for example, the first coordinate value of the second endpoint is determined to be the same as the first coordinate value of the fourth interpolation point. In other words, when the target image contains a sufficient number of bright area pixels, correction can be performed on a section of the initial tone factor curve close to the second endpoint.

[0207] Optionally, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the dynamic metadata includes P coordinate information corresponding to P key pixel points one by one, P is a positive integer, and the multiple interpolation points are determined according to the brightness information of the target image and the dynamic metadata, including: determining that the P key pixel points are bright area pixel points according to the P coordinate information and the brightness information of the target image, and the brightness value of the bright area pixel points is greater than a third preset threshold; when the P key pixel points are bright area pixel points, determining that the multiple interpolation points include a fourth interpolation point, and the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

[0208] For example, if the third preset threshold is 200, it can be considered that the pixel whose value or average value of the Y component in the YUV format is greater than 200 is a bright pixel, or the pixel whose value or average value of the R / G / B component in the RGB format is greater than 200 is a bright pixel. For another example, if the third preset threshold is 400nit, it can be considered that the pixel whose brightness or average brightness is greater than 400nit is a bright pixel.

[0209] Therefore, by indicating a number of key pixels through dynamic metadata, it is possible to determine whether the key pixels are bright area pixels based on the average brightness or maximum brightness of the key pixels. In the case where the key pixels are bright area pixels, it is determined that the first coordinate value of the second endpoint is the same as the first coordinate value of the fourth interpolation point. In other words, in the case where the key pixels are bright area pixels, a section close to the second endpoint on the initial tone factor curve can be corrected.

[0210] In the present application, bright area pixels can be understood as pixels with high brightness. Bright area pixels can be key pixels indicated in dynamic metadata. Bright area pixels can also be determined by the electronic device based on the brightness distribution of the pixels. In other words, whether the target image contains bright area pixels can be determined in different ways.

[0211] Optionally, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at two ends of the initial tone mapping curve, the dynamic metadata includes regional brightness indication information, and when the regional brightness indication information indicates a bright area, the multiple interpolation points include a fourth interpolation point, and a first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

[0212] When the dynamic metadata includes indication information indicating a bright area, the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint. Figure 4 For example, when the indication information is 1, or includes "light" or other indications related to a bright area, it is determined that the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

[0213] like Figure 4 The figure shows an initial tone mapping curve. The two ends of the initial tone mapping curve are 1 point and 5 point respectively. The first endpoint can be Figure 4 1 point shown, the second endpoint can be Figure 4 As shown in the figure, the brightness value corresponding to the first endpoint is low, and the brightness value corresponding to the second endpoint is high. Figure 4As shown, the fourth interpolation point may be point 5', the first endpoint may be point 5, and the first coordinate value of point 5 is the same as the first coordinate value of point 5'. In some cases, point 5 coincides with point 5', that is, the first coordinate value of point 5 is the same as the first coordinate value of point 5', and the second coordinate value of point 5 is the same as the second coordinate value of point 5'. The second coordinate value of the fourth interpolation point may be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0214] Next, we need to determine the correction range of the initial tone mapping curve. That is, we can determine a fifth interpolation point to correct the initial tone mapping curve between the fourth interpolation point and the fifth interpolation point to ensure that high dynamic range images can be displayed on display devices with low dynamic range display capabilities or to ensure that the image details of bright scenes are better presented on the display device. Figure 4 As shown, the fifth interpolation point may be point 6'.

[0215] Optionally, the multiple interpolation points further include a fifth interpolation point, and determining the multiple interpolation points according to the brightness information of the target image includes: determining a first coordinate value of the fifth interpolation point according to the number of pixels in the bright area.

[0216] When the number of bright area pixels is large, or the proportion of bright area pixels to all pixels is high, the first coordinate value of the fifth interpolation point is close to the first coordinate value of the fourth interpolation point. When the number of bright area pixels is small, or the proportion of bright area pixels to all pixels is low, the first coordinate value of the fifth interpolation point is far away from the first coordinate value of the fourth interpolation point. In other words, when the number of bright area pixels is large, the correction range of the initial tone mapping curve close to the second endpoint is small, and when the number of bright area pixels is small and greater than the fourth preset threshold, the correction range of the initial tone mapping curve close to the second endpoint is large. The second coordinate value of the fifth interpolation point can be obtained based on empirical values, user input values, and the initial tone mapping curve.

[0217] In addition, the modified tone mapping curve needs to be smooth and differentiable, so a sixth interpolation point can be determined so that the spline function used to modify the initial tone mapping curve can be smoothly connected to the initial tone mapping curve at the sixth interpolation point. Figure 4 As shown, the sixth interpolation point may be point 8'.

[0218] The correction range for correcting the initial tone mapping curve may include a strong correction range and a weak correction range. The correction range between the fourth interpolation point and the fifth interpolation point may be understood as a strong correction range, and the correction range between the fifth interpolation point and the sixth interpolation point may be understood as a weak correction range. The weak correction range is a transition correction range between the strong correction range and the uncorrected range. Similarly, the spline function for correcting the initial tone mapping curve may include a strong correction spline function and a weak correction spline function. The spline function between the fourth interpolation point and the fifth interpolation point may be understood as a strong correction spline function, and the spline function between the fifth interpolation point and the sixth interpolation point may be understood as a weak correction spline function. The weak correction spline function is a transition correction spline function between the strong correction spline function and the initial tone mapping curve. In addition, since the spline function between the fifth interpolation point and the sixth interpolation point is different from the initial tone mapping curve, the spline function between the fifth interpolation point and the sixth interpolation point has the function of a correction curve in addition to the function of a transition curve.

[0219] Optionally, the multiple interpolation points also include a sixth interpolation point, the target image also includes a bright area transition pixel point, the brightness value of the bright area transition pixel point is less than or equal to the third preset threshold, and the multiple interpolation points are determined according to the brightness information of the target image, including: determining the first coordinate value of the sixth interpolation point according to the number of the bright area transition pixel points.

[0220] Among them, the bright area transition pixel is a pixel whose brightness is greater than the third preset threshold mentioned above. For example, the third preset threshold is 20, and the pixel whose Y component value in the YUV format is greater than 20 can be considered as a bright area transition pixel, or the pixel whose R / G / B component value in the RGB format is greater than 20 can be considered as a bright area transition pixel. For another example, the third preset threshold is 100nit, and the pixel whose brightness is greater than 100nit can be considered as a bright area transition pixel. In some cases, the brightness of the bright area transition pixel will be greater than the sixth preset threshold, and the sixth preset threshold is less than the third preset threshold.

[0221] Optionally, the multiple interpolation points also include a sixth interpolation point, the dynamic metadata includes Q coordinate information corresponding one-to-one to the Q key pixel points, Q is a positive integer, and when the multiple interpolation points include the fourth interpolation point, the multiple interpolation points are determined according to the brightness information of the target image and the dynamic metadata, including: determining the Q key pixel points as bright area transition pixel points according to the Q coordinate information and the brightness information of the target image, and the brightness value of the bright area pixel points is less than or equal to the first preset threshold; when the Q key pixel points are bright area pixel points, determining that the multiple interpolation points include a sixth interpolation point, and the first coordinate value of the sixth interpolation point is the same as the first coordinate value of the first endpoint.

[0222] Among them, the bright area transition pixel is a pixel whose brightness or average brightness is less than the third preset threshold mentioned above. For example, the third preset threshold is 200, and the pixel whose value or average value of the Y component in the YUV format is less than 200 can be considered as a bright area transition pixel, or the pixel whose value or average value of the R / G / B component in the RGB format is less than 200 is a bright area transition pixel. For another example, the third preset threshold is 400nit, and the pixel whose brightness or average brightness is less than 400nit can be considered as a bright area transition pixel. In some cases, the brightness of the bright area transition pixel will be greater than the sixth preset threshold, and the sixth preset threshold is less than the third preset threshold.

[0223] When the number of bright area transition pixel points is large, or the proportion of bright area transition pixel points to all pixel points is high, the first coordinate value of the third interpolation point is close to the first coordinate value of the sixth interpolation point. When the number of bright area transition pixel points is small, or the proportion of bright area transition pixel points to all pixel points is low, the first coordinate value of the third interpolation point is far from the first coordinate value of the sixth interpolation point. That is, when the number of bright area transition pixel points is large, the weakly corrected spline function is shorter, and when the number of bright area transition pixel points is small, the weakly corrected spline function is longer.

[0224] In the present application, bright area transition pixels can be understood as pixels with brighter brightness and less brightness than bright area pixels. Bright area transition pixels can be key pixels indicated in dynamic metadata. Bright area transition pixels can also be obtained by the electronic device according to the brightness distribution of the pixels. In other words, the number of bright area transition pixels in the target image can be determined in different ways. The second coordinate value of the sixth interpolation point can be obtained based on empirical values, user input values, and the initial tone mapping curve.

[0225] The scene 3 image contains some pixels with small brightness differences. The schematic diagram of the interpolation points and spline function in this scene is as follows: Figure 7 shown.

[0226] Optionally, the target image includes target pixel points having brightness values ​​between a first brightness value and a second brightness value, the proportion of the target pixel points to all pixel points of the target image is greater than an eighth preset threshold, the difference between the first brightness value and the second brightness value is less than a seventh preset threshold, the multiple interpolation points include a twelfth interpolation point, and determining the multiple interpolation points based on the brightness information of the target image includes: determining the twelfth interpolation point based on the average brightness of the target pixel points, the minimum brightness value of the target image, and the maximum brightness value of the target image.

[0227] Luminance values ​​can be represented in YUV, RGB, or luminance.

[0228] For example, the maximum brightness value of the target image is 500nit, and the minimum brightness value is 100nit. Assume that the seventh preset threshold is 50nit and the eighth preset threshold is 30%. 50% of the pixels in the target image have brightness values ​​within the range of 240 to 260nit, that is, the first brightness value is 240nit, and the second brightness value is 260nit. The target pixel accounts for 50% of all the pixels in the target image. It can be seen that the difference between the first brightness value and the second brightness value is less than the seventh preset threshold, and the target pixel accounts for a greater proportion of all the pixels in the target image than the eighth preset threshold.

[0229] Assume that the average brightness of the target pixel is 250 nit.

[0230] In one example, the first coordinate value of the twelfth interpolation point or the second coordinate value of the twelfth interpolation point can be determined to be 250 / (500-100), that is, 3 / 8, or the first coordinate value of the twelfth interpolation point or the second coordinate value of the twelfth interpolation point can be determined to be a value close to 3 / 8. Figure 4 The second coordinate value of the twelfth interpolation point can be obtained according to the empirical value, the user input value, and the initial tone mapping curve.

[0231] In one example, it can be determined that the multiple interpolation points include a twelfth interpolation point and a thirteenth interpolation point, wherein the first coordinate value of the twelfth interpolation point is less than 3 / 8, and the first coordinate value of the thirteenth interpolation point is greater than 3 / 8, or the second coordinate value of the twelfth interpolation point is less than 3 / 8, and the second coordinate value of the thirteenth interpolation point is greater than 3 / 8. The twelfth interpolation point is, for example, Figure 4 The 13th interpolation point can be Figure 4 The second coordinate values ​​of the twelfth interpolation point and the thirteenth interpolation point can be obtained according to the empirical value, the user input value, and the initial tone mapping curve.

[0232] Therefore, when the target image contains a sufficient number of target pixels, it means that the display device may not be able to highlight the light and dark contrast between the target pixels. In order to vividly display the target pixels, the initial tone mapping curve can be corrected within the brightness range corresponding to the target pixels.

[0233] Optionally, the dynamic metadata includes coordinate information of the target pixel point.

[0234] That is, the target pixel point can be a pixel point indicated in the dynamic metadata, or can be determined by the electronic device itself according to the brightness distribution of the pixel point. In other words, it can be determined in different ways whether the target image contains the target pixel point.

[0235] Next, we need to determine the correction range of the initial tone mapping curve. That is, we can determine a thirteenth interpolation point, or determine the range between the twelfth interpolation point and the thirteenth interpolation point, so as to correct the initial tone mapping curve between the twelfth interpolation point and the thirteenth interpolation point to ensure the display effect of the picture on the display device. Figure 4 As shown, the thirteenth interpolation point can be Figure 4 11' point in the match.

[0236] Optionally, the multiple interpolation points further include a thirteenth interpolation point, and determining the multiple interpolation points according to the brightness information of the target image includes: determining the thirteenth interpolation point according to the number of the target pixel points.

[0237] When the number of target pixels is large, or the proportion of target pixels to all pixels is high, the distance between the twelfth interpolation point and the thirteenth interpolation point is small. When the number of target pixels is small, or the proportion of target pixels to all pixels is low, the distance between the twelfth interpolation point and the thirteenth interpolation point is large. That is to say, when the number of target pixels is relatively large, the correction range of the initial tone mapping curve is small, and when the number of target pixels is relatively small, the correction range of the initial tone mapping curve is large. The second coordinate value of the thirteenth interpolation point can be obtained according to the empirical value, the user input value, and the initial tone mapping curve.

[0238] In addition, the modified tone mapping curve needs to be smooth and differentiable, so a fourteenth interpolation point can be determined so that the spline function used to modify the initial tone mapping curve can be smoothly connected to the initial tone mapping curve at the fourteenth interpolation point.

[0239] Optionally, the multiple interpolation points also include a fourteenth interpolation point, which is the intersection point of the smooth transition between the spline function and the initial tone mapping curve, and the target image also includes a target transition pixel point, and the brightness value of the target transition pixel point is less than or equal to the first brightness value, or the brightness value of the target transition pixel point is greater than or equal to the second brightness value, and the first brightness value is less than the second brightness value. The determining of the multiple interpolation points based on the brightness information of the target image includes: determining the first coordinate value of the fourteenth interpolation point based on the number of the target transition pixel points.

[0240] In one example, the target transition pixel point may be a pixel point with a brightness greater than the target pixel point, then the first coordinate value of the fourteenth interpolation point is greater than the first coordinate value of the twelfth interpolation point, and greater than the first coordinate value of the thirteenth interpolation point. Figure 4 As shown, the fourteenth interpolation point can be Figure 4 12' point in the middle.

[0241] In one example, the target transition pixel point may be a pixel point with a brightness lower than the target pixel point, then the first coordinate value of the fourteenth interpolation point is lower than the first coordinate value of the twelfth interpolation point, and lower than the first coordinate value of the thirteenth interpolation point. Figure 4 As shown, the fourteenth interpolation point can be Figure 4 9' point in the middle.

[0242] When the number of target transition pixels is large, or the proportion of target transition pixels to all pixels is high, the distance between the thirteenth interpolation point and the fourteenth interpolation point is small. When the number of target transition pixels is small, or the proportion of target transition pixels to all pixels is low, the distance between the thirteenth interpolation point and the fourteenth interpolation point is large. That is to say, when the number of target transition pixels is relatively large, the correction range of the initial tone mapping curve is relatively small, and when the number of target pixels is relatively small, the correction range of the initial tone mapping curve is relatively large. The second coordinate value of the fourteenth interpolation point can be obtained based on the empirical value, the user input value, and the initial tone mapping curve.

[0243] Optionally, the dynamic metadata includes coordinate information of the target transition pixel point.

[0244] That is, the target transition pixel point can be a pixel point indicated in the dynamic metadata, or can be determined by the electronic device itself according to the brightness distribution of the pixel point. In other words, it can be determined in different ways whether the target image contains the target pixel point.

[0245] Optionally, the dynamic metadata includes coordinate information of each interpolation point in the multiple interpolation points.

[0246] The coordinate information of the interpolation point may be the coordinate of the interpolation point, or the first coordinate value or the second coordinate value of the interpolation point. The coordinate of the interpolation point may be expressed as (x, y), where x may represent the first coordinate value and y may represent the second coordinate value. The electronic device may determine multiple interpolation points based on the brightness information of the image, or based on the dynamic metadata of the image, or based on the brightness information of the image and the dynamic metadata of the image. For example, the first coordinate value of interpolation point 1, the second coordinate value of interpolation point 2, and interpolation point 3 are determined.

[0247] For example, dynamic metadata includes Figure 4The first coordinate value of point 1', the coordinate of point 2' and the second coordinate value of point 4'. Among them, according to the first coordinate value of point 1' given by the dynamic metadata, it can be judged that point 1' may be a point that coincides with point 1 or coincides with the origin, thereby obtaining the coordinate of point 1'. In addition, the coordinate of point 2' is given in the dynamic metadata. Moreover, according to the second coordinate value of point 4' given by the dynamic metadata, the point whose ordinate is the same as the second coordinate value can be found on the initial tone mapping function, and the abscissa corresponding to the point is determined to be the first coordinate value of point 4', thereby obtaining the coordinate of point 4'. The common method for determining the spline function can be used to obtain the spline function passing through points 1', 2' and 4'.

[0248] For example, dynamic metadata includes Figure 4 The first coordinate value of point 5', the coordinate value of point 6', and the second coordinate value of point 8'. According to the first coordinate value of point 5' given by the dynamic metadata, it can be judged that point 5' can be a point that coincides with point 5 or coincides with (1,1), thereby obtaining the coordinate value of point 5'. In addition, the coordinate value of point 6' is given in the dynamic metadata. Moreover, according to the second coordinate value of point 8' given by the dynamic metadata, the first coordinate value of point 8' can be determined on the initial tone mapping function and the coordinate value of point 8' can be obtained. The common method for determining the spline function can be used to obtain the spline function passing through points 5', 6', and 8'.

[0249] For example, dynamic metadata includes Figure 4 The second coordinate value of point 9', the coordinate of point 10', and the first coordinate value of point 12'. Among them, the first coordinate value of point 12' can be determined on the initial tone mapping function and the coordinate of point 12' can be obtained according to the second coordinate value of point 9' given by the dynamic metadata. In addition, the coordinate of point 10' has been given in the dynamic metadata. And, according to the first coordinate value of point 12' given by the dynamic metadata, the second coordinate value of point 12' can be determined on the initial tone mapping function and the coordinate of point 12' can be obtained. The common method for determining the spline function can be used to obtain the spline function passing through points 9', 10', and 12'.

[0250] 203 : Determine a spline function passing through the multiple interpolation points, where the spline function is smoothly connected to the initial tone mapping curve at at least one interpolation point among the multiple interpolation points.

[0251] The so-called spline function is a piecewise polynomial function. The spline function S(x) defined in the interval [a,b] is continuous in the interval [a,b], and the first-order derivative S'(x) and second-order derivative S" (x) of the spline function are both continuous in the interval [a,b]. Figure 4The spline function 1 passing through points 1', 2', and 4', the spline function 2 passing through points 5', 6', and 8', and the spline function 3 passing through points 9', 10', and 12' as shown.

[0252] After determining multiple interpolation points, a spline function passing through the multiple interpolation points can be obtained based on the coordinates of each interpolation point among the multiple interpolation points. For Figure 4 the spline function 1 in, the multiple interpolation points are points 1', 2', and 4'; for Figure 4 the spline function 2 in, the multiple interpolation points are points 5', 6', and 8'; for Figure 4 the spline function 3 in, the multiple interpolation points are points 9', 10', and 12'.

[0253] The interval [a, b] contains n nodes x1,..., x n , where a < x1 <... < x n . When the spline function is an N - th order spline function, in the interval [x k , x k+1 between any two adjacent nodes, the function is an N - th order polynomial function, where k < n - 1 and k is a positive integer.

[0254] When the spline function includes spline function 1 and spline function 2, spline function 1 can be an M - th order spline function in the interval [x M1 , x M2 , and spline function 2 can be an L - th order spline function in the interval [x L1 , x L2 , where the intersection of the interval [x M1 , x M2 and the interval [x L1 , x L2 is empty, or there is exactly one value in the intersection of the interval [x M1 , x M2 and the interval [x L1 , x L2 . M can be the same as L or different from L.

[0255] When the spline function includes a first - order spline function, at least two known conditions are required to determine the first - order spline function. For example, the coordinates of the points passing through the first - order spline function and the slope of the spline function are known in advance. Another example is that the coordinates of two points passing through the first - order spline function are known in advance.

[0256] When the spline function includes a second - order spline function, at least three known conditions are required to determine the second - order spline function. For example, the function form of the second - order spline function is y = ax 2+ bx + c, and a ≠ 0. To determine this quadratic spline function, the coordinates of three points passing through this quadratic spline function need to be known in advance; or, the coordinates of two points and the derivative of the quadratic spline function at one of the points need to be known in advance.

[0257] When the spline function includes a cubic spline function, at least four known conditions are required to determine this cubic spline function. For example, the functional form of this quadratic spline function is y = ax 3 + bx 2 + cx + d, and a ≠ 0. To determine this cubic spline function, the coordinates of four points passing through this cubic spline function need to be known in advance; or, the coordinates of two points and the derivatives of the cubic spline function at these two points need to be known in advance.

[0258] When the spline function includes an N - th spline function, at least N + 1 known conditions are required to determine this N - th spline function.

[0259] Assume that the initial tone - mapping curve is T(x), and the spline function S(x) is defined in the interval [a, b].

[0260] In one example, the spline function S(x) can be smoothly connected to the initial tone - mapping curve T(x) at x = a, or can be smoothly connected to the initial tone - mapping curve T(x) at x = b, or can be smoothly connected to the initial tone - mapping curve T(x) both at x = a and at x = b.

[0261] In one example, multiple interpolation points include a point with the first coordinate value x0, where a < x0 < b, and the spline function S(x) can be smoothly connected to the initial tone - mapping curve T(x) at x = x0.

[0262] The spline function S(x) is smoothly connected to the initial tone - mapping curve T(x) at x = a, that is, S(a) = T(a), and S’(a) = T’(a).

[0263] The spline function S(x) is smoothly connected to the initial tone - mapping curve T(x) at x = b, that is, S(b) = T(b), and S’(b) = T’(b).

[0264] The spline function S(x) is smoothly connected to the initial tone - mapping curve T(x) at x = x0, that is, S(x0) = T(x0), and S’(x0) = T’(x0).

[0265] The purpose of the smooth connection between the spline function and the initial tone - mapping curve at an interpolation point is that by using the spline function to replace a part of the initial tone - mapping curve, a modified tone - mapping curve can be obtained. Since the spline function is smoothly connected to the initial tone - mapping curve at the interpolation point, this modified tone - mapping curve can be a smooth function.

[0266] Optionally, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, and determining the spline function passing through the multiple interpolation points includes: obtaining a first spline function passing through the seventh interpolation point and the eighth interpolation point; determining a second spline function passing through the eighth interpolation point and the ninth interpolation point, the second spline function being smoothly connected to the first spline function at the eighth interpolation point, and the second spline function being smoothly connected to the initial tone mapping curve at the ninth interpolation point.

[0267] In the above scenes 1, 2 and 3, the spline function may include a first spline function and a second spline function, the first spline function is a strong correction spline function, and the second spline function is used to achieve a smooth transition between the first spline function and the initial brightness mapping curve.

[0268] Obtaining the first spline function. This may be receiving parameters input by a user, or using pre-set parameters to determine the first spline function. For example, the first spline function is pre-set as a linear spline function, and the slope of the linear spline function is 1.

[0269] Optionally, the first spline function may also be determined in combination with the coordinates of the seventh interpolation point and / or the eighth interpolation point.

[0270] The first spline function may also be a quadratic spline function or a cubic spline function. The first spline function is determined in combination with the conditions required for determining a quadratic spline function or a cubic spline function mentioned above.

[0271] That is, when determining the first spline function, in addition to combining the acquired data, some empirical values ​​or user instructions may be introduced to determine the first spline function.

[0272] Then, the user determines the second spline function that smoothly connects the first spline function and the initial tone mapping curve. The first spline function is determined before the second spline function is determined, so the function value of the first spline function at the eighth interpolation point is known, and the derivative value of the first spline function at the eighth interpolation point is known; the second spline function is smoothly connected to the first spline function at the eighth interpolation point, which means that the function value of the first spline function at the eighth interpolation point is the same as the function value of the second spline function at the eighth interpolation point, and the derivative value of the first spline function at the eighth interpolation point is the same as the derivative value of the second spline function at the eighth interpolation point. In addition, the function value of the initial tone mapping function at the ninth interpolation point is known, and the derivative value of the initial tone mapping function at the ninth interpolation point is known; the second spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point, which means that the function value of the second spline function at the ninth interpolation point is the same as the function value of the initial tone mapping function at the ninth interpolation point, and the derivative value of the second spline function at the ninth interpolation point is the same as the derivative value of the initial tone mapping function at the ninth interpolation point.

[0273] For the convenience of description, in this application, unless otherwise specified, the “derivative value” may be understood as a first-order derivative value.

[0274] Optionally, the first spline function is a linear spline function, and the second spline function is a tertiary spline function.

[0275] Optionally, the derivative of the spline function is always greater than 0.

[0276] That is to say, the derivative of the spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the final spline function and the initial tone mapping curve, thus avoiding over-correction.

[0277] Optionally, the derivative of the first spline function is always greater than 0.

[0278] That is to say, the derivative of the first spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the final first spline function and the initial tone mapping curve, thereby avoiding over-correction.

[0279] Optionally, the derivative of the second spline function is always greater than 0.

[0280] That is to say, the derivative of the second spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the final second spline function and the initial tone mapping curve, thereby avoiding over-correction.

[0281] Optionally, the multiple interpolation points further include an eleventh interpolation point, the first spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0282] The point where the second-order derivative is 0 may be the inflection point of the first spline function. For example, the first spline function increases rapidly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point.

[0283] Optionally, the multiple interpolation points further include an eleventh interpolation point, the second spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0.

[0284] The point where the second-order derivative is 0 can be the inflection point of the first spline function. For example, the second spline function increases rapidly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. Figure 4 As shown, the eleventh interpolation point may be the 3' point.

[0285] Optionally, the multiple interpolation points further include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0286] On one side of the fifteenth interpolation point, the first spline function is located above the initial tone mapping function, and on the other side of the fifteenth interpolation point, the first spline function is located below the initial tone mapping function. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0287] Optionally, the multiple interpolation points further include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0288] On one side of the fifteenth interpolation point, the second spline function is located above the initial tone mapping function, and on the other side of the fifteenth interpolation point, the second spline function is located below the initial tone mapping function. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0289] Scene 1

[0290] When the image is dark, the portion of the initial tone mapping curve close to the first endpoint can be corrected. The seventh interpolation point can be the first interpolation point mentioned above, the eighth interpolation point can be the second interpolation point mentioned above, and the ninth interpolation point can be the third interpolation point mentioned above. In this scenario, the spline function includes a first spline function passing through the first interpolation point and the second interpolation point and a second spline function passing through the second interpolation point and the third interpolation point, wherein the first spline function is a strongly corrected spline function, and the second spline function is used to achieve a smooth transition between the first spline function and the initial brightness mapping curve. The positions of the specific interpolation points and the spline functions are as follows: Figure 5 shown.

[0291] The above has described how to obtain the first coordinate values ​​of the first interpolation point, the second interpolation point, and the third interpolation point. That is, the first coordinate values ​​of the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point are all known.

[0292] First, obtain the seventh interpolation point and the eighth interpolation point (i.e., through Figure 4 The first spline function f1(x) (point 1', point 2').

[0293] For example, the first spline function is preset as a linear spline function with a slope of 1. Since the first coordinate value of the seventh interpolation point is the same as the first coordinate value of the first endpoint of the initial tone mapping function, it can be determined that the first spline function is a function passing through the origin, that is, the coordinate of the seventh interpolation point is (0,0). Therefore, the first spline function can be expressed as f1(x)=x.

[0294] Then, determine the eighth interpolation point and the ninth interpolation point (in this embodiment, the eighth interpolation point is Figure 4 The 2' point and the ninth interpolation point are Figure 4 The second spline function f2(x) of the 4' point in the figure.

[0295] According to the first spline function f1(x) and the first coordinate value x2 of the eighth interpolation point, the function of the first spline function at the eighth interpolation point can be determined to be f1(x2) and the derivative f1'(x2) of the first spline function at the eighth interpolation point.

[0296] According to the initial tone mapping function f0(x) and the first coordinate value x4 of the ninth interpolation point, the function of the second spline function at the ninth interpolation point can be determined to be f0(x4) and the derivative of the first spline function at the ninth interpolation point f0'(x4).

[0297] According to f1(x2)=f2(x2), f1'(x2)=f2'(x2), f0(x4)=f2(x4), f0'(x4)=f2'(x4), the second spline function f2(x) is determined.

[0298] Optionally, the second spline function is a cubic spline function.

[0299] Optionally, the multiple interpolation points further include an eleventh interpolation point, the first spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0300] Optionally, the plurality of interpolation points further include an eleventh interpolation point, the second spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be Figure 4 3' point shown.

[0301] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the first spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0302] Optionally, the plurality of interpolation points further include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the second spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be Figure 4 3' point shown.

[0303] Scene 2

[0304] When the image is bright, the portion of the initial tone mapping curve close to the second endpoint can be corrected. In this case, the seventh interpolation point can be the fourth interpolation point mentioned above, the eighth interpolation point can be the fifth interpolation point mentioned above, and the ninth interpolation point can be the sixth interpolation point mentioned above. In this scenario, the spline function includes a first spline function passing through the fourth interpolation point and the fifth interpolation point and a second spline function passing through the fifth interpolation point and the sixth interpolation point, wherein the first spline function is a strongly corrected spline function, and the second spline function is used to achieve a smooth transition between the first spline function and the initial brightness mapping curve. The positions of the specific interpolation points and the spline functions are as follows: Figure 6 shown.

[0305] The above has described how to obtain the first coordinate values ​​of the fourth interpolation point, the fifth interpolation point, and the sixth interpolation point. That is, the first coordinate values ​​of the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point are all known.

[0306] First, obtain the seventh interpolation point and the eighth interpolation point (i.e., through Figure 4 The first spline function f1(x) of (points 5' and 6').

[0307] For example, the first spline function is preset as a linear spline function with a slope of 1. Since the first coordinate value of the seventh interpolation point is the same as the first coordinate value of the first endpoint of the initial tone mapping function, it can be determined that the first spline function is a function passing through (1,1), that is, the coordinate of the seventh interpolation point is (1,1). Therefore, the first spline function can be expressed as f1(x)=x.

[0308] Then, determine the eighth interpolation point, the ninth interpolation point (i.e., the Figure 4 The second spline function f2(x) (points 6' and 8' in the middle).

[0309] According to the first spline function f1(x) and the first coordinate value x6 of the eighth interpolation point, it can be determined that the function of the first spline function at the eighth interpolation point is f1(x6) and the derivative of the first spline function at the eighth interpolation point is f1'(x6).

[0310] According to the initial tone mapping function f0(x) and the first coordinate value x8 of the ninth interpolation point, the function of the second spline function at the ninth interpolation point can be determined to be f0(x8) and the derivative of the first spline function at the ninth interpolation point f0'(x8).

[0311] According to f1(x6)=f2(x6), f1'(x6)=f2'(x6), f0(x8)=f2(x8), f0'(x8)=f2'(x8), the second spline function f2(x) is determined.

[0312] Optionally, the second spline function is a cubic spline function.

[0313] Optionally, the multiple interpolation points further include an eleventh interpolation point, the first spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0314] Optionally, the plurality of interpolation points further include an eleventh interpolation point, the second spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be Figure 4 7' point shown.

[0315] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the first spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0316] Optionally, the plurality of interpolation points further include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the second spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be Figure 4 7' point shown.

[0317] Scene 3

[0318] The seventh interpolation point may be the twelfth interpolation point mentioned above, the eighth interpolation point may be the thirteenth interpolation point mentioned above, and the ninth interpolation point may be the fourteenth interpolation point mentioned above. In this scenario, the spline function includes a first spline function passing through the twelfth interpolation point and the thirteenth interpolation point and a second spline function passing through the thirteenth interpolation point and the fourteenth interpolation point, wherein the first spline function is a strongly modified spline function, and the second spline function is used to achieve a smooth transition between the first spline function and the initial brightness mapping curve. The positions of the specific interpolation points and the spline functions are as follows: Figure 8 shown.

[0319] The above has described how to obtain the first coordinate values ​​of the twelfth interpolation point, the thirteenth interpolation point, and the fourteenth interpolation point. That is, the first coordinate values ​​of the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point are all known.

[0320] First, obtain the seventh interpolation point and the eighth interpolation point (i.e., through Figure 4 The first spline function f1(x) (points 10' and 11' in the figure).

[0321] For example, the first spline function is preset as a linear spline function with a slope of 1. Since the seventh interpolation point is located below the initial tone mapping function and the eighth interpolation point is located above the initial tone mapping function, assuming that the first coordinate value of the seventh interpolation point is x 10 , the first coordinate value of the eighth interpolation point is x 11 , the first spline function passes through The initial tone mapping function is f0(x). Therefore,

[0322] Then, determine the eighth interpolation point, the ninth interpolation point (i.e., the Figure 4 At 11', 12', or after Figure 4 The second spline function f2(x) (points 10' and 9' in the middle).

[0323] According to the first spline function f1(x) and the first coordinate value x of the eighth interpolation point 11 , we can determine that the function of the first spline function at the eighth interpolation point is f1(x 11 ), and the derivative of the first spline function at the eighth interpolation point f1'(x 11 ).

[0324] According to the initial tone mapping function f0(x) and the first coordinate value x of the ninth interpolation point 12 , we can determine that the function of the second spline function at the ninth interpolation point is f0(x 12 ), and the derivative of the first spline function at the ninth interpolation point f0'(x 12 ).

[0325] According to f1(x 11 )=f2(x 11 ), f1'(x 11 )=f2'(x 11 ), f0(x 12 )=f2(x 12 ), f0'(x 12 )=f2'(x 12 ), determine the second spline function f2(x).

[0326] Optionally, the second spline function is a cubic spline function.

[0327] Optionally, the multiple interpolation points further include an eleventh interpolation point, the first spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0328] Optionally, the plurality of interpolation points further include an eleventh interpolation point, the second spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be Figure 4 13' point shown.

[0329] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the first spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0330] Optionally, the plurality of interpolation points further include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the second spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be Figure 4 13' point shown.

[0331] Optionally, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, the first coordinate value of the eighth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point, and the first coordinate value of the eighth interpolation point is a first correction position; before determining the multiple interpolation points, the method further includes: obtaining a first offset, the first offset being used to indicate a correction amplitude of the initial tone mapping curve at the first correction position degrees, the first correction position is the first coordinate value of the eighth interpolation point; the determining of multiple interpolation points includes: determining the second coordinate value of the eighth interpolation point according to the function value of the initial tone mapping curve at the first correction position and the first offset; the determining of the spline function passing through the multiple interpolation points includes: determining the third spline function according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point. Obtaining the first offset can be a parameter received from the user or a pre-set parameter. For example, the first offset is pre-set to 0.01.

[0332] The first coordinate value of the eighth interpolation point corresponds to the first correction position, that is, the eighth interpolation point deviating from the initial tone mapping function is determined at the first correction position, that is, the second coordinate value of the eighth interpolation point is not equal to the function value of the initial tone mapping function at the first correction position.

[0333] The first offset may represent the degree of offset of the eighth interpolation point relative to the initial tone mapping function. When the first offset is positive, the eighth interpolation point is located above the initial tone mapping function; when the first offset is negative, the eighth interpolation point is located below the initial tone mapping function.

[0334] For example, the second coordinate value of the eighth interpolation point=the first offset+the function value of the initial tone mapping function at the first correction position.

[0335] For example, the shortest distance from the eighth interpolation point to the initial tone mapping function = the first offset.

[0336] Therefore, according to the first offset, the coordinates of the eighth interpolation point can be determined.

[0337] Then the third spline function is determined.

[0338] In one example, the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and the ninth interpolation point. It can be considered that the seventh interpolation point and the ninth interpolation point are both points on the initial tone mapping function, so the coordinates of the seventh interpolation point and the coordinates of the ninth interpolation point can be obtained; and the derivative of the third spline function at the seventh interpolation point is equal to the derivative of the initial tone mapping function at the seventh interpolation point, and the derivative of the third spline function at the ninth interpolation point is equal to the derivative of the initial tone mapping function at the ninth interpolation point. Due to the properties of the spline function itself, the third spline function is continuous at the eighth interpolation point, and the first-order derivative of the third spline function at the eighth interpolation point is continuous, and the second-order derivative of the third spline function at the eighth interpolation point is continuous. Therefore, the third spline function can be divided into a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point. The function value of the fourth spline function at the eighth interpolation point is the same as the function value of the fifth spline function at the eighth interpolation point, the first-order derivative value of the fourth spline function at the eighth interpolation point is the same as the first-order derivative value of the fifth spline function at the eighth interpolation point, and the second-order derivative value of the fourth spline function at the eighth interpolation point is the same as the second-order derivative value of the fifth spline function at the eighth interpolation point.

[0339] In the above scenarios one, two, and three, the spline functions passing through multiple interpolation points are collectively referred to as the third spline function. The multiple interpolation points in these three scenarios include the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point. The spline function passing through the seventh interpolation point and the eighth interpolation point is the fourth spline function, and the spline function passing through the seventh interpolation point and the eighth interpolation point is the fifth spline function.

[0340] Optionally, the derivative of the fourth spline function is always greater than 0.

[0341] That is to say, the derivative of the fourth spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the final first spline function and the initial tone mapping curve, thereby avoiding overcorrection.

[0342] Optionally, the derivative of the fifth spline function is always greater than 0.

[0343] That is to say, the derivative of the fifth spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the final second spline function and the initial tone mapping curve, thereby avoiding over-correction.

[0344] Optionally, the multiple interpolation points further include an eleventh interpolation point, the fourth spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0.

[0345] The point where the second-order derivative is 0 may be the inflection point of the fourth spline function. For example, the fourth spline function increases rapidly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. Figure 4 As shown, the eleventh interpolation point may be the 3' point.

[0346] Optionally, the multiple interpolation points further include an eleventh interpolation point, the fifth spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0.

[0347] The point where the second-order derivative is 0 may be the inflection point of the fifth spline function. For example, the fifth spline function increases rapidly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. Figure 4 As shown, the eleventh interpolation point may be the 3' point.

[0348] Optionally, the multiple interpolation points further include a fifteenth interpolation point, the fourth spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0349] On one side of the fifteenth interpolation point, the fourth spline function is located above the initial tone mapping function, and on the other side of the fifteenth interpolation point, the fourth spline function is located below the initial tone mapping function. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0350] Optionally, the multiple interpolation points further include a fifteenth interpolation point, the fifth spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0351] On one side of the fifteenth interpolation point, the fifth spline function is located above the initial tone mapping function, and on the other side of the fifteenth interpolation point, the fifth spline function is located below the initial tone mapping function. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0352] Optionally, the multiple interpolation points also include a tenth interpolation point, the third spline function passes through the tenth interpolation point, and the first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point. Before determining the multiple interpolation points, the method also includes: obtaining a second offset, the second offset is used to represent the correction amplitude of the initial tone mapping curve at a second correction position, and the second correction position is the first coordinate value of the tenth interpolation point; the determining of the multiple interpolation points also includes: determining the second coordinate value of the tenth interpolation point according to the function value of the initial tone mapping curve at the second correction position and the second offset; the determining of the third spline function according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point includes: determining the third spline function according to the first coordinate value and the second coordinate value of the eighth interpolation point and the first coordinate value and the second coordinate value of the tenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

[0353] The second offset may be obtained by receiving a parameter input by a user or by using a preset parameter, for example, the second offset is preset to 0.01.

[0354] The first coordinate value of the tenth interpolation point corresponds to the second correction position, that is, the tenth interpolation point deviates from the initial tone mapping function at the second correction position, that is, the second coordinate value of the tenth interpolation point is not equal to the function value of the initial tone mapping function at the second correction position.

[0355] The second offset may represent the offset degree of the tenth interpolation point relative to the initial tone mapping function. When the second offset is positive, the tenth interpolation point is located above the initial tone mapping function; when the second offset is negative, the tenth interpolation point is located below the initial tone mapping function.

[0356] For example, the second coordinate value of the tenth interpolation point=the second offset+the function value of the initial tone mapping function at the second correction position.

[0357] For example, the shortest distance from the tenth interpolation point to the initial tone mapping function = the second offset.

[0358] Optionally, the first offset and the second offset are reciprocal numbers of each other.

[0359] Therefore, according to the second offset, the coordinates of the tenth interpolation point can be determined.

[0360] Then further determine the third spline function. Due to the properties of the spline function itself, the third spline function is continuous at the tenth interpolation point, and the first-order derivative of the third spline function at the tenth interpolation point is continuous, and the second-order derivative of the third spline function at the tenth interpolation point is continuous. Therefore, the third spline function can be divided into a sixth spline function passing through the seventh interpolation point and the tenth interpolation point, and a seventh spline function passing through the tenth interpolation point and the ninth interpolation point. The function value of the sixth spline function at the tenth interpolation point is the same as the function value of the seventh spline function at the tenth interpolation point, the first-order derivative value of the sixth spline function at the tenth interpolation point is the same as the first-order derivative value of the seventh spline function at the tenth interpolation point, and the second-order derivative value of the sixth spline function at the tenth interpolation point is the same as the second-order derivative value of the seventh spline function at the tenth interpolation point.

[0361] Optionally, the derivative of the sixth spline function is always greater than 0.

[0362] That is to say, the derivative of the sixth spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the first spline function finally obtained and the initial tone mapping curve, thereby avoiding over-correction.

[0363] Optionally, the derivative of the seventh spline function is always greater than 0.

[0364] That is to say, the derivative of the seventh spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the final second spline function and the initial tone mapping curve, thereby avoiding overcorrection.

[0365] Optionally, the multiple interpolation points further include an eleventh interpolation point, the sixth spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0.

[0366] The point where the second-order derivative is 0 may be the inflection point of the sixth spline function. For example, the sixth spline function increases rapidly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. Figure 4 As shown, the eleventh interpolation point may be the 3' point.

[0367] Optionally, the multiple interpolation points further include an eleventh interpolation point, the seventh spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0.

[0368] The point where the second-order derivative is 0 may be the inflection point of the seventh spline function. For example, the seventh spline function increases rapidly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. Figure 4 As shown, the eleventh interpolation point may be the 3' point.

[0369] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the fourth spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fourth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0370] On one side of the fifteenth interpolation point, the fourth spline function is located above the initial tone mapping function, and on the other side of the fifteenth interpolation point, the fourth spline function is located below the initial tone mapping function. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0371] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the fifth spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fifth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0372] On one side of the fifteenth interpolation point, the fifth spline function is located above the initial tone mapping function, and on the other side of the fifteenth interpolation point, the fifth spline function is located below the initial tone mapping function. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0373] In one example, the third spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point. The ninth interpolation point can be considered as a point on the initial tone mapping function, so the coordinates of the ninth interpolation point can be obtained. In addition, the derivative of the third spline function at the ninth interpolation point is equal to the derivative of the initial tone mapping function at the ninth interpolation point. Due to the properties of the spline function itself, the first-order derivative of the third spline function at the eighth interpolation point is continuous, and the second-order derivative of the third spline function at the eighth interpolation point is continuous. Therefore, the third spline function can be divided into a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point. The first-order derivative value of the fourth spline function at the eighth interpolation point is the same as the first-order derivative value of the fifth spline function at the eighth interpolation point, and the second-order derivative value of the fourth spline function at the eighth interpolation point is the same as the second-order derivative value of the fifth spline function at the eighth interpolation point. Therefore, it is also necessary to determine the coordinates of the seventh interpolation point.

[0374] The coordinates of the seventh interpolation point may be determined based on empirical values, for example, (0,0) or (1,1) may be determined as the coordinates of the seventh interpolation point.

[0375] The case where the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point is similar to the case where the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point, and no further details are given here.

[0376] Optionally, the derivative of the third spline function is always greater than 0.

[0377] That is to say, the derivative of the spline function is always greater than 0. With this restriction, it is possible to avoid a large gap between the final spline function and the initial tone mapping curve, thus avoiding over-correction.

[0378] Optionally, the multiple interpolation points further include an eleventh interpolation point, the third spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0379] The point where the second-order derivative is 0 may be the inflection point of the first spline function. For example, the third spline function increases rapidly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point.

[0380] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the third spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the third spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0381] On one side of the fifteenth interpolation point, the third spline function is located above the initial tone mapping function, and on the other side of the fifteenth interpolation point, the third spline function is located below the initial tone mapping function. By limiting the position of the fifteenth interpolation point, the correction method of correcting the initial tone mapping function can be limited.

[0382] Scene 1

[0383] When the image is dark, the portion of the initial tone mapping curve close to the first endpoint can be corrected. The seventh interpolation point can be the first interpolation point mentioned above, the eighth interpolation point can be the second interpolation point mentioned above, and the ninth interpolation point can be the third interpolation point mentioned above. The positions of the specific interpolation points and the spline function are as follows: Figure 5 shown.

[0384] The above has described how to obtain the first coordinate values ​​of the first interpolation point, the second interpolation point, and the third interpolation point. That is, the first coordinate values ​​of the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point are all known.

[0385] First, a first offset is obtained. For example, the first offset is preset to be 0.01.

[0386] Then, according to the first offset, the eighth interpolation point (such as Figure 4 For example, f3(x2)=f0(x2)+0.01, where f0(x) represents the initial tone mapping function.

[0387] Since the first coordinate value of the seventh interpolation point is the same as the first coordinate value of the first endpoint of the initial tone mapping function, it can be determined that the third spline function is a function passing through the origin, that is, the coordinate of the seventh interpolation point is (0,0).

[0388] Then, determine the seventh interpolation point, the eighth interpolation point, the ninth interpolation point (i.e., the Figure 4 1' point, 2'

[0389] The third spline function f3(x) can be divided into a fourth spline function f4(x) passing through the seventh interpolation point and the eighth interpolation point and a fifth spline function f5(x) passing through the eighth interpolation point and the ninth interpolation point.

[0390] According to the coordinates of the seventh interpolation point and the coordinates of the eighth interpolation point, it can be determined that f4(x1)=0, f4(x2)=

[0391] f0(x2)+0.01, f5(x2)=f0(x2)+0.01.

[0392] According to the initial tone mapping function f0(x) and the first coordinate value x4 of the ninth interpolation point, it can be determined that the function of the fifth spline function at the ninth interpolation point is f5(x4)=f0(x4), and the derivative of the fifth spline function at the ninth interpolation point is f5'(x4)=f0'(x4).

[0393] According to the properties of the spline function, it can be determined that the first-order derivative of the fourth spline function at the eighth interpolation point is the same as the first-order derivative of the fifth spline function at the eighth interpolation point, and the second-order derivative of the fourth spline function at the eighth interpolation point is the same as the second-order derivative of the fifth spline function at the eighth interpolation point, that is, f5'(x2)=f4'(x2), f5"(x2)=f4"(x2).

[0394] Optionally, the third spline function is a cubic spline function, that is, the fourth spline function and the fifth spline function are cubic spline functions.

[0395] Optionally, the multiple interpolation points further include an eleventh interpolation point, the fourth spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0396] Optionally, the plurality of interpolation points further include an eleventh interpolation point, the fifth spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be Figure 4 3' point shown.

[0397] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fourth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0398] Optionally, the plurality of interpolation points further include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fifth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be Figure 4 3' point shown.

[0399] Scene 2

[0400] When the image is bright, the portion of the initial tone mapping curve close to the second endpoint can be corrected. The seventh interpolation point can be the fourth interpolation point mentioned above, the eighth interpolation point can be the fifth interpolation point mentioned above, and the ninth interpolation point can be the sixth interpolation point mentioned above. The positions of the specific interpolation points and the spline function are as follows: Figure 6 shown.

[0401] The above has described how to obtain the first coordinate values ​​of the fourth interpolation point, the fifth interpolation point, and the sixth interpolation point. That is, the first coordinate values ​​of the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point are all known.

[0402] First, a first offset is obtained. For example, the first offset is preset to -0.01.

[0403] Then, according to the first offset, the eighth interpolation point (such as Figure 4 For example, f3(x6)=f0(x6)-0.01, where f0(x) represents the initial tone mapping function.

[0404] Since the first coordinate value of the seventh interpolation point is the same as the first coordinate value of the first endpoint of the initial tone mapping function, it can be determined that the third spline function is a function passing through (1,1), that is, the coordinate of the seventh interpolation point is (1,1).

[0405] Then, determine the seventh interpolation point, the eighth interpolation point, the ninth interpolation point (i.e., the Figure 4 5' point, 6'

[0406] The third spline function f3(x) can be divided into a fourth spline function f4(x) passing through the seventh interpolation point and the eighth interpolation point and a fifth spline function f5(x) passing through the eighth interpolation point and the ninth interpolation point.

[0407] According to the coordinates of the seventh interpolation point and the coordinates of the eighth interpolation point, it can be determined that f4(x5)=1, f4(x6)=f0(x6)-0.01, and f5(x6)=f0(x6)-0.01.

[0408] According to the initial tone mapping function f0(x) and the first coordinate value x8 of the ninth interpolation point, it can be determined that the function of the fifth spline function at the ninth interpolation point is f5(x8)=f0(x8), and the derivative of the fifth spline function at the ninth interpolation point is f5'(x8)=f0'(x8).

[0409] According to the properties of the spline function, it can be determined that the first-order derivative of the fourth spline function at the eighth interpolation point is the same as the first-order derivative of the fifth spline function at the eighth interpolation point, and the second-order derivative of the fourth spline function at the eighth interpolation point is the same as the second-order derivative of the fifth spline function at the eighth interpolation point, that is, f5'(x6)=f4'(x6), f5"(x6)=f4"(x6).

[0410] Optionally, the third spline function is a cubic spline function, that is, the fourth spline function and the fifth spline function are cubic spline functions.

[0411] Optionally, the multiple interpolation points further include an eleventh interpolation point, the fourth spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0412] Optionally, the plurality of interpolation points further include an eleventh interpolation point, the fifth spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be Figure 4 7' point shown.

[0413] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fourth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0414] Optionally, the plurality of interpolation points further include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fifth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be Figure 4 7' point shown.

[0415] Scene 3

[0416] Method 1

[0417] The eighth interpolation point may be the twelfth interpolation point mentioned above, the seventh interpolation point may be the fourteenth interpolation point whose first coordinate value is less than the first coordinate value of the twelfth interpolation point and less than the first coordinate value of the thirteenth interpolation point mentioned above, and the ninth interpolation point may be the fourteenth interpolation point whose first coordinate value is greater than the first coordinate value of the twelfth interpolation point and greater than the first coordinate value of the thirteenth interpolation point mentioned above. In this scenario, the spline function includes a fourth spline function passing through the twelfth interpolation point and the thirteenth interpolation point and a fifth spline function passing through the thirteenth interpolation point and the fourteenth interpolation point. The positions of the specific interpolation points and the spline functions are as follows: Figure 7 shown.

[0418] The above has described how to obtain the first coordinate values ​​of the twelfth interpolation point, the thirteenth interpolation point, and the fourteenth interpolation point. That is, the first coordinate values ​​of the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point are all known.

[0419] First, a first offset is obtained. For example, the first offset is preset to be 0.01.

[0420] Then, according to the first offset, the eighth interpolation point (such as Figure 4 The second coordinate value of the 10' point in the image. For example, f3(x 10 )=f0(x 10 )+0.01, f0(x) represents the initial tone mapping function.

[0421] Since the third spline function is smoothly connected to the initial tone mapping function at the seventh interpolation point, the coordinates of the seventh interpolation point can be determined to be (x9, f0(x9)), and f3(x9)=f0(x9), f3'(x9)=f0'(x9).

[0422] Since the third spline function is smoothly connected to the initial tone mapping function at the ninth interpolation point, the coordinates of the ninth interpolation point can be determined as (x 12 ,f0(x 12 )), and f3(x 12 )=f0(x 12 ), f3'(x12 )=f0'(x 12 ).

[0423] Then, determine the seventh interpolation point, the eighth interpolation point, the ninth interpolation point (i.e., the Figure 4 The third spline function f3(x) can be divided into a fourth spline function f4(x) passing through the seventh interpolation point and the eighth interpolation point and a fifth spline function f5(x) passing through the eighth interpolation point and the ninth interpolation point.

[0424] According to the initial tone mapping function f0(x), the coordinates of the seventh interpolation point, and the coordinates of the eighth interpolation point, it can be determined that f4(x9)=f0(x9), f4'(x9)=f0'(x9), f4(x 10 )=f0(x 10 )+0.01.

[0425] According to the initial tone mapping function f0(x), the coordinates of the eighth interpolation point and the coordinates of the ninth interpolation point, f5(x 10 )=f0(x 10 )+0.01,f5(x 12 )=f0(x 12 ), f5'(x 12 )=f0'(x 12 ).

[0426] According to the properties of the spline function, it can be determined that the first-order derivative of the fourth spline function at the eighth interpolation point is the same as the first-order derivative of the fifth spline function at the eighth interpolation point, and the second-order derivative of the fourth spline function at the eighth interpolation point is the same as the second-order derivative of the fifth spline function at the eighth interpolation point, that is, f5'(x 10 )=f4'(x 10 ), f5" (x 10 )=f4”(x 10 ).

[0427] Optionally, the third spline function is a cubic spline function, that is, the fourth spline function and the fifth spline function are cubic spline functions.

[0428] Optionally, the multiple interpolation points further include an eleventh interpolation point, the fourth spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0429] Optionally, the plurality of interpolation points further include an eleventh interpolation point, the fifth spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be Figure 4 11' o'clock shown.

[0430] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fourth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0431] Optionally, the plurality of interpolation points further include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fifth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be Figure 4 11' o'clock shown.

[0432] Method 2

[0433] The eighth interpolation point may be the twelfth interpolation point mentioned above, the tenth interpolation point may be the thirteenth interpolation point mentioned above, the seventh interpolation point may be the fourteenth interpolation point whose first coordinate value is less than the first coordinate value of the twelfth interpolation point and less than the first coordinate value of the thirteenth interpolation point mentioned above, and the ninth interpolation point may be the fourteenth interpolation point whose first coordinate value is greater than the first coordinate value of the twelfth interpolation point and greater than the first coordinate value of the thirteenth interpolation point mentioned above. In this scenario, the spline function includes a fourth spline function passing through the twelfth interpolation point and the thirteenth interpolation point and a fifth spline function passing through the thirteenth interpolation point and the fourteenth interpolation point. The positions of the specific interpolation points and the spline functions are as follows: Figure 7 shown.

[0434] The above has explained how to obtain the first coordinate values ​​of the twelfth interpolation point, the thirteenth interpolation point, and the fourteenth interpolation point. That is to say, the first coordinate value of the seventh interpolation point, the first coordinate value of the eighth interpolation point, the first coordinate value of the ninth interpolation point, and the first coordinate value of the tenth interpolation point are all known.

[0435] First, a first offset is obtained. For example, the first offset is preset to -0.01.

[0436] Then, according to the first offset, the eighth interpolation point (such as Figure 4 The second coordinate value of the 10' point in the image. For example, f3(x 10 )=f0(x 10 )-0.01, f0(x) represents the initial tone mapping function.

[0437] Then, a second offset is obtained. For example, the second offset is preset to +0.01.

[0438] Then, according to the first offset, the eighth interpolation point (such as Figure 4 11' in the second coordinate value of the point). For example, f3(x 11 )=f0(x 11 )+0.01, f0(x) represents the initial tone mapping function.

[0439] Since the third spline function is smoothly connected to the initial tone mapping function at the seventh interpolation point, the coordinates of the seventh interpolation point can be determined to be (x9, f0(x9)), and f3(x9)=f0(x9), f3'(x9)=f0'(x9).

[0440] Since the third spline function is smoothly connected to the initial tone mapping function at the ninth interpolation point, the coordinates of the ninth interpolation point can be determined as (x 12 ,f0(x 12 )), and f3(x 12 )=f0(x 12 ), f3'(x 12 )=f0'(x 12 ).

[0441] Then, determine the seventh interpolation point, the eighth interpolation point, the ninth interpolation point (i.e., the Figure 4 The third spline function f3(x) can be divided into a seventh spline function f7(x) passing through the seventh interpolation point and the eighth interpolation point, an eighth spline function f8(x) passing through the eighth interpolation point and the tenth interpolation point, and a ninth spline function f9(x) passing through the tenth interpolation point and the ninth interpolation point.

[0442] According to the initial tone mapping function f0(x), the coordinates of the seventh interpolation point, and the coordinates of the eighth interpolation point, it can be determined that f7(x9)=f0(x9), f7'(x9)=f0'(x9), f7(x 10 )=f0(x 10 )-0.01.

[0443] According to the coordinates of the eighth interpolation point and the coordinates of the tenth interpolation point, f8(x 10 )=f0(x 10 )-0.01, f8(x 11 )=f0(x 11 )+0.01.

[0444] According to the initial tone mapping function f0(x), the coordinates of the tenth interpolation point, and the coordinates of the ninth interpolation point, f9(x 12 )=f0(x 12 ), f9'(x 12 )=f0'(x12 ), f9(x 11 )=f0(x 11 )+0.01.

[0445] According to the properties of the spline function, it can be determined that the first-order derivative of the seventh spline function at the eighth interpolation point is the same as the first-order derivative of the eighth spline function at the eighth interpolation point, and the second-order derivative of the seventh spline function at the eighth interpolation point is the same as the second-order derivative of the eighth spline function at the eighth interpolation point, that is, f7'(x 10 )=f8'(x 10 ), f7" (x 10 )=f8”(x 10 ).

[0446] According to the properties of the spline function, it can be determined that the first-order derivative of the eighth spline function at the tenth interpolation point is the same as the first-order derivative of the ninth spline function at the tenth interpolation point, and the second-order derivative of the eighth spline function at the tenth interpolation point is the same as the second-order derivative of the ninth spline function at the tenth interpolation point, that is, f8'(x 11 )=f9'(x 11 ), f8" (x 11 )=f9”(x 11 ).

[0447] According to the above conditions, the third spline function f3(x) can be determined.

[0448] Optionally, the third spline function is a cubic spline function, that is, the fourth spline function and the fifth spline function are cubic spline functions.

[0449] Optionally, the multiple interpolation points further include an eleventh interpolation point, the fourth spline function passes through the eleventh interpolation point, and the second-order derivative of the first spline function at the eleventh interpolation point is 0.

[0450] Optionally, the multiple interpolation points further include an eleventh interpolation point, the fifth spline function passes through the eleventh interpolation point, and the second-order derivative of the second spline function at the eleventh interpolation point is 0.

[0451] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fourth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0452] Optionally, the multiple interpolation points also include a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point, and the derivative value of the fifth spline function at the fifteenth interpolation point is different from the derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0453] 204 , modify the initial tone mapping curve according to the spline function to obtain a modified tone mapping curve of the target image.

[0454] By correcting the initial tone mapping curve, the spline function can replace a part of the initial tone mapping curve. The final corrected tone mapping curve contains a part of the data content originally existing on the initial tone mapping curve and the data content of the spline function. Since the initial spline function is usually smooth, the spline function itself has a smooth property, and the spline function can be smoothly connected to the initial spline function at the interpolation point, the corrected tone mapping curve also has a smooth property.

[0455] 205 , performing tone mapping on the target image according to the modified tone mapping curve.

[0456] The electronic device can refer to the prior art and perform tone mapping on the target image according to the modified tone mapping curve, which is not necessary to elaborate here.

[0457] The electronic device implementing steps 201 to 205 may be Figure 1 The shooting end device in Figure 1 The image processing device in the embodiment may also be a display terminal device including an image processor.

[0458] The present application provides a method for correcting an already obtained initial tone mapping curve, so that the way of obtaining the tone mapping curve is more flexible. For example, the form of the tone mapping curve can be flexibly determined according to the device properties of the display device and the shooting device, so that the image display is more vivid and natural. For another example, in a video, the tone mapping curve of a frame can be determined, and for the frames adjacent to the frame, the parameters of the corrected tone mapping curve are determined, so that the video display effect can be improved without adding too much data.

[0459] Figure 5 is a schematic structural diagram of an image processing device provided in an embodiment of the present application. Figure 5 As shown, the apparatus 500 includes an acquisition module 510 , a processing module 520 and a tone mapping module 530 .

[0460] The acquisition module 510 is used to acquire a target image and an initial tone mapping curve of the target image.

[0461] The processing module 520 is used to determine a plurality of interpolation points according to the brightness information and / or dynamic metadata of the target image.

[0462] The processing module 520 is further configured to determine a spline function passing through the plurality of interpolation points, wherein the spline function is smoothly connected to the initial tone mapping curve at at least one interpolation point among the plurality of interpolation points.

[0463] The processing module 520 is further configured to correct the initial tone mapping curve according to the spline function to obtain a corrected tone mapping curve of the target image.

[0464] The tone mapping module 530 is configured to perform tone mapping on the target image according to the modified tone mapping curve.

[0465] Figure 6 FIG. 1 is a schematic structural diagram of an image processing device provided by another embodiment of the present application. Figure 6 As shown, the apparatus 600 may include a transmission interface 610 , a processor 620 , and a memory 630 .

[0466] Figure 6 Only one memory and processor are shown. In an actual image processing device product, there may be one or more processors and one or more memories. The memory may also be referred to as a storage medium or a storage device, etc. The memory may be set independently of the processor or may be integrated with the processor, and the embodiments of the present application do not limit this.

[0467] The transmission interface 610 , the processor 620 , and the memory 630 communicate with each other through internal connection paths to transfer control and / or data signals.

[0468] Specifically, the transmission interface 610 is used to receive or send image data; the processor 620 is used to obtain an image to be processed; it is used to process the image to be processed through a first branch of a pre-trained neural network model to obtain a first type of parameter, and the first type of parameter is used to perform global color processing on the image; it is also used to process the image to be processed through a second branch of the neural network model to obtain a second type of parameter, and the second type of parameter is used to perform local color processing on the image; it is also used to perform color processing on the image to be processed according to the first type of parameter and the second type of parameter to obtain a color-processed image.

[0469] The memory 630 described in each embodiment of the present application is used to store computer instructions and parameters required for the operation of the processor.

[0470] The specific working process and beneficial effects of the device 600 can be found in Figure 2 The description in the illustrated embodiment will not be repeated here.

[0471] The embodiment of the present application also provides an electronic device, which may be a terminal device. The device may be used to execute the functions / steps in the above method embodiment.

[0472] Figure 7 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 includes a processor 710 and a transceiver 720. Optionally, the electronic device 700 may also include a memory 730. The processor 710, the transceiver 720 and the memory 730 may communicate with each other through an internal connection path to transmit control and / or data signals, the memory 730 is used to store a computer program, and the processor 710 is used to call and run the computer program from the memory 730.

[0473] Optionally, the electronic device 700 may further include an antenna 740 for transmitting the wireless signal output by the transceiver 720 .

[0474] The processor 710 and the memory 730 may be combined into a processing device, or more commonly, they are independent components. The processor 710 is used to execute the program code stored in the memory 730 to implement the above functions. In specific implementation, the memory 730 may also be integrated into the processor 710, or may be independent of the processor 710. Figure 6 Corresponding to the processing module 620 in the device 600.

[0475] In addition, in order to make the functions of the electronic device 700 more complete, the electronic device 700 may further include one or more of an input unit 760, a display unit 770, an audio circuit 780, a camera 770, and a sensor 701, and the audio circuit may further include a speaker 782, a microphone 784, etc. The display unit 770 may include a display screen.

[0476] Optionally, the electronic device 700 may further include a power supply 750 for providing power to various devices or circuits in the terminal device.

[0477] It should be understood that Figure 7 The electronic device 700 shown can implement Figures 2 to 6The various processes of the method embodiment shown. The operations and / or functions of the various modules in the electronic device 700 are respectively to implement the corresponding processes in the above method embodiments. For details, please refer to the description in the above method embodiments. In order to avoid repetition, the detailed description is appropriately omitted here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0478] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0479] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0480] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0481] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0482] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0483] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for image processing, characterized in that: include: Obtaining a target image and an initial tone mapping curve of the target image; Determining a plurality of interpolation points according to brightness information and / or dynamic metadata of the target image; Determine a spline function passing through the plurality of interpolation points, wherein the spline function is smoothly connected to the initial tone mapping curve at at least one interpolation point among the plurality of interpolation points; Obtaining a modified tone mapping curve of the target image according to the spline function and the initial tone mapping curve; Performing tone mapping on the target image according to the modified tone mapping curve; Among them, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at the two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of dark area pixels included in the target image is greater than or equal to the second preset threshold, the brightness value of the dark area pixel is less than the first preset threshold, and the multiple interpolation points include a first interpolation point, and the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

2. The method according to claim 1, characterized in that The multiple interpolation points also include a second interpolation point, and the determining the multiple interpolation points according to the brightness information of the target image includes: The first coordinate value of the second interpolation point is determined according to the number of the dark area pixels.

3. The method according to claim 1 or 2, characterized in that: The multiple interpolation points further include a third interpolation point, the target image further includes a dark area transition pixel point, the brightness value of the dark area transition pixel point is greater than or equal to the first preset threshold, and the multiple interpolation points are determined according to the brightness information of the target image, including: The first coordinate value of the third interpolation point is determined according to the number of the dark area transition pixel points.

4. The method according to claim 1 or 2, characterized in that: The number of bright area pixels included in the target image is greater than or equal to a fourth preset threshold, the brightness value of the bright area pixels is greater than a third preset threshold, the multiple interpolation points include a fourth interpolation point, and determining the multiple interpolation points according to the brightness information of the target image includes: It is determined that the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

5. The method according to claim 4, characterized in that The multiple interpolation points further include a fifth interpolation point, and the determining the multiple interpolation points according to the brightness information of the target image includes: The first coordinate value of the fifth interpolation point is determined according to the number of the bright area pixels.

6. The method according to claim 4, characterized in that The multiple interpolation points further include a sixth interpolation point, the target image further includes a bright area transition pixel point, the brightness value of the bright area transition pixel point is less than or equal to the third preset threshold, and the multiple interpolation points are determined according to the brightness information of the target image, including: The first coordinate value of the sixth interpolation point is determined according to the number of the bright area transition pixel points.

7. The method according to claim 1, characterized in that The dynamic metadata includes coordinate information of each interpolation point in the plurality of interpolation points.

8. The method according to claim 1, characterized in that The dynamic metadata includes regional brightness indication information, In a case where the area brightness indication information indicates a dark area, the plurality of interpolation points include the first interpolation point, and a first coordinate value of the first interpolation point is the same as a first coordinate value of the first endpoint; and / or, In a case where the area brightness indication information indicates a bright area, the plurality of interpolation points include a second interpolation point, and a first coordinate value of the second interpolation point is the same as a first coordinate value of the second endpoint.

9. The method according to claim 1, characterized in that: The dynamic metadata includes P coordinate information corresponding to P key pixel points one by one, where P is a positive integer. The determining of multiple interpolation points according to the brightness information of the target image and the dynamic metadata includes: According to the P coordinate information and the brightness information of the target image, determine that the P key pixels are dark area pixels or bright area pixels, the brightness value of the dark area pixels is less than a first preset threshold, and the brightness value of the bright area pixels is greater than a third preset threshold; In the case where the P key pixels are dark area pixels, determining that the multiple interpolation points include the first interpolation point, and the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint; or In the case that the P key pixels are bright area pixels, it is determined that the multiple interpolation points include a second interpolation point, and a first coordinate value of the second interpolation point is the same as a first coordinate value of the second endpoint.

10. The method according to claim 1 or 2, characterized in that: The plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, and the determining of the spline function passing through the plurality of interpolation points includes: Obtaining a first spline function passing through the seventh interpolation point and the eighth interpolation point; A second spline function passing through the eighth interpolation point and the ninth interpolation point is determined, wherein the second spline function is smoothly connected to the first spline function at the eighth interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point.

11. The method according to claim 10, characterized in that The first spline function is a linear spline function, and the second spline function is a cubic spline function.

12. The method according to claim 1 or 2, characterized in that: The multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and the first coordinate value of the eighth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; Before determining the plurality of interpolation points, the method further includes: Acquire a first offset, where the first offset is used to represent a correction amplitude of the initial tone mapping curve at a first correction position, where the first correction position is a first coordinate value of the eighth interpolation point; The determining of a plurality of interpolation points comprises: determining a second coordinate value of the eighth interpolation point according to a function value of the initial tone mapping curve at the first correction position and the first offset; The step of determining the spline function passing through the plurality of interpolation points comprises: The third spline function is determined according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

13. The method according to claim 12, characterized in that The third spline function includes a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point, and the second-order derivative value of the fourth spline function at the eighth interpolation point is the same as the second-order derivative value of the fifth spline function at the eighth interpolation point.

14. The method according to claim 12, characterized in that The plurality of interpolation points further include a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point, Before determining the plurality of interpolation points, the method further includes: Acquire a second offset, where the second offset is used to represent a correction amplitude of the initial tone mapping curve at a second correction position, where the second correction position is a first coordinate value of the tenth interpolation point; The determining of the plurality of interpolation points further includes: determining a second coordinate value of the tenth interpolation point according to the function value of the initial tone mapping curve at the second correction position and the second offset; The determining the third spline function according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point comprises: The third spline function is determined according to the first coordinate value and the second coordinate value of the eighth interpolation point and the first coordinate value and the second coordinate value of the tenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

15. The method according to claim 1 or 2, characterized in that: The derivative of the spline function is always greater than 0.

16. The method according to claim 1 or 2, characterized in that: The initial tone mapping curve includes one of the following formulas: L′=a×L+b; Among them, L represents the linear signal value, L' represents the nonlinear signal value, p, a, m, b, c1, c2, c3, n, N, ρ are all dynamic range mapping coefficients, 0≤L≤1.

17. An image processing device, characterized in that: include: An acquisition module, used to acquire a target image and an initial tone mapping curve of the target image; A processing module, configured to determine a plurality of interpolation points according to brightness information and / or dynamic metadata of the target image; The processing module is further used to determine a spline function passing through the plurality of interpolation points, wherein the spline function is smoothly connected to the initial tone mapping curve at at least one interpolation point among the plurality of interpolation points; The processing module is further used to obtain a modified tone mapping curve of the target image according to the spline function and the initial tone mapping curve; A tone mapping module, configured to perform tone mapping on the target image according to the modified tone mapping curve; Among them, the initial tone mapping curve includes a first endpoint and a second endpoint, the first endpoint and the second endpoint are respectively located at the two ends of the initial tone mapping curve, the brightness value corresponding to the first endpoint is lower than the brightness value corresponding to the second endpoint, the number of dark area pixels included in the target image is greater than or equal to the second preset threshold, the brightness value of the dark area pixel is less than the first preset threshold, the multiple interpolation points include a first interpolation point, and the processing module is specifically used to determine that the first coordinate value of the first interpolation point is the same as the first coordinate value of the first endpoint.

18. The device according to claim 17, characterized in that The plurality of interpolation points further includes a second interpolation point, The processing module is specifically configured to determine the first coordinate value of the second interpolation point according to the number of pixels in the dark area.

19. The device according to claim 17 or 18, characterized in that The multiple interpolation points further include a third interpolation point, the target image further includes a dark area transition pixel point, and the brightness value of the dark area transition pixel point is greater than or equal to the first preset threshold value, The processing module is specifically configured to determine the first coordinate value of the third interpolation point according to the number of transition pixel points in the dark area.

20. The device according to claim 17 or 18, characterized in that The number of bright area pixels included in the target image is greater than or equal to a fourth preset threshold, the brightness value of the bright area pixels is greater than a third preset threshold, and the multiple interpolation points include a fourth interpolation point. The processing module is specifically configured to determine that the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second endpoint.

21. The device according to claim 20, characterized in that The plurality of interpolation points further includes a fifth interpolation point, The processing module is specifically configured to determine the first coordinate value of the fifth interpolation point according to the number of pixels in the bright area.

22. The device according to claim 20, characterized in that The plurality of interpolation points further include a sixth interpolation point, the target image further includes a bright area transition pixel point, and the brightness value of the bright area transition pixel point is less than or equal to the third preset threshold value, The processing module is specifically configured to determine the first coordinate value of the sixth interpolation point according to the number of transition pixel points in the bright area.

23. The device according to claim 17, characterized in that The dynamic metadata includes coordinate information of each interpolation point in the plurality of interpolation points.

24. The device according to claim 17, characterized in that The dynamic metadata includes regional brightness indication information, In a case where the area brightness indication information indicates a dark area, the plurality of interpolation points include the first interpolation point, and a first coordinate value of the first interpolation point is the same as a first coordinate value of the first endpoint; and / or, In a case where the area brightness indication information indicates a bright area, the plurality of interpolation points include a second interpolation point, and a first coordinate value of the second interpolation point is the same as a first coordinate value of the second endpoint.

25. The device according to claim 17, characterized in that The dynamic metadata includes P coordinate information corresponding to P key pixel points one by one, where P is a positive integer. The processing module is specifically used to determine, according to the P coordinate information and the brightness information of the target image, that the P key pixels are dark area pixels or bright area pixels, the brightness value of the dark area pixels is less than a first preset threshold, and the brightness value of the bright area pixels is greater than a third preset threshold; In the case where the P key pixels are dark area pixels, the processing module is specifically configured to determine that the plurality of interpolation points include the first interpolation point, and a first coordinate value of the first interpolation point is the same as a first coordinate value of the first endpoint; or In the case where the P key pixels are bright area pixels, the processing module is specifically configured to determine that the multiple interpolation points include a second interpolation point, and a first coordinate value of the second interpolation point is the same as a first coordinate value of the second endpoint.

26. The device according to claim 17 or 18, characterized in that The plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, and the initial tone mapping curve passes through the ninth interpolation point. The processing module is specifically used to obtain a first spline function passing through the seventh interpolation point and the eighth interpolation point; The processing module is specifically used to determine a second spline function passing through the eighth interpolation point and the ninth interpolation point, wherein the second spline function is smoothly connected to the first spline function at the eighth interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point.

27. The device according to claim 26, characterized in that The first spline function is a linear spline function, and the second spline function is a cubic spline function.

28. The device according to claim 17 or 18, characterized in that The multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and the first coordinate value of the eighth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; Before the processing module determines the plurality of interpolation points, the acquisition module is further used to acquire a first offset, where the first offset is used to represent a correction amplitude of the initial tone mapping curve at a first correction position, where the first correction position is a first coordinate value of the eighth interpolation point; The processing module is specifically configured to determine a second coordinate value of the eighth interpolation point according to a function value of the initial tone mapping curve at the first correction position and the first offset; The processing module is specifically used to determine the third spline function according to the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

29. The device according to claim 28, characterized in that The third spline function includes a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point, and the second-order derivative value of the fourth spline function at the eighth interpolation point is the same as the second-order derivative value of the fifth spline function at the eighth interpolation point.

30. The device according to claim 29, characterized in that The plurality of interpolation points further include a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point, Before the processing module determines the plurality of interpolation points, the acquisition module is further used to acquire a second offset, where the second offset is used to represent a correction amplitude of the initial tone mapping curve at a second correction position, where the second correction position is a first coordinate value of the tenth interpolation point; The processing module is specifically configured to determine a second coordinate value of the tenth interpolation point according to a function value of the initial tone mapping curve at the second correction position and the second offset; The processing module is specifically used to determine the third spline function according to the first coordinate value and the second coordinate value of the eighth interpolation point and the first coordinate value and the second coordinate value of the tenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

31. The device according to claim 17 or 18, characterized in that The derivative of the spline function is always greater than 0.

32. An image processing device, characterized in that: Including processor and transmission interface, The transmission interface is used to receive or send image data; The processor is configured to call program instructions stored in a memory, and execute the program instructions to perform the method according to any one of claims 1 to 16.

33. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed on a computer or a processor, the computer or the processor is caused to execute the method according to any one of claims 1 to 16.

Images