Image processing method and device

Abstract

To provide an image processing method and apparatus that enable a display device to vividly and realistically reproduce a scene in a real world.SOLUTION: An image processing method includes: obtaining a target picture and an initial tone mapping curve of the target picture; determining a plurality of interpolation points on the basis of luminance information and / or dynamic metadata of the target picture; and determining a spline function passing through the plurality of interpolation points. The spline function is smoothly connected to the initial tone mapping curve in at least one of the plurality of interpolation points. The method further includes: modifying the initial tone mapping curve on the basis of the spline function to obtain a modified tone mapping curve of the target picture; and performing tone mapping for the target picture on the basis of the modified tone mapping curve.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] This application is a divisional application of Patent Application No. 2021-575058, filed on May 26, 2020, which claims priority to Chinese Patent Application No. 201910626171.8, filed with the State Intellectual Property Office of China on July 11, 2019, entitled "IMAGE PROCESSING METHOD AND APPARATUS," the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of image processing, and more particularly to image processing methods and apparatus. [Background technology]

[0003] Dynamic range (DR) can be used to represent the ratio of the maximum grayscale value to the minimum grayscale value of a picture. Dynamic range can also be used to represent the ratio of the maximum luminance value to the minimum luminance value of a picture. In the real world, dynamic range is about 10 -3 From 10 6In other words, the luminance value of an object in the real world can be very large or very small. However, the display capability of a display device is generally limited. For example, the luminance of object A in the real world is relatively high. For example, the luminance of object A is 2000 nit. However, the maximum luminance that can be displayed by the display device is lower than the luminance of object A. For example, the maximum luminance that can be displayed by the display device is only 500 nit. If no processing is performed, the display device cannot reproduce object A in the real world. In another case, the luminance of object B in the real world is relatively low. For example, the luminance value of object B is 10 nit. However, the minimum luminance that can be displayed by the display device is higher than the luminance of object B. For example, the minimum luminance that can be displayed by the display device is only 100 nit. If no processing is performed, the display device cannot reproduce object B in the real world. This application provides a method that enables a display device to reproduce a scene in the real world vividly and realistically. Summary of the Invention

[0004] The present application provides an image processing method and apparatus that enables a display device to reproduce real-world scenes vividly and realistically.

[0005] According to a first aspect, there is provided an image processing method comprising the steps of: obtaining a target picture and an initial tone mapping curve for the target picture; determining a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture; determining a spline function that passes through the plurality of interpolation points, the spline function smoothly connecting to the initial tone mapping curve at at least one of the plurality of interpolation points; modifying the initial tone mapping curve based on the spline function to obtain a modified tone mapping curve for the target picture; and performing tone mapping on the target picture based on the modified tone mapping curve.

[0006] Determining a plurality of interpolation points based on the luminance information and / or dynamic metadata of the target picture may be determining a modification interval for modifying the initial tone mapping curve based on the luminance information and / or dynamic metadata of the target picture, determining a spline function that passes through the plurality of interpolation points may be determining a spline function for modifying the initial tone mapping curve in the modification interval, and modifying the initial tone mapping curve based on the spline function to obtain a modified tone mapping curve for the target picture may be modifying the initial tone mapping curve to the spline function in the modification interval to obtain a modified tone mapping curve for the target picture.

[0007] The present application provides a method for modifying the acquired initial tone mapping curve, so that the manner of acquiring the tone mapping curve becomes more flexible. By analyzing the luminance of the target picture and appropriately modifying the tone mapping curve, the image display effect can be adjusted for different luminance regions, and the details of the luminance change of the image can be enhanced, thereby providing the user with a better visual experience. By fine-tuning the tone mapping curve in various forms, multiple styles of display effects can be obtained, such as gradual, clear, bright, dark, etc. For example, the form of the tone mapping curve can be flexibly determined based on the device attributes of the display device and the shooting device, so that the image display becomes clearer and more natural. In addition, in the process of displaying an image, the tone mapping curve of a frame can be determined, and for the frame adjacent to the frame, the parameters for modifying the tone mapping curve are determined, so that the image display effect can be improved without additional excess data.

[0008] With reference to the first aspect, in some implementations of the first aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the number of dark area pixels included in the target picture is greater than or equal to a second preset threshold, the luminance values ​​of the dark area pixels are smaller than the first preset threshold, the multiple interpolation points include a first interpolation point, and determining the multiple interpolation points based on the luminance information of the target picture includes determining that a first coordinate value of the first interpolation point is identical to the first coordinate value of the first end point.

[0009] In this embodiment of the present application, when the image is dark, the electronic device may determine that the interpolation point is at or near the first end point, thereby improving the display effect of the dark area. Therefore, the modification position for modifying the initial tone mapping curve may be determined based on the luminance of the image, so that pixels whose luminance is lower than the display capability of the display device may be displayed on the display device.

[0010] With reference to the first aspect, in some implementations of the first aspect, the multiple interpolation points further include a second interpolation point, and determining the multiple interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the second interpolation point based on the number of dark area pixels.

[0011] In this embodiment of the present application, when the image is dark, the electronic device can determine the modification range of the initial tone mapping curve based on the area of ​​the dark area in the target image, so that pixels whose brightness is lower than the display capability of the display device can have a relatively good display effect on the display device.

[0012] With reference to the first aspect, in some implementations of the first aspect, the plurality of interpolation points further includes a third interpolation point, the target picture further includes a dark area transition pixel, the luminance value of the dark area transition pixel is greater than or equal to a first preset threshold, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the third interpolation point based on the number of dark area transition pixels.

[0013] The third interpolation point can be a transition point, so that the spline function and the initial tone mapping curve smoothly transition at the third interpolation point.In this embodiment of the present application, when the image is dark, one end of the initial tone mapping curve can be modified.In order to ensure that the modified tone mapping curve is smooth, that is, to ensure the image display effect, an interpolation point having the effect of smooth connection can be determined, so that the spline function is smoothly connected to the initial tone mapping curve.

[0014] With reference to the first aspect, in some implementations of the first aspect, the initial tone mapping curve passes through a third interpolation point, and determining a spline function passing through the multiple interpolation points includes obtaining a first spline function passing through the first interpolation point and the second interpolation point, and determining a second spline function passing through the second interpolation point and the third interpolation point, wherein the second spline function is smoothly connected to the first spline function at the second interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the third interpolation point.

[0015] With reference to the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function and the second spline function is a cubic spline function.

[0016] In some implementations of the first aspect, the spline function includes a third spline function passing through the first interpolation point, the second interpolation point, and the third interpolation point. For example, the third spline function is a cubic spline function.

[0017] With reference to the first aspect, in some implementations of the first aspect, the method includes a step of obtaining a first offset corresponding to a second interpolation point, where the first offset is used to represent a modification range of the initial tone mapping curve at the first modified position, and the first modified position is a first coordinate value of the second interpolation point; and a step of determining a second coordinate value of the second interpolation point based on the function value of the initial tone mapping curve at the first modified position and the first offset, and the step of determining a spline function that passes through the multiple interpolation points includes a step of determining a third spline function based on the first coordinate value of the second interpolation point and the second coordinate value of the second interpolation point, and the third spline function is smoothly connected to the initial tone mapping curve at the third interpolation point.

[0018] With reference to the first aspect, in some implementations of the first aspect, the third spline function includes a fourth spline function passing through the first interpolation point and the second interpolation point, and a fifth spline function passing through the second interpolation point and the third interpolation point, and a second derivative value of the fourth spline function at the second interpolation point is identical to a second derivative value of the fifth spline function at the second interpolation point.

[0019] For example, the third spline function is a cubic spline function.

[0020] For example, the derivative of the spline function in the above scenario is always greater than zero.

[0021] Referring to the first aspect, in some implementations of the first aspect, the plurality of interpolation points further includes an eleventh interpolation point, and a second derivative of the spline function at the eleventh interpolation point is zero.

[0022] With reference to the first aspect, in some implementations of the first aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, the luminance value corresponding to the first end point is smaller than the luminance value corresponding to the second end point, the number of bright area pixels included in the target picture is greater than or equal to a fourth preset threshold, the luminance values ​​of the bright area pixels are greater than a third preset threshold, the multiple interpolation points include a fourth interpolation point, and the step of determining the multiple interpolation points based on the luminance information of the target picture includes a step of determining that a first coordinate value of the fourth interpolation point is identical to a first coordinate value of the second end point.

[0023] In this embodiment of the present application, when the image is bright, the electronic device may determine that the interpolation point is at or near the second end point, thereby improving the display effect of the bright area. Therefore, the modification position for modifying the initial tone mapping curve may be determined based on the luminance of the image, so that the pixel whose luminance is higher than the display capability of the display device may be displayed on the display device.

[0024] With reference to the first aspect, in some implementations of the first aspect, the multiple interpolation points further include a fifth interpolation point, and determining the multiple interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the fifth interpolation point based on the number of bright area pixels.

[0025] In this embodiment of the present application, when the image is bright, the electronic device can determine a modification range for modifying the initial tone mapping curve based on the area of ​​the bright area in the target image, so that pixels whose brightness is higher than the display capability of the display device can have a relatively good display effect on the display device.

[0026] With reference to the first aspect, in some implementations of the first aspect, the multiple interpolation points further include a sixth interpolation point, the target picture further includes a bright area transition pixel, the brightness value of the bright area transition pixel is less than or equal to a third preset threshold, and determining the multiple interpolation points based on the brightness information of the target picture includes determining a first coordinate value of the sixth interpolation point based on the number of bright area transition pixels.

[0027] The sixth interpolation point can be a transition point, so that the spline function and the initial tone mapping curve smoothly transition at the sixth interpolation point.In this embodiment of the present application, when the image is bright, one end of the initial tone mapping curve can be modified.In order to ensure that the modified tone mapping curve is smooth, that is, to ensure the image display effect, an interpolation point having the effect of smooth connection can be determined, so that the spline function is smoothly connected to the initial tone mapping curve.

[0028] With reference to the first aspect, in some implementations of the first aspect, the initial tone mapping curve passes through the sixth interpolation point, and determining a spline function passing through the multiple interpolation points includes obtaining a first spline function passing through the fourth interpolation point and the fifth interpolation point, and determining a second spline function passing through the fifth interpolation point and the sixth interpolation point, wherein the second spline function is smoothly connected to the first spline function at the fifth interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the sixth interpolation point.

[0029] With reference to the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function and the second spline function is a cubic spline function.

[0030] Referring to the first aspect, in some implementations of the first aspect, the spline function includes a third spline function passing through the fourth, fifth, and sixth interpolation points. For example, the third spline function is a cubic spline function.

[0031] With reference to the first aspect, in some implementations of the first aspect, the method includes a step of obtaining a first offset corresponding to a fifth interpolation point, where the first offset is used to represent a modification range of the initial tone mapping curve at the first modified position, and the first modified position is a first coordinate value of the fifth interpolation point; and a step of determining a second coordinate value of the fifth interpolation point based on the function value of the initial tone mapping curve at the first modified position and the first offset, and the step of determining a spline function passing through the multiple interpolation points includes a step of determining a third spline function based on the first coordinate value of the fifth interpolation point and the second coordinate value of the fifth interpolation point, and the third spline function is smoothly connected to the initial tone mapping curve at the sixth interpolation point.

[0032] With reference to the first embodiment, in some implementations of the first embodiment, the third spline function includes a fourth spline function passing through the fourth and fifth interpolation points, and a fifth spline function passing through the fifth and sixth interpolation points, and a second derivative value of the fourth spline function at the fifth interpolation point is identical to a second derivative value of the fifth spline function at the fifth interpolation point. With reference to the first embodiment, in some implementations of the first embodiment, the dynamic metadata includes at least one of the following items: coordinate information of each of the plurality of interpolation points, region luminance indication information, or coordinate information of the plurality of key pixels.

[0033] Referring to the first aspect, in some implementations of the first aspect, the dynamic metadata includes coordinate information for each of the multiple interpolation points.

[0034] In this embodiment of the present application, the interpolation point information may be preset in the dynamic metadata, so that the image processing device may not analyze the pixels of the image and may determine multiple interpolation points based only on the dynamic metadata, thereby reducing the computational complexity of the image processing.

[0035] With reference to the first aspect, in some implementations of the first aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the dynamic metadata includes regional luminance indication information, when the regional luminance indication information indicates a dark area, the multiple interpolation points include a first interpolation point, and a first coordinate value of the first interpolation point is identical to a first coordinate value of the first end point, and / or when the regional luminance indication information indicates a light area, the multiple interpolation points include a second interpolation point, and a first coordinate value of the second interpolation point is identical to a first coordinate value of the second end point.

[0036] In this embodiment of the present application, when the image is dark, the electronic device may determine that the interpolation point is at or near the first end point. Thus, the modification position for modifying the initial tone mapping curve may be determined based on the luminance of the image, so that pixels whose luminance is lower than the display capability of the display device may be displayed on the display device. When the image is bright, the electronic device may determine that the interpolation point is at or near the second end point. Thus, the modification position for modifying the initial tone mapping curve may be determined based on the luminance of the image, so that pixels whose luminance is higher than the display capability of the display device may be displayed on the display device. Because the dynamic metadata specifies "dark" or "light," the image processing device may not analyze the pixels of the image, but may determine multiple interpolation points based only on the dynamic metadata, thereby reducing the amount of computation in the image processing.

[0037] With reference to the first aspect, in some implementations of the first aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the dynamic metadata includes P pieces of coordinate information having a one-to-one correspondence with the P key pixels, where P is a positive integer, and the step of determining a plurality of interpolation points based on the luminance information of the target picture and the dynamic metadata includes: determining P The method includes determining that the P key pixels are dark area pixels or light area pixels, where a luminance value of a dark area pixel is less than a first preset threshold and a luminance value of a light area pixel is greater than a third preset threshold; and determining that when the P key pixels are dark area pixels, the plurality of interpolation points includes a first interpolation point, where a first coordinate value of the first interpolation point is equal to a first coordinate value of a first end point; or determining that when the P key pixels are light area pixels, the plurality of interpolation points includes a second interpolation point, where a first coordinate value of the second interpolation point is equal to a first coordinate value of a second end point.

[0038] In this embodiment of the present application, when the image is dark, the electronic device may determine that the interpolation point is at or near the first end point. Therefore, the modification position for modifying the initial tone mapping curve may be determined based on the luminance of the image, so that pixels whose luminance is lower than the display capability of the display device may be displayed on the display device. When the image is bright, the electronic device may determine that the interpolation point is at or near the second end point. Therefore, the modification position for modifying the initial tone mapping curve may be determined based on the luminance of the image, so that pixels whose luminance is higher than the display capability of the display device may be displayed on the display device. Since the dynamic metadata specifies the pixels, the image processing device does not need to determine the target pixels when performing scene analysis on the target picture, but directly determines the dark area pixels and the light area pixels of the picture based on the target pixels specified by the dynamic metadata. Therefore, the key protection region and the multiple corresponding interpolation points may be determined more flexibly, and the computational amount of the image processing is reduced.

[0039] With reference to the first aspect, in some implementations of the first aspect, the target picture includes a target pixel whose luminance value is between a first luminance value and a second luminance value, a ratio of the target pixel to all pixels of the target picture is greater than an eighth preset threshold, a difference between the first luminance value and the second luminance value is less than a seventh preset threshold, the multiple interpolation points include a twelfth interpolation point, and determining the multiple interpolation points based on the luminance information of the target picture includes determining the twelfth interpolation point based on an average luminance of the target pixel, a minimum luminance value of the target picture, and a maximum luminance value of the target picture.

[0040] The first coordinate value of the twelfth interpolation point may be greater than the first coordinate value of the first interpolation point mentioned above.

[0041] The first coordinate value of the twelfth interpolation point may be greater than the first coordinate value of the second interpolation point mentioned above.

[0042] The first coordinate value of the twelfth interpolation point may be greater than the first coordinate value of the third interpolation point mentioned above.

[0043] The first coordinate value of the twelfth interpolation point may be smaller than the first coordinate value of the fourth interpolation point mentioned above.

[0044] The first coordinate value of the twelfth interpolation point may be smaller than the first coordinate value of the fifth interpolation point mentioned above.

[0045] The first coordinate value of the twelfth interpolation point may be smaller than the first coordinate value of the sixth interpolation point mentioned above.

[0046] In this embodiment of the present application, the modification range can be determined on the initial tone mapping function based on the luminance interval where the pixel to be modified is located. In other words, the modification can be performed on any segment of the initial tone mapping function, and the modification flexibility is relatively high. However, when the luminance interval is relatively narrow, the luminance contrast of the image is not obvious. The initial tone mapping curve is modified, so that the image whose original luminance contrast is not obvious can be displayed more clearly on the electronic device.

[0047] With reference to the first aspect, in some implementations of the first aspect, the multiple interpolation points further include a 13th interpolation point, and determining the multiple interpolation points based on luminance information of the target picture includes determining the 13th interpolation point based on the number of target pixels.

[0048] In this embodiment of the present application, the modification range may be determined based on the number of pixels to be modified, so that an image with poor luminance contrast appears clearer on the electronic device.

[0049] With reference to the first aspect, in some implementations of the first aspect, the plurality of interpolation points further includes a 14th interpolation point, the target picture further includes a target transition pixel, and the luminance value of the target transition pixel is less than or equal to a first luminance value, or the luminance value of the target transition pixel is greater than or equal to a second luminance value and the first luminance value is less than the second luminance value, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the 14th interpolation point based on the number of target transition pixels.

[0050] In the example, the first coordinate value of the fourteenth interpolation point is greater than the first coordinate value of the twelfth interpolation point, which is also greater than the first coordinate value of the thirteenth interpolation point.

[0051] In the example, the first coordinate value of the fourteenth interpolation point is less than the first coordinate value of the twelfth interpolation point, and less than the first coordinate value of the thirteenth interpolation point.

[0052] The 14th interpolation point may be a transition point, so that the spline function and the initial tone mapping curve smoothly transition at the 14th interpolation point. In this embodiment of the present application, in order to ensure that the modified tone mapping curve is smooth, i.e., to ensure the image display effect, an interpolation point having the effect of smooth connection may be determined, so that the spline function is smoothly connected to the initial tone mapping curve.

[0053] With reference to the first aspect, in some implementations of the first aspect, the initial tone mapping curve passes through the 14th interpolation point, and determining a spline function passing through the multiple interpolation points includes obtaining a first spline function passing through the 12th interpolation point and the 13th interpolation point, and determining a second spline function passing through the 13th interpolation point and the 14th interpolation point, wherein the second spline function is smoothly connected to the first spline function at the 13th interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the 14th interpolation point.

[0054] With reference to the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function and the second spline function is a cubic spline function.

[0055] With reference to the first aspect, in some implementations of the first aspect, the spline function includes a third spline function passing through the twelfth interpolation point, the thirteenth interpolation point, and the fourteenth interpolation point, a first coordinate value of the thirteenth interpolation point is between the first coordinate value of the twelfth interpolation point and the first coordinate value of the fourteenth interpolation point; before determining the plurality of interpolation points, the method further includes a step of obtaining a first offset, the first offset being used to represent a modification range of the initial tone mapping curve at the first modification position, the first modification position being the first coordinate value of the thirteenth interpolation point; the step of determining the plurality of interpolation points includes a step of determining a second coordinate value of the thirteenth interpolation point based on the function value of the initial tone mapping curve at the first modification position and the first offset; and the step of determining the spline function passing through the plurality of interpolation points includes a step of determining a third spline function based on the first coordinate value of the thirteenth interpolation point and the second coordinate value of the thirteenth interpolation point, the third spline function being smoothly connected to the initial tone mapping curve at the twelfth interpolation point and / or the fourteenth interpolation point.

[0056] With reference to the first embodiment, in some implementations of the first embodiment, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, and the step of determining a spline function passing through the multiple interpolation points includes a step of obtaining a first spline function passing through the seventh interpolation point and the eighth interpolation point, and a step of determining a second spline function passing through the eighth interpolation point and the ninth interpolation point, wherein the second spline function is smoothly connected to the first spline function at the eighth interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point.

[0057] In an example, the seventh interpolation point may be the first interpolation point mentioned above, the eighth interpolation point may be the second interpolation point mentioned above, and the ninth interpolation point may be the third interpolation point mentioned above.

[0058] In an example, the seventh interpolation point may be the fourth interpolation point mentioned above, the eighth interpolation point may be the fifth interpolation point mentioned above, and the ninth interpolation point may be the sixth interpolation point mentioned above.

[0059] In an example, the eighth interpolation point may be the twelfth interpolation point mentioned above, the seventh interpolation point may be the fourteenth interpolation point mentioned above, whose first coordinate value is less than the first coordinate value of the twelfth interpolation point and less than the first coordinate value of the thirteenth interpolation point, and the ninth interpolation point may be the fourteenth interpolation point mentioned above, whose first coordinate value is greater than the first coordinate value of the twelfth interpolation point and greater than the first coordinate value of the thirteenth interpolation point.

[0060] In this embodiment of the present application, the empirical value of the first spline function is obtained in advance, so that the correction effect within the key correction range can be ensured, and then the second spline function is used to realize the weak correction effect and the effect of smooth connection within the weak correction range.

[0061] With reference to the first aspect, in some implementations of the first aspect, the first spline function is a linear spline function and the second spline function is a cubic spline function.

[0062] In this embodiment of the present application, the parameters of the first spline function are empirical values, and for a linear spline function, there are fewer empirical values ​​that need to be known in advance, so it is easier to determine the specific form of the first spline function. When the degree of the second spline function is relatively low, the effect of the smooth connection of the second spline function is relatively weak. When the degree of the second spline function is relatively high, the preconditions and calculation amount of the second spline function are relatively large. Therefore, it is more appropriate to select a cubic spline function for the second spline function.

[0063] Referring to the first embodiment, in some implementations of the first embodiment, the plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, and the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and the first coordinate value of the eighth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; before the step of determining the plurality of interpolation points, the method further includes a step of obtaining a first offset, the first offset being used to represent a modification range of the initial tone mapping curve at the first modification position; the positive position is a first coordinate value of an eighth interpolation point, the determining a plurality of interpolation points includes determining a second coordinate value of the eighth interpolation point based on a function value of the initial tone mapping curve at the first modified position and the first offset, and the determining a spline function passing through the plurality of interpolation points includes determining a third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

[0064] In an example, the seventh interpolation point may be the first interpolation point mentioned above, the eighth interpolation point may be the second interpolation point mentioned above, and the ninth interpolation point may be the third interpolation point mentioned above.

[0065] In an example, the seventh interpolation point may be the fourth interpolation point mentioned above, the eighth interpolation point may be the fifth interpolation point mentioned above, and the ninth interpolation point may be the sixth interpolation point mentioned above.

[0066] In the example, the seventh interpolation point may be the twelfth interpolation point mentioned above, the eighth interpolation point may be the thirteenth interpolation point mentioned above, and the ninth interpolation point may be the fourteenth interpolation point mentioned above.

[0067] In this embodiment of the present application, the empirical value of the first offset can be obtained in advance, so that the correction effect within the key correction range can be ensured, and the third spline function can be used to realize the weak correction effect and the effect of smooth connection within the weak correction range.

[0068] With reference to the first embodiment, in some implementations of the first embodiment, the third spline function includes a fourth spline function passing through the seventh and eighth interpolation points, and a fifth spline function passing through the eighth and ninth interpolation points, and the second derivative value of the fourth spline function at the eighth interpolation point is identical to the second derivative value of the fifth spline function at the eighth interpolation point.

[0069] Referring to the first aspect, in some implementations of the first aspect, the plurality of interpolation points further includes a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; before the step of determining the plurality of interpolation points, the method further includes a step of obtaining a second offset, the second offset is used to represent a modification range of the initial tone mapping curve at a second modification position, and the second modification position is the first coordinate value of the tenth interpolation point; and the step of determining the plurality of interpolation points further includes a step of obtaining a second offset, the second offset is used to represent a modification range of the initial tone mapping curve at a second modification position, and the second modification position is the first coordinate value of the tenth interpolation point; determining a second coordinate value of the tenth interpolation point based on the function value of the initial tone mapping curve at the second modified position and the second offset; determining a third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point includes determining a third spline function based on the first and second coordinate values ​​of the eighth interpolation point and the first and second coordinate values ​​of the tenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at both the seventh and ninth interpolation points.

[0070] In this embodiment of the present application, the empirical values ​​of the first offset and the second offset are obtained in advance, so that the correction effect in the key correction range is controlled. In addition, the third spline function can realize the effect of weak correction and smooth connection in the weak correction range.

[0071] Referring to the first aspect, in some implementations of the first aspect, the third spline function is a cubic spline function.

[0072] In this embodiment of the present application, when the degree of the third spline function is relatively low, the effect of the smooth connection of the third spline function is relatively weak. When the degree of the third spline function is relatively high, the precondition and calculation amount of the third spline function are relatively large. Therefore, it is more appropriate to select a cubic spline function for the third spline function.

[0073] With reference to the first aspect, in some implementations of the first aspect, the derivative of the spline function is always greater than zero.

[0074] In this embodiment of the application, the restriction that the derivative at any point is greater than 0 may be used to avoid the occurrence of excessively large differences between the final obtained spline function and the initial tone mapping curve, e.g., an oscillating spline function, and to avoid excessive corrections.

[0075] Referring to the first aspect, in some implementations of the first aspect, the plurality of interpolation points further includes an eleventh interpolation point, and a second derivative of the spline function at the eleventh interpolation point is zero.

[0076] In this embodiment of the application, by limiting the second derivative of the 11th interpolation point to zero, the spline function can be controlled to increase quickly on one side of the 11th interpolation point and increase slowly on the other side of the 11th interpolation point.

[0077] With reference to the first aspect, in some implementations of the first aspect, the plurality of interpolation points further includes a 15th interpolation point, where the spline function is higher than the initial tone mapping function on one side of the 15th interpolation point and the spline function is lower than the initial tone mapping function on the other side of the 15th interpolation point.

[0078] In this embodiment of the application, the spline function passes through the 15th interpolation point, and the initial tone mapping function passes through the 15th interpolation point, and the derivative value of the spline function at the 15th interpolation point is different from the derivative value of the initial tone mapping function at the 15th interpolation point. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0079] Referring to the first aspect, in some implementations of the first aspect, the initial tone mapping curve is expressed by the following formula:

number

number

number

number

[0080] In this embodiment of the present application, the method provided in the first aspect can be applied to multiple initial tone mapping curves and has high flexibility.

[0081] According to a second aspect, an image processing device is provided, comprising: an acquisition module configured to acquire a target picture and an initial tone mapping curve for the target picture; a processing module configured to determine a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture, the processing module further configured to determine a spline function passing through the plurality of interpolation points, the spline function smoothly connecting to the initial tone mapping curve at at least one of the plurality of interpolation points, the processing module further configured to modify the initial tone mapping curve based on the spline function to obtain a modified tone mapping curve for the target picture; and a tone mapping module configured to perform tone mapping for the target picture based on the modified tone mapping curve.

[0082] With reference to the second aspect, in some implementations of the second aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the number of dark area pixels included in the target picture is greater than or equal to a second preset threshold, the luminance values ​​of the dark area pixels are smaller than the first preset threshold, the multiple interpolation points include a first interpolation point, and the processing module is specifically configured to determine that a first coordinate value of the first interpolation point is identical to the first coordinate value of the first end point.

[0083] With reference to the second aspect, in some implementations of the second aspect, the plurality of interpolation points further includes a second interpolation point, and the processing module is specifically configured to determine a first coordinate value of the second interpolation point based on the number of dark area pixels.

[0084] With reference to the second aspect, in some implementations of the second aspect, the plurality of interpolation points further includes a third interpolation point, the target picture further includes a dark area transition pixel, and the luminance value of the dark area transition pixel is greater than or equal to a first preset threshold, and the processing module is specifically configured to determine a first coordinate value of the third interpolation point based on the number of dark area transition pixels.

[0085] With reference to the second aspect, in some implementations of the second aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the number of bright area pixels included in the target picture is greater than or equal to a fourth preset threshold, the luminance values ​​of the bright area pixels are greater than a third preset threshold, the multiple interpolation points include a fourth interpolation point, and the processing module is specifically configured to determine that a first coordinate value of the fourth interpolation point is identical to a first coordinate value of the second end point.

[0086] With reference to the second aspect, in some implementations of the second aspect, the plurality of interpolation points further includes a fifth interpolation point, and the processing module is specifically configured to determine a first coordinate value of the fifth interpolation point based on the number of bright area pixels.

[0087] With reference to the second aspect, in some implementations of the second aspect, the plurality of interpolation points further includes a sixth interpolation point, the target picture further includes a light area transition pixel, and the brightness value of the light area transition pixel is less than or equal to a third pre-set threshold, and the processing module is specifically configured to determine a first coordinate value of the sixth interpolation point based on the number of light area transition pixels.

[0088] Referring to the second aspect, in some implementations of the second aspect, the dynamic metadata includes coordinate information for each of the multiple interpolation points.

[0089] With reference to the second aspect, in some implementations of the second aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the dynamic metadata includes regional luminance indication information, when the regional luminance indication information indicates a dark area, the multiple interpolation points include a first interpolation point, and a first coordinate value of the first interpolation point is identical to a first coordinate value of the first end point, and / or when the regional luminance indication information indicates a light area, the multiple interpolation points include a second interpolation point, and a first coordinate value of the second interpolation point is identical to a first coordinate value of the second end point.

[0090] Referring to the second aspect, in some implementations of the second aspect, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the dynamic metadata includes P pieces of coordinate information having a one-to-one correspondence with the P key pixels, where P is a positive integer, and the processing module specifically determines that the P key pixels are dark area pixels or bright area pixels based on the P pieces of coordinate information and the luminance information of the target picture. wherein the luminance values ​​of the dark area pixels are less than a first preset threshold and the luminance values ​​of the light area pixels are greater than a third preset threshold; when the P key pixels are dark area pixels, the processing module is specifically configured to determine that the plurality of interpolation points includes a first interpolation point, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point; or when the P key pixels are light area pixels, the processing module is specifically configured to determine that the plurality of interpolation points includes a second interpolation point, and a first coordinate value of the second interpolation point is equal to a first coordinate value of the second end point.

[0091] With reference to the second aspect, in some implementations of the second aspect, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, the processing module is specifically configured to obtain a first spline function that passes through the seventh interpolation point and the eighth interpolation point, the processing module is specifically configured to determine a second spline function that passes through the eighth interpolation point and the ninth interpolation point, the second spline function is smoothly connected to the first spline function at the eighth interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point.

[0092] With reference to the second aspect, in some implementations of the second aspect, the first spline function is a linear spline function and the second spline function is a cubic spline function.

[0093] With reference to the second aspect, in some implementations of the second aspect, the multiple interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and a first coordinate value of the eighth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point, before the processing module determines the multiple interpolation points, the acquisition module is further configured to acquire a first offset, the first offset is used to represent a modification range of the initial tone mapping curve at the first modified position, and the first modified position is the first coordinate value of the eighth interpolation point, the processing module is specifically configured to determine a second coordinate value of the eighth interpolation point based on a function value of the initial tone mapping curve at the first modified position and the first offset, the processing module is specifically configured to determine a third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, and the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

[0094] With reference to the second aspect, in some implementations of the second aspect, the third spline function includes a fourth spline function passing through the seventh and eighth interpolation points, and a fifth spline function passing through the eighth and ninth interpolation points, and the second derivative value of the fourth spline function at the eighth interpolation point is identical to the second derivative value of the fifth spline function at the eighth interpolation point.

[0095] With reference to the second aspect, in some implementations of the second aspect, the plurality of interpolation points further includes a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; before the processing module determines the plurality of interpolation points, the acquisition module is further configured to acquire a second offset, the second offset is used to represent a modification range of the initial tone mapping curve at the second modification position, and the second modification position is the first coordinate value of the tenth interpolation point; the processing module is specifically configured to determine the second coordinate value of the tenth interpolation point based on the function value of the initial tone mapping curve at the second modification position and the second offset; the processing module is specifically configured to determine the third spline function based on the first and second coordinate values ​​of the eighth interpolation point and the first and second coordinate values ​​of the tenth interpolation point, and the third spline function is smoothly connected to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

[0096] Referring to the second aspect, in some implementations of the second aspect, the third spline function is a cubic spline function.

[0097] With reference to the second aspect, in some implementations of the second aspect, the derivative of the spline function is always greater than zero.

[0098] Referring to the second aspect, in some implementations of the second aspect, the plurality of interpolation points further includes an eleventh interpolation point, and a second derivative of the spline function at the eleventh interpolation point is zero.

[0099] Referring to the second aspect, in some implementations of the second aspect, the initial tone mapping curve is expressed by the following formula:

number

number

number

number

[0100] According to a third aspect, the present application provides an image processing device including modules configured to perform any one of the implementations of the first aspect.

[0101] According to a fourth aspect, the present application provides an image processing device comprising a processor and a transmission interface, the transmission interface being configured to receive or transmit image data, and the processor being configured to call program instructions stored in a memory and execute the program instructions to perform a method according to the first aspect or any one of the implementations of the first aspect.

[0102] In a possible implementation, the image processing device may further include a memory.

[0103] For example, the image processor may be a chip.

[0104] According to a fifth aspect, the present application provides an electronic device including a processor and a memory configured to perform the method according to the first aspect or any one of the implementations of the first aspect.

[0105] According to a sixth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing program instructions which, when operating on a computer or processor, enable the computer or processor to perform a method according to the first aspect or any one of the implementations of the first aspect.

[0106] According to a seventh aspect, the present application provides a computer program product comprising a computer program which, when executed on a computer or a processor, enables the computer or the processor to perform a method according to the first aspect or any one of the implementations of the first aspect. [Brief description of the drawings]

[0107] [Figure 1] FIG. 1 is a block diagram of an image or video transmission system for implementing an embodiment of the present application.

[0108] [Diagram 2] 1 is a schematic flow chart of an image processing method according to an embodiment of the present application;

[0109] [Diagram 3] 2 is a schematic diagram of a number of initial tone mapping curves according to an embodiment of the present application;

[0110] [Figure 4] FIG. 2 is a schematic diagram of multiple spline functions according to an embodiment of the present application;

[0111] [Diagram 5] FIG. 4 is a schematic diagram of a spline function for modifying an initial tone mapping curve for dark regions according to an embodiment of the present application;

[0112] [Figure 6] FIG. 4 is a schematic diagram of a spline function for modifying an initial tone mapping curve for a light region according to an embodiment of the present application;

[0113] [Figure 7] FIG. 4 is a schematic diagram of a spline function for modifying an initial tone mapping curve for an area with weak luminance contrast according to an embodiment of the present application.

[0114] [Figure 8] FIG. 1 is a structural diagram of an image processing device according to an embodiment of the present application;

[0115] [Figure 9] FIG. 2 is a structural diagram of an image processing apparatus according to another embodiment of the present application;

[0116] [Figure 10] 1 is a structural diagram of an electronic device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0117] The technical solutions of the present application will be described below with reference to the accompanying drawings.

[0118] In an embodiment of the present application, an electronic device includes a hardware layer, an operating system layer that operates on the hardware layer, and an application layer that operates on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also referred to as a main memory). The operating system may be any one or more types of computer operating systems that implement service processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. The application layer includes applications such as a browser, an address book, a document creation software, and an instant messaging software. In addition, as long as a program that records the code of the method provided in the embodiment of the present application can be executed to execute image processing according to the method provided in the embodiment of the present application, the specific mechanism of the execution subject of the method provided in the embodiment of the present application is not specifically limited to the embodiment of the present application. For example, the method provided in the embodiment of the present application may be executed by an electronic device that can call and execute a program, or a functional module in an electronic device.

[0119] Additionally, aspects or features of the present application may be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" as used herein encompasses a computer program accessible from any computer-readable component, carrier, or medium. For example, computer-readable media may include, but are not limited to, magnetic storage components (e.g., hard disks, floppy disks, or magnetic tapes), optical disks (e.g., compact discs (CDs), or digital versatile discs (DVDs), or smart cards and flash memory components (e.g., erasable programmable read-only memory (EPROM), cards, sticks, or key drives). Additionally, various storage media described herein may refer to one or more devices configured to store information and / or other machine-readable media. The term "machine-readable media" may include, but is not limited to, wireless channels, as well as various other media capable of storing, containing, and / or carrying instructions and / or data.

[0120] The following describes a system architecture used in the embodiments of the present application. FIG. 1 is a schematic block diagram of an exemplary image display system 10 to which the embodiments of the present application are applied. As shown in FIG. 1, the image display system 10 may include an image acquisition device 12 and an image processing device 14. The image acquisition device 12 generates a high dynamic range image. The image processing device 14 may process the high dynamic range image generated by the image acquisition device 12. The image acquisition device 12, the image processing device 14, or the image processing device shown in each implementation of the present application may include one or more processors and a memory coupled to the one or more processors. The memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an EEPROM, a flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures accessible by a computer as described herein. The image capture device 12 and the image processing device 14 may include a variety of devices, including a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video game console, an in-vehicle computer, a wireless communication device, or the like.

[0121] 1 illustrates image capture device 12 and image processing device 14 as separate devices, an embodiment of the device may also include both image capture device 12 and image processing device 14, or may include the functionality of both image capture device 12 and image processing device 14, i.e., image capture device 12 and image processing device 14 may be two different parts of the same device. For example, the image capture device may be a camera in a mobile smartphone, and the image processing device may be a processor in a mobile smartphone. In such an embodiment, image capture device 12 or corresponding functionality and image processing device 14 or corresponding functionality may be implemented by using the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0122] The image capture device 12 and the image processing device 14 may be communicatively connected by using a link 13. The image processing device 14 may receive high dynamic range images from the image capture device 12 by using the link 13. The link 13 may include one or more media or devices capable of moving the high dynamic range images from the image capture device 12 to the image processing device 14. In an example, the link 13 may include one or more communication media that enable the image capture device 12 to transmit the high dynamic range images directly to the image processing device 14 in real time. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum, or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or any other devices that facilitate communication from the image capture device 12 to the image processing device 14.

[0123] For example, image capture device 12 includes a picture source 16, which may also be referred to as a transmission interface, and a communication interface 22. In addition, optionally, image capture device 12 may further include an image processor 20. In a particular implementation, image processor 20, picture source 16, and communication interface 22 may be hardware components in image capture device 12 or may be software programs in image capture device 12. Descriptions are provided separately below.

[0124] Picture source 16 may include or be, for example, any type of imaging device configured to capture real-world pictures and / or any type of device for generating pictures or comments (for encoding screen content, some text on the screen is also considered as part of the picture or image to be encoded), for example, a computer graphics processing unit configured to generate computer animated images, or any type of device configured to obtain and / or provide real-world pictures or computer animated images (e.g., screen content or virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). Picture source 16 may be a camera configured to capture pictures or a memory configured to store pictures. Picture source 16 may further include any type of interface (internal or external) in which previously captured or generated pictures are stored and / or through which pictures are acquired or received. When the picture source 16 is a camera, the picture source 16 can be, for example, a local camera or a camera integrated in an image acquisition device. When the picture source 16 is a memory, the picture source 16 can be, for example, a local memory or a memory integrated in an image acquisition device. When the picture source 16 includes an interface, the interface can be, for example, an external interface for receiving pictures from an external video source. The external video source is, for example, an external imaging device such as a camera, an external memory or an external picture generation device. The external picture generation device is, for example, an external computer graphics processing unit, a computer, or a server.The interface can be any type of interface following any proprietary or standard interface protocol, such as high definition multimedia interface (HDMI), mobile industry processor interface (MIPI), MIPI standard display serial interface (DSI), Video Electronics Standards Association (VESA) standard embedded display port (eDP), or V-by-One interface, which is a digital interface standard developed for image transmission and various wired or wireless interfaces, optical interfaces, or the like.

[0125] A picture may be considered as a two-dimensional array or matrix of picture elements. A pixel in the array may also be referred to as a sample. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, three color components are usually used. Specifically, a picture may be represented as or may contain three sample arrays. For example, in an RGB format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is usually represented in a luma / chroma format or color space. For example, a picture in a YUV format contains a luma component denoted by Y (or sometimes by L) and two chroma components denoted by U and V. The luma component Y represents the luminance or gray level intensity (e.g., both are the same in a grayscale picture), and the two chrominance (chroma) components U and V represent the chroma or color information components. Correspondingly, a picture in YUV format includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(U and V). A picture in RGB format may be transformed or converted to YUV format, and vice versa. This process is also referred to as color conversion or color format conversion. If the picture is monochrome, the picture may include only a luma sample array. In this embodiment of the present application, the picture transmitted by the picture source 16 to the image processor may also be referred to as raw image data 17.

[0126] Image processor 20 is configured to perform image processing such as shaping, color format conversion (eg, from RGB format to YUV format), toning, or noise removal.

[0127] The communication interface 22 may be configured to receive the processed image data and transmit the image processed picture data 19 over the link 13 to the image processing device 14 or any other device (e.g., memory) for storage or direct reconstruction. The other device may be any device used for storage. The communication interface 22 may be configured, for example, to encapsulate the image processed picture data 19 into a suitable format, for example, into data packets, for transmission over the link 13.

[0128] The image processing device 14 includes a communication interface 28, an image processor 32, and a display device 34. A description is provided separately below.

[0129] The communication interface 28 may be configured to receive the image-processed picture data 19 from the image capture device 12 or any other source. The other source may be, for example, a storage device, which is a device for storing the image-processed picture data. For example, specific examples of the communication interface 22 and the communication interface 28 are referred to in the above description of the interfaces. The details will not be described again here. The communication interface 28 may be configured to transmit or receive the image-processed picture data 19 through the link 13 between the image capture device 12 and the image processing device 14, or through any type of network. The link 13 may be, for example, a direct wired or wireless connection. The any type of network may be, for example, a wired or wireless network, or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface 28 may be, for example, configured to decapsulate the data packets transmitted through the communication interface 22 to obtain the image-processed picture data 19.

[0130] Both communication interface 28 and communication interface 22 may be configured as unidirectional or bidirectional communication interfaces, for example, to send and receive messages to establish connections, acknowledge and exchange any other information related to the communication link and / or data transmission, such as the transmission of image-processed picture data.

[0131] The image processor 32 is configured to perform a tone mapping operation on the image processed picture data 19 to obtain tone mapped picture data 33. The processing performed by the image processor 32 may further include color format conversion (e.g. from YUV format to RGB format), toning, shaping, resampling, or any other processing. The image processor 32 may further be configured to transmit the tone mapped picture data 33 to a display device 34.

[0132] A display device 34 is configured to receive the tone mapped picture data 33 and display the picture, for example to a user or viewer. The display device 34 may be or include any type of display for presenting the reconstructed image, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCOS), a digital light processor (DLP), or any other type of display.

[0133] 1 illustrates image capture device 12 and image processing device 14 as separate devices, an embodiment of the device may also include both image capture device 12 and image processing device 14, or may include functionality of both image capture device 12 and image processing device 14, i.e., image capture device 12 or corresponding functionality and image processing device 14 or corresponding functionality. In such an embodiment, image capture device 12 or corresponding functionality and image processing device 14 or corresponding functionality may be implemented by using the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0134] 1 illustrates image processing device 14 and display device 34 as separate devices, an embodiment of the device may also include both image processing device 14 and display device 34, or may include functionality of both image processing device 14 and display device 34, i.e., image processing device 14 or corresponding functionality and display device 34 or corresponding functionality. In such an embodiment, image processing device 14 or corresponding functionality and display device 34 or corresponding functionality may be implemented by using the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0135] Based on the description, a person skilled in the art will readily understand that the presence and (exact) division of the functions of different units or functions / functionality of image capture device 12 and / or image processing device 14 shown in Fig. 1 may vary for each actual device and application. Image capture device 12 and image processing device 14 may include any one of a wide range of devices including any type of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet-type computer, a video camera, a desktop computer, a set-top box, a television, a camera, an in-vehicle device, a display device, a digital media player, a video game console, a video streaming device (e.g., a content service server or a content distribution server), a broadcast receiving device, or a broadcast transmitting device, or may or may not use any type of operating system.

[0136] Image processor 20 and image processor 32 may each be implemented as any one of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques are implemented in part by using software, a device may store software instructions in a suitable non-transitory computer-readable storage medium and execute the instructions by using hardware, such as one or more processors, to perform the techniques of this disclosure. Any one of the above (including hardware, software, combinations of hardware and software, and the like) may be considered to be one or more processors.

[0137] An image capture device may capture a light signal from a natural scene or a picture or video including a natural scene. To facilitate the transmission of the image or video, the light signal needs to be converted into an electrical signal, and the image information of each pixel needs to be recorded by using a luminance or chrominance value with a fixed value range (e.g., a grayscale value between 0 and 255). The curve that converts the light signal into an electrical signal is an optical-to-electrical conversion function. Common optical-to-electrical conversion functions include a perceptual quantizer (PQ) optical-to-electrical conversion function, a hybrid log-gamma (HLG) optical-to-electrical conversion function, and a scene luminance fidelity (SLF) optical-to-electrical conversion function. A display device may reproduce a natural scene by displaying the picture or video. Generally, the image capture device and the display device are located in two different positions. Therefore, the image capture device is often referred to as a front-end device, and the display device is referred to as a back-end device. The display device may determine the brightness of the object being photographed based on the light-to-electricity conversion function and the brightness or grayscale value of each pixel in the image, i.e., the electronic device may convert the YUV or RGB information into brightness in nits. The brightness of the object may exceed the display capability of the display device. Since the brightness information acquired by the image acquisition device does not match the brightness display capability of the display device, a display device with low brightness display capability may be used to display a high brightness image, and a display device with high brightness display capability may be used to display a low brightness image. The first case is also called a high-low tone mapping process, and the second case is also called a low-high tone mapping process. Tone mapping is a computer graphics technique for approximately displaying a high dynamic range image on a display device with a limited dynamic range display capability.In one approach, the tones of a high dynamic range image may be mapped to a display device having a low dynamic range display capability based on a tone mapping curve. Note that the tone mapping curve may be determined by the display device or may be determined by a processing device located outside the display device.

[0138] However, in some cases, when the determined tone mapping curve is inaccurate, the display effect of the high dynamic range image is low. For example, in the video playback process, if only one tone mapping curve is determined for the video, the image content displayed between different video frames may be significantly different. For example, the image displayed in some video frames is very dark, while the image displayed in other video frames is very bright. If a fixed tone mapping curve is used for the whole video, the picture of each frame in the video cannot be reproduced perfectly on the display device. In addition, the tone mapping curve holds a large amount of data. If one tone mapping curve is determined for each image or picture frame, data redundancy may result.

[0139] The present application provides an image processing method and apparatus to avoid the above-mentioned problems.

[0140] 2 is a schematic flow chart of an image processing method according to the present application. The method shown in FIG. 2 may be performed by a processor used for tone mapping. For example, the processor may be the image processing device 14 shown in FIG. 1, an image acquisition device including a tone mapping processor, or a display device including a tone mapping processor.

[0141] 201: Obtain a target picture and an initial tone mapping curve for the target picture.

[0142] The target picture may be obtained by using the picture source 16 shown in Figure 1. The target picture may be a frame of a picture in a video, or may be a still picture, or may be a region in a picture, for example, multiple image blocks in a picture.

[0143] The initial tone mapping curve may be any form of curve that can be used in the tone mapping process. Specifically, FIG. 3 shows several possible forms of the initial tone mapping curve.

[0144] The initial tone mapping curve shown by the solid line in FIG. 3 is also called the SLF tone mapping curve.

number

[0145] The initial tone mapping curve, shown by the dashed-dotted line in FIG. 3, is a first-order polynomial tone mapping function,

number

[0146] The initial tone mapping curve, shown by the dashed line in FIG. 3, is also called a sigmoid tone mapping curve.

number

[0147] The initial tone mapping curve, shown by the dashed-dotted line in FIG. 3, is also referred to as the Bessel tone mapping curve.

number

[0148] The electronic device may determine parameters on the initial tone mapping curve based on luminance information and / or dynamic metadata of the target picture.

[0149] It should be understood that both luminance values ​​in YUV format and grayscale values ​​in RGB format can reflect the luminance of an image. Therefore, luminance information in this application can be, for example, luminance values ​​in YUV format or grayscale values ​​in RGB format.

[0150] It should be understood that the dynamic metadata may include information about the luminance ranges or about the proportion of each luminance range. For example, the number of pixels in the luminance range of 0-10 nits is 200. The dynamic metadata may further include the average luminance of the pixels in the luminance range. For example, the average luminance of the pixels in the luminance range of 0-10 nits is 6.2 nits.

[0151] Using the formula of the SLF is an example, where p and m may be determined by the overall brightness of the image. When the overall brightness of the image is high, the value of p is relatively small and the value of m is relatively large. When the overall brightness of the image is low, the value of p is relatively large and the value of m is relatively small. The value range of p may be 3 to 6. The value range of m may be 2 to 4, and m is usually 2.4. The electronic device may statistically analyze the brightness information to determine the overall brightness of the image. Based on the information included in the dynamic metadata, the electronic device may determine that the overall brightness of the image is high when the ratio of bright pixels is relatively high, and that the overall brightness of the image is low when the ratio of dark pixels is relatively high.

[0152] Optionally, the initial tone mapping curve may be a curve that has undergone a normalization process, i.e., the coordinates of any point on the initial tone mapping curve are (first coordinate value 1, second coordinate value 1). The value range of the first coordinate value 1 is 0 to 1. The value range of the second coordinate value 1 is 0 to 1. In the following, the initial tone mapping curve that has undergone a normalization process will be described. It should be understood that this is not a limitation in the present application.

[0153] 202: Determining a plurality of interpolation points based on luminance information and / or dynamic metadata of a target picture.

[0154] The purpose of determining multiple interpolation points is to determine a spline function for modifying the initial tone mapping curve. By determining multiple interpolation points, the spline function can be defined to pass through the multiple interpolation points, and an image display effect with a better tone mapping effect is obtained.

[0155] Determining the interpolation points may be determining coordinates of the interpolation points, or first or second coordinate values ​​of the interpolation points. The coordinates of the interpolation points may be expressed as (x, y), where x may represent the first coordinate value and y may represent the second coordinate value. The electronic device may determine the multiple interpolation points based on luminance information of the image, or based on dynamic metadata of the image, or based on luminance information of the image and dynamic metadata of the image. For example, the electronic device determines a first coordinate value of interpolation point 1, a second coordinate value of interpolation point 2, and a coordinate of interpolation point 3.

[0156] In this application, the purpose of determining an interpolation point is to determine a modification position of an initial tone mapping curve. In other words, there may be a modification range between two interpolation points included in a plurality of interpolation points, and the electronic device may modify the initial tone mapping curve within the modification range.

[0157] In an example, the electronic device may determine that when the first coordinate value of interpolation point 1 is first coordinate value 1 and the first coordinate value of interpolation point 2 is first coordinate value 2, first coordinate value 1 is smaller than first coordinate value 2, and the electronic device may decide to modify the initial tone mapping curve in the interval from first coordinate value 1 to first coordinate value 2.

[0158] Similarly, the electronic device may determine that when the second coordinate value of interpolation point 1 is second coordinate value 1 and the second coordinate value of interpolation point 2 is second coordinate value 2, second coordinate value 1 is smaller than second coordinate value 2, and the electronic device may decide to modify the initial tone mapping curve in the interval from second coordinate value 1 to second coordinate value 2.

[0159] In the example, the electronic device determines that the first coordinate value of the interpolation point 1 is the first coordinate value 1 and the second coordinate value of the interpolation point 2 is the second coordinate value 2. The electronic device may determine that the first coordinate value corresponding to the second coordinate value 2 is the first coordinate value 2 based on the initial tone mapping curve. The electronic device may determine that the first coordinate value 1 is smaller than the first coordinate value 2, and decide to modify the initial tone mapping curve in the interval from the second coordinate value 1 to the second coordinate value 2.

[0160] Similarly, the electronic device determines that the second coordinate value of the interpolation point 1 is the first coordinate value 1, and that the second coordinate value of the interpolation point 2 is the second coordinate value 2. The electronic device may determine, based on the initial tone mapping curve, that the second coordinate value corresponding to the first coordinate value 1 is the second coordinate value 1. The electronic device may determine that the second coordinate value 1 is smaller than the second coordinate value 2, and decide to modify the initial tone mapping curve in the interval from the second coordinate value 1 to the second coordinate value 2.

[0161] In an example, the electronic device may determine that when the coordinates of interpolation point 1 are (first coordinate value 1, second coordinate value 1) and the coordinates of interpolation point 2 are (first coordinate value 2, second coordinate value 2), the first coordinate value 1 is smaller than the first coordinate value 2 (because the initial tone mapping curve is typically a monotonically increasing function, the second coordinate value 1 is smaller than the second coordinate value 2), and may decide to modify the initial tone mapping curve in the interval from the first coordinate value 1 to the first coordinate value 2 (or the interval from the second coordinate value 1 to the second coordinate value 2).

[0162] The initial tone mapping curve includes two end points. These are a first end point and a second end point, for example, point 1 and point 5 shown in FIG. 4, respectively. The luminance value corresponding to the first end point is relatively low, and the first end point is located in a dark area of ​​the image. The luminance value corresponding to the second end point is relatively high, and the second end point is located in a light area of ​​the image. When the image is dark, the electronic device may determine that the interpolation point is located at or near the first end point. When the image is bright, the electronic device may determine that the interpolation point is located at or near the second end point. When the image includes a portion of pixels with a relatively small luminance difference, the location of the interpolation point may be determined based on the luminance of the portion of the pixels and the overall luminance of the image. For example, when the luminance range of the picture is between 100 nit and 300 nit, and 50% of the pixels are within the range of 195 nit and 205 nit, the interpolation point may be determined to be located near the central segment of the initial tone mapping curve.

[0163] Scenario 1: When the image is dark, the electronic device may determine that the interpolation point is at or near the first endpoint. Figure 5 shows the interpolation points and spline function in this scenario.

[0164] Optionally, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point being respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, a number of dark area pixels included in the target picture is equal to or greater than a second preset threshold, and the luminance values ​​of the dark area pixels are smaller than the first preset threshold, the plurality of interpolation points includes a first interpolation point, and the step of determining the plurality of interpolation points based on the luminance information of the target picture includes a step of determining that a first coordinate value of the first interpolation point is identical to a first coordinate value of the first end point.

[0165] In this application, a luminance value corresponding to a first endpoint may correspond to a first coordinate value of the first endpoint and / or a second coordinate value of the first endpoint, and a luminance value corresponding to a second endpoint may correspond to a first coordinate value of the second endpoint and / or a second coordinate value of the second endpoint.

[0166] For example, if the first preset threshold is 20, a pixel whose Y component value in YUV format is less than 20 may be considered as a dark area pixel, or a pixel whose R / G / B component value in RGB format is less than 20 may be considered as a dark area pixel. In another example, if the first preset threshold is 20 nit, a pixel whose luminance is less than 20 nit may be considered as a dark area pixel.

[0167] Therefore, when the target picture includes a sufficient number of dark area pixels, the first end point can be selected as the first interpolation point, and the first coordinate value of the first end point is determined to be the same as the first coordinate value of the first interpolation point. In other words, when the target picture includes a sufficient number of dark area pixels, the segment close to the first end point on the initial tone mapping curve can be modified.

[0168] Optionally, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point being respectively located at two ends of the initial tone mapping curve; the dynamic metadata includes P pieces of coordinate information having a one-to-one correspondence with the P key pixels, where P is a positive integer; determining a plurality of interpolation points based on the luminance information of the target picture and the dynamic metadata includes determining, based on the P pieces of coordinate information and the luminance information of the target picture, that the P key pixels are dark area pixels, where the luminance values ​​of the dark area pixels are smaller than a first preset threshold; and determining that when the P key pixels are dark area pixels, the plurality of interpolation points include a first interpolation point, where a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point.

[0169] For example, if the first preset threshold is 20, a pixel whose Y component value or average value in the YUV format is less than 20 may be considered to be a dark area pixel, or a pixel whose R / G / B component value or average value in the RGB format is less than 20 may be considered to be a dark area pixel. In another example, if the first preset threshold is 20 nit, a pixel whose luminance or average luminance is less than 20 nit may be considered to be a dark area pixel.

[0170] Thus, the dynamic metadata indicates a number of key pixels, and whether the number of key pixels are dark area pixels can be determined based on the average luminance or maximum luminance of the key pixels. When the number of key pixels are dark area pixels, a first end point of the luminance mapping curve is used as a first interpolation point, and a first coordinate value of the first end point is determined to be the same as a first coordinate value of the first interpolation point. In other words, when the number of key pixels are dark area pixels, a segment close to the first end point on the initial tone mapping curve can be modified.

[0171] In this application, a dark area pixel can be understood as a pixel having low luminance. A dark area pixel can be a key pixel indicated in dynamic metadata. Alternatively, a dark area pixel can be determined by an electronic device autonomously based on the luminance distribution of pixels. In other words, whether a target picture includes a dark area pixel can be determined in different manners.

[0172] Optionally, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point being respectively located at two ends of the initial tone mapping curve, and the dynamic metadata includes regional luminance indication information, and when the regional luminance indication information indicates a dark region, the plurality of interpolation points includes a first interpolation point, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point.

[0173] When the dynamic metadata includes an indication information indicating a dark region, the first coordinate value of the first interpolation point is the same as the first coordinate value of the first end point. The first interpolation point is, for example, point 1' shown in Fig. 4. For example, when the indication information is 0 or includes an indication of "dark", it is determined that the first coordinate value of the first interpolation point is the same as the first coordinate value of the first end point.

[0174] The initial tone mapping curve is shown in FIG. 4. The two ends of the initial tone mapping curve are point 1 and point 5, respectively. The first end point may be point 1 shown in FIG. 4, and the second end point may be point 5 shown in FIG. 4. It can be seen from the figure that the luminance value corresponding to the first end point is low, and the luminance value corresponding to the second end point is high. As shown in FIG. 4, the first interpolation point may be point 1', the first end point may be point 1, and the first coordinate value of point 1 is the same as the first coordinate value of point 1'. In some cases, point 1 coincides with point 1', that is, the first coordinate value of point 1 is the same as the first coordinate value of point 1', and the second coordinate value of point 1 is the same as the second coordinate value of point 1'. The second coordinate value of the first interpolation point may be obtained based on an experience value, a user input value, or the initial tone mapping curve.

[0175] Then, the modification range of the initial tone mapping curve needs to be determined. Specifically, the second interpolation point can be determined, so that the initial tone mapping curve between the first and second interpolation points is modified to ensure that the high dynamic range picture can be displayed on the display device with low dynamic range display capability, or ensure that the image details of the dark scene are better presented on the display device. As shown in FIG. 4, the fifth interpolation point can be point 2'.

[0176] Optionally, the plurality of interpolation points further includes a second interpolation point, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the second interpolation point based on the number of dark area pixels.

[0177] When the number of dark area pixels is relatively large or the ratio of dark area pixels to all pixels is relatively high, the first coordinate value of the second interpolation point is close to the first coordinate value of the first interpolation point. When the number of dark area pixels is relatively small or the ratio of dark area pixels to all pixels is relatively low, the first coordinate value of the second interpolation point is far from the first coordinate value of the first interpolation point. In other words, when the number of dark area pixels is relatively large, the modification range of the initial tone mapping curve close to the first end point is relatively small, but when the number of dark area pixels is relatively small and is larger than the second preset threshold, the modification range of the initial tone mapping curve close to the first end point is relatively large. The second coordinate value of the second interpolation point can be obtained based on an experience value, a user input value, or an initial tone mapping curve.

[0178] In addition, the modified tone mapping curve needs to be smooth and differentiable. Therefore, a third interpolation point may be determined so that the spline function for modifying the initial tone mapping curve can be smoothly connected to the initial tone mapping curve at the third interpolation point.

[0179] The correction range for correcting the initial tone mapping curve may include a strong correction range and a weak correction range. The correction range between the first and second interpolation points may be understood as a strong correction range, and the correction range between the second and third interpolation points may be understood as a weak correction range. The weak correction range is a transition correction range between the strong correction range and the non-correction range. Similarly, the spline function for correcting the initial tone mapping curve may include a strong correction spline function and a weak correction spline function. The spline function between the first and second interpolation points may be understood as a strong correction spline function, and the spline function between the second and third interpolation points may be understood as a weak correction spline function. The weak correction spline function is a transition correction spline function between the strong correction spline function and the initial tone mapping curve. In addition, since the spline function between the second and third interpolation points is different from the initial tone mapping curve, the spline function between the second and third interpolation points has a curve correction function in addition to the transition curve function. As shown in FIG. 4, the third interpolation point may be point 4'.

[0180] Optionally, the plurality of interpolation points further includes a third interpolation point, the target picture further includes a dark area transition pixel, a luminance value of the dark area transition pixel being greater than or equal to a first preset threshold, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the third interpolation point based on a number of the dark area transition pixels.

[0181] A dark area transition pixel is a pixel whose luminance is greater than the first preset threshold mentioned above. For example, if the first preset threshold is 20, a pixel whose Y component value in YUV format is greater than 20 may be considered to be a dark area transition pixel, or a pixel whose R / G / B component value in RGB format is greater than 20 may be considered to be a dark area transition pixel. For another example, if the third preset threshold is 100 nit, a pixel whose luminance is greater than 100 nit may be considered to be a dark area transition pixel. In some cases, the luminance of a dark area transition pixel is less than the fifth preset threshold, and the fifth preset threshold is greater than the first preset threshold.

[0182] Optionally, the plurality of interpolation points further includes a third interpolation point, and the dynamic metadata includes Q pieces of coordinate information having a one-to-one correspondence with the Q key pixels, where Q is a positive integer, and when the plurality of interpolation points includes the first interpolation point, determining the plurality of interpolation points based on the luminance information and dynamic metadata of the target picture includes: determining, based on the Q pieces of luminance information and coordinate information of the target picture, that the Q key pixels are dark area transition pixels (the luminance values ​​of the dark area pixels are greater than or equal to a first preset threshold); and determining that the plurality of interpolation points includes a third interpolation point when the Q key pixels are dark area pixels (the first coordinate value of the third interpolation point is the same as the first coordinate value of the first end point).

[0183] The dark area transition pixel is a pixel whose luminance or average luminance is greater than the first preset threshold mentioned above. For example, if the first preset threshold is 20, a pixel whose Y component value or average value in YUV format is greater than 20 may be considered to be a dark area transition pixel, or a pixel whose R / G / B component value or average value in RGB format is greater than 20 may be considered to be a dark area transition pixel. For another example, if the third preset threshold is 100 nit, a pixel whose luminance or average luminance is greater than 100 nit may be considered to be a dark area transition pixel. In some cases, the luminance of the dark area transition pixel is less than the fifth preset threshold, and the fifth preset threshold is greater than the first preset threshold.

[0184] When the number of dark area transition pixels is relatively large or the ratio of the dark area transition pixels to all pixels is relatively high, the first coordinate value of the third interpolation point is close to the first coordinate value of the second interpolation point. When the number of dark area transition pixels is relatively small or the ratio of the dark area transition pixels to all pixels is relatively low, the first coordinate value of the third interpolation point is far from the first coordinate value of the second interpolation point. In other words, when the number of dark area transition pixels is relatively large, the weak modified spline function is relatively short, but when the number of dark area transition pixels is relatively small, the weak modified spline function is relatively long.

[0185] In this application, the dark area transition pixel may be understood as a pixel that is lower in luminance and higher in luminance than the dark area pixel. The dark area transition pixel may be a key pixel indicated in the dynamic metadata. Alternatively, the dark area transition pixel may be determined autonomously by the electronic device based on the luminance distribution of the pixel. In other words, the number of the dark area transition pixels in the target picture may be determined in different manners. The second coordinate value of the third interpolation point may be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0186] Scenario 2: When the image is bright, the electronic device may determine that the interpolation point is at or near the second endpoint. Figure 6 shows the interpolation points and spline function in this scenario.

[0187] Optionally, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point being respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, a number of bright area pixels included in the target picture is equal to or greater than a fourth preset threshold, and the luminance values ​​of the bright area pixels are greater than a third preset threshold, the plurality of interpolation points includes a fourth interpolation point, and the step of determining the plurality of interpolation points based on the luminance information of the target picture includes a step of determining that a first coordinate value of the fourth interpolation point is identical to a first coordinate value of the second end point.

[0188] In this application, a luminance value corresponding to a first endpoint may correspond to a first coordinate value of the first endpoint and / or a second coordinate value of the first endpoint, and a luminance value corresponding to a second endpoint may correspond to a first coordinate value of the second endpoint and / or a second coordinate value of the second endpoint.

[0189] For example, if the third preset threshold is 200, a pixel whose Y component value in YUV format is greater than 200 may be considered to be a bright area pixel, or a pixel whose R / G / B component value in RGB format is greater than 200 may be considered to be a bright area pixel. In another example, if the third preset threshold is 400 nit, a pixel whose luminance is greater than 400 nit may be considered to be a bright area pixel.

[0190] Therefore, when the target picture includes a sufficient number of bright area pixels, the second end point can be determined as an interpolation point (corresponding to the fourth interpolation point in this embodiment) for modifying the bright area. For example, it is determined that the first coordinate value of the second end point is the same as the first coordinate value of the fourth interpolation point. In other words, when the target picture includes a sufficient number of bright area pixels, the segment close to the second end point on the initial tone mapping curve can be modified.

[0191] Optionally, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point being respectively located at two ends of the initial tone mapping curve; the dynamic metadata includes P pieces of coordinate information having a one-to-one correspondence with the P key pixels, where P is a positive integer; determining a plurality of interpolation points based on the luminance information of the target picture and the dynamic metadata includes determining, based on the P pieces of coordinate information and the luminance information of the target picture, that the P key pixels are bright area pixels, where the luminance values ​​of the bright area pixels are greater than a third preset threshold; and when the P key pixels are bright area pixels, the plurality of interpolation points include a fourth interpolation point, where a first coordinate value of the fourth interpolation point is equal to a first coordinate value of the second end point.

[0192] For example, if the third preset threshold is 200, a pixel whose Y component value or average value in YUV format is greater than 200 may be considered to be a bright area pixel, or a pixel whose R / G / B component value or average value in RGB format is greater than 200 may be considered to be a bright area pixel. In another example, if the third preset threshold is 400 nit, a pixel whose luminance or average luminance is greater than 400 nit may be considered to be a bright area pixel.

[0193] Thus, the dynamic metadata indicates a number of key pixels, and whether the number of key pixels are bright area pixels may be determined based on the average luminance or maximum luminance of the key pixels. When the number of key pixels are bright area pixels, it is determined that the first coordinate value of the second end point is the same as the first coordinate value of the fourth interpolation point. In other words, when the number of key pixels are bright area pixels, a segment on the initial tone mapping curve close to the second end point may be modified.

[0194] In this application, a bright area pixel can be understood as a pixel having high luminance. A bright area pixel can be a key pixel indicated in dynamic metadata. Alternatively, a bright area pixel can be determined autonomously by an electronic device based on the luminance distribution of pixels. In other words, whether a target picture includes a bright area pixel can be determined in different manners.

[0195] Optionally, the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point being respectively located at two ends of the initial tone mapping curve, and the dynamic metadata includes regional luminance indication information, and when the regional luminance indication information indicates a light region, the plurality of interpolation points includes a fourth interpolation point, and a first coordinate value of the fourth interpolation point is equal to a first coordinate value of the second end point.

[0196] When the dynamic metadata includes an indication information indicating a bright region, the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second end point. The fourth interpolation point is, for example, point 5' shown in Figure 4. For example, when the indication information includes a "light" or another indication related to a bright region, it is determined that the first coordinate value of the fourth interpolation point is the same as the first coordinate value of the second end point.

[0197] The initial tone mapping curve is shown in FIG. 4. The two ends of the initial tone mapping curve are point 1 and point 5, respectively. The first end point may be point 1 shown in FIG. 4, and the second end point may be point 5 shown in FIG. 4. It can be seen from the figure that the luminance value corresponding to the first end point is low, and the luminance value corresponding to the second end point is high. As shown in FIG. 4, the fourth interpolation point may be point 5', the first end point may be point 5, and the first coordinate value of point 5 is the same as the first coordinate value of point 5'. In some cases, point 5 coincides with point 5', that is, the first coordinate value of point 5 is the same as the first coordinate value of point 5', and the second coordinate value of point 5 is the same as the second coordinate value of point 5'. The second coordinate value of the fourth interpolation point may be obtained based on an experience value, a user input value, or the initial tone mapping curve.

[0198] Then, the modification range of the initial tone mapping curve needs to be determined. Specifically, the fifth interpolation point can be determined, so that the initial tone mapping curve between the fourth and fifth interpolation points is modified to ensure that the high dynamic range picture can be displayed on the display device with low dynamic range display capability, or ensure that the image details of the bright scene are better presented on the display device. As shown in FIG. 4, the fifth interpolation point can be point 6'.

[0199] Optionally, the plurality of interpolation points further includes a fifth interpolation point, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the fifth interpolation point based on the number of bright area pixels.

[0200] When the number of bright area pixels is relatively large or the ratio of bright area pixels to all pixels is relatively high, the first coordinate value of the fifth interpolation point is close to the first coordinate value of the fourth interpolation point. When the number of bright area pixels is relatively small or the ratio of bright area pixels to all pixels is relatively low, the first coordinate value of the fifth interpolation point is far from the first coordinate value of the fourth interpolation point. In other words, when the number of bright area pixels is relatively large, the modification range of the initial tone mapping curve near the second end point is relatively small, but when the number of bright area pixels is relatively small and is greater than the fourth preset threshold, the modification range of the initial tone mapping curve near the second end point is relatively large. The second coordinate value of the fifth interpolation point can be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0201] In addition, the modified tone mapping curve needs to be smooth and differentiable. Therefore, a sixth interpolation point may be determined, so that the spline function for modifying the initial tone mapping curve may be smoothly connected to the initial tone mapping curve at the sixth interpolation point. As shown in FIG. 4, the sixth interpolation point may be point 8'.

[0202] The correction range for correcting the initial tone mapping curve may include a strong correction range and a weak correction range. The correction range between the fourth and fifth interpolation points may be understood as a strong correction range, and the correction range between the fifth and sixth interpolation points may be understood as a weak correction range. The weak correction range is a transition correction range between the strong correction range and the non-correction range. Similarly, the spline function for correcting the initial tone mapping curve may include a strong correction spline function and a weak correction spline function. The spline function between the fourth and fifth interpolation points may be understood as a strong correction spline function, and the spline function between the fifth and sixth interpolation points may be understood as a weak correction spline function. The weak correction spline function is a transition correction spline function between the strong correction spline function and the initial tone mapping curve. In addition, since the spline function between the fifth and sixth interpolation points is different from the initial tone mapping curve, the spline function between the fifth and sixth interpolation points has a curve correction function in addition to the transition curve function.

[0203] Optionally, the plurality of interpolation points further includes a sixth interpolation point, the target picture further includes a light area transition pixel, the luminance value of the light area transition pixel is less than or equal to a third preset threshold, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the sixth interpolation point based on the number of the light area transition pixels.

[0204] A bright region transition pixel is a pixel whose luminance is greater than the third preset threshold mentioned above. For example, if the third preset threshold is 20, a pixel whose Y component value in YUV format is greater than 20 may be considered to be a bright region transition pixel, or a pixel whose R / G / B component value in RGB format is greater than 20 may be considered to be a bright region transition pixel. In another example, if the third preset threshold is 100 nit, a pixel whose luminance is greater than 100 nit may be considered to be a bright region transition pixel. In some cases, the luminance of a bright region transition pixel is greater than the sixth preset threshold, and the sixth preset threshold is less than the third preset threshold.

[0205] Optionally, the plurality of interpolation points further includes a sixth interpolation point, and the dynamic metadata includes Q pieces of coordinate information that have a one-to-one correspondence with the Q key pixels, where Q is a positive integer, and when the plurality of interpolation points includes a fourth interpolation point, determining the plurality of interpolation points based on the luminance information and the dynamic metadata of the target picture includes: determining, based on the Q pieces of coordinate information and the luminance information of the target picture, that the Q key pixels are bright area transition pixels (the luminance values ​​of the bright area pixels are less than or equal to a first preset threshold); and determining that the plurality of interpolation points includes a sixth interpolation point when the Q key pixels are bright area pixels (the first coordinate value of the sixth interpolation point is the same as the first coordinate value of the first end point).

[0206] A bright region transition pixel is a pixel whose luminance or average luminance is less than the third preset threshold mentioned above. For example, if the third preset threshold is 200, a pixel whose Y component value or average value in a YUV format is less than 200 may be considered to be a bright region transition pixel, or a pixel whose R / G / B component value or average value in an RGB format is less than 200 may be considered to be a bright region transition pixel. In another example, if the third preset threshold is 400 nit, a pixel whose luminance or average luminance is less than 400 nit may be considered to be a bright region transition pixel. In some cases, the luminance of a bright region transition pixel is greater than the sixth preset threshold, and the sixth preset threshold is less than the third preset threshold.

[0207] When the number of light region transition pixels is relatively large or the ratio of the light region transition pixels to all pixels is relatively high, the first coordinate value of the third interpolation point is close to the first coordinate value of the sixth interpolation point. When the number of light region transition pixels is relatively small or the ratio of the light region transition pixels to all pixels is relatively low, the first coordinate value of the third interpolation point is far from the first coordinate value of the sixth interpolation point. In other words, when the number of light region transition pixels is relatively large, the weak modified spline function is relatively short, but when the number of light region transition pixels is relatively small, the weak modified spline function is relatively long.

[0208] In this application, the bright area transition pixels may be understood as pixels that are brighter and less bright than the bright area pixels. The bright area transition pixels may be key pixels indicated in the dynamic metadata. Alternatively, the bright area transition pixels may be determined autonomously by the electronic device based on the luminance distribution of the pixels. In other words, the number of the bright area transition pixels in the target picture may be determined in different manners. The second coordinate value of the sixth interpolation point may be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0209] Scenario 3: The image contains a portion of pixels with relatively small luminance differences. Figure 7 shows the interpolation points and spline function in this scenario.

[0210] Optionally, the target picture includes a target pixel having a luminance value between a first luminance value and a second luminance value, a ratio of the target pixel to all pixels of the target picture is greater than an eighth preset threshold, a difference between the first luminance value and the second luminance value is less than a seventh preset threshold, and the plurality of interpolation points includes a twelfth interpolation point, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining the twelfth interpolation point based on an average luminance of the target pixel, a minimum luminance value of the target picture, and a maximum luminance value of the target picture.

[0211] The luminance values ​​may be expressed in YUV, RGB, or Luminance.

[0212] For example, the maximum luminance value of the target picture is 500nit and the minimum luminance value is 100nit. It is assumed that the seventh preset threshold is 50nit and the eighth preset threshold is 30%. The luminance values ​​of 50% of the pixels in the target picture are in the range of 240-260nit, i.e., the first luminance value is 240nit, the second luminance value is 260nit, and the ratio of the target pixel to all pixels of the target picture is 50%. It can be seen that the difference between the first luminance value and the second luminance value is smaller than the seventh preset threshold, and the ratio of the target pixel to all pixels of the target picture is larger than the eighth preset threshold.

[0213] The average luminance of the target pixel is assumed to be 250 nits.

[0214] In an example, the first coordinate value of the 12th interpolation point or the second coordinate value of the 12th interpolation point may be determined to be 250 / (500-100), i.e., 3 / 8, or the first coordinate value of the 12th interpolation point or the second coordinate value of the 12th interpolation point may be determined to be a value close to 3 / 8. The 12th interpolation point is, for example, point 10' shown in Fig. 4. The second coordinate value of the 12th interpolation point may be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0215] In an example, the plurality of interpolation points may be determined to include a twelfth interpolation point and a thirteenth interpolation point, where a first coordinate value of the twelfth interpolation point is at least 3 / 8 and a first coordinate value of the thirteenth interpolation point is greater than 3 / 8, or a second coordinate value of the twelfth interpolation point is less than 3 / 8 and a second coordinate value of the thirteenth interpolation point is greater than 3 / 8. The twelfth interpolation point may be, for example, point 10' shown in Figure 4. The thirteenth interpolation point may be, for example, point 11' shown in Figure 4. The second coordinate values ​​of the twelfth interpolation point and the thirteenth interpolation point may be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0216] This means that when the target picture contains a sufficient number of target pixels, the display device may not be able to enhance the luminance contrast between the target pixels. To display the target pixels clearly, the initial tone mapping curve may be modified within the luminance range corresponding to the target pixels.

[0217] Optionally, the dynamic metadata includes coordinate information for the target pixel.

[0218] In other words, the target pixel may be a pixel indicated in the dynamic metadata, or may be determined autonomously by the electronic device based on the luminance distribution of the pixel. In other words, whether the target picture contains a target pixel may be determined in different manners.

[0219] Then, the modification range of the initial tone mapping curve needs to be determined. Specifically, the 13th interpolation point can be determined, or the range between the 12th and 13th interpolation points can be determined, so that the initial tone mapping curve between the 12th and 13th interpolation points is modified, and the display effect of the picture on the display device is ensured. As shown in Fig. 4, the 13th interpolation point can be point 11' in Fig. 4.

[0220] Optionally, the plurality of interpolation points further includes a thirteenth interpolation point, and determining the plurality of interpolation points based on luminance information of the target picture includes determining the thirteenth interpolation point based on a number of target pixels.

[0221] When the number of target pixels is relatively large or the ratio of the target pixels to all pixels is relatively high, the distance between the twelfth and thirteenth interpolation points is relatively short. When the number of target pixels is relatively small or the ratio of the target pixels to all pixels is relatively low, the distance between the twelfth and thirteenth interpolation points is relatively long. In other words, when the number of target pixels is relatively large, the modification range of the initial tone mapping curve is relatively small, but when the number of target pixels is relatively small, the modification range of the initial tone mapping curve is relatively large. The second coordinate value of the thirteenth interpolation point can be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0222] In addition, the modified tone mapping curve needs to be smooth and differentiable. Therefore, a fourteenth interpolation point can be determined so that the spline function for modifying the initial tone mapping curve can be smoothly connected to the initial tone mapping curve at the fourteenth interpolation point.

[0223] Optionally, the plurality of interpolation points further includes a fourteenth interpolation point, the fourteenth interpolation point being a smooth transition point between the spline function and the initial tone mapping curve, the target picture further includes a target transition pixel, the target transition pixel having a luminance value less than or equal to a first luminance value, or the target transition pixel having a luminance value greater than or equal to a second luminance value and the first luminance value less than the second luminance value, and determining the plurality of interpolation points based on the luminance information of the target picture includes determining a first coordinate value of the fourteenth interpolation point based on the number of the target transition pixels.

[0224] In the example, the target transition pixel may be a pixel having a higher luminance than the target pixel. In this case, the first coordinate value of the 14th interpolation point is greater than the first coordinate value of the 12th interpolation point, and greater than the first coordinate value of the 13th interpolation point. As shown in FIG. 4, the 14th interpolation point may be point 12' in FIG. 4.

[0225] In the example, the target transition pixel may be a pixel having a lower luminance than the target pixel. In this case, the first coordinate value of the 14th interpolation point is less than the first coordinate value of the 12th interpolation point and less than the first coordinate value of the 13th interpolation point. As shown in FIG. 4, the 14th interpolation point may be point 9' in FIG.

[0226] When the number of target transition pixels is relatively large or the ratio of the target transition pixels to all pixels is relatively high, the distance between the thirteenth interpolation point and the fourteenth interpolation point is relatively short. When the number of target transition pixels is relatively small or the ratio of the target transition pixels to all pixels is relatively low, the distance between the thirteenth interpolation point and the fourteenth interpolation point is relatively long. In other words, when the number of target transition pixels is relatively large, the modification range of the initial tone mapping curve is relatively small, but when the number of target pixels is relatively small, the modification range of the initial tone mapping curve is relatively large. The second coordinate value of the fourteenth interpolation point can be obtained based on an empirical value, a user input value, or an initial tone mapping curve.

[0227] Optionally, the dynamic metadata includes coordinate information of the object transition pixel.

[0228] In other words, the target transition pixel may be a pixel indicated in the dynamic metadata, or may be determined autonomously by the electronic device based on the luminance distribution of the pixel. In other words, whether the target picture includes a target transition pixel may be determined in different manners.

[0229] Optionally, the dynamic metadata includes coordinate information for each of the plurality of interpolation points.

[0230] The coordinate information of the interpolation point may be a coordinate of the interpolation point, or may be a first coordinate value or a second coordinate value of the interpolation point. The coordinate of the interpolation point may be expressed as (x, y), where x may represent the first coordinate value and y may represent the second coordinate value. The electronic device may determine the multiple interpolation points based on the luminance information of the image, or based on the dynamic metadata of the image, or based on the luminance information of the image and the dynamic metadata of the image. For example, the electronic device determines a first coordinate value of the interpolation point 1, a second coordinate value of the interpolation point 2, and a coordinate of the interpolation point 3.

[0231] For example, the dynamic metadata includes a first coordinate value of point 1', a coordinate of point 2', and a second coordinate value of point 4' in FIG. 4. Based on the first coordinate value of point 1' given in the dynamic metadata, it can be determined that point 1' may be a point that coincides with point 1 or coincides with the origin. In this way, the coordinate of point 1' is recognized. In addition, the coordinate of point 2' is given in the dynamic metadata. Furthermore, based on the second coordinate value of point 4' given in the dynamic metadata, a point whose vertical coordinate is the same as the second coordinate value can be found on the initial tone mapping function, and the horizontal coordinate of the point is determined to be the first coordinate value of point 4'. In this way, the coordinate of point 4' is recognized. A general method for determining a spline function can be used to obtain a spline function that passes through point 1', point 2', and point 4'.

[0232] In another example, the dynamic metadata includes a first coordinate value of point 5', a coordinate of point 6', and a second coordinate value of point 8' in FIG. 4. Based on the first coordinate value of point 5' given in the dynamic metadata, it can be determined that point 5' is coincident with point 5 or may be a point coincident with (1, 1). In this way, the coordinate of point 5' is recognized. In addition, the coordinate of point 6' is given in the dynamic metadata. Furthermore, based on the second coordinate value of point 8' given in the dynamic metadata, the first coordinate value of point 8' can be determined on the initial tone mapping function, and the coordinate of point 8' can be recognized. A general method for determining a spline function can be used to obtain a spline function that passes through points 5', 6', and 8'.

[0233] In another example, the dynamic metadata includes the second coordinate value of point 9', the coordinate of point 10', and the first coordinate value of point 12' in FIG. 4. Based on the second coordinate value of point 9' given in the dynamic metadata, the first coordinate value of point 12' can be determined on the initial tone mapping function, and the coordinate of point 12' can be recognized. In addition, the coordinate of point 10' is given in the dynamic metadata. Furthermore, based on the first coordinate value of point 12' given in the dynamic metadata, the second coordinate value of point 12' can be determined on the initial tone mapping function, and the coordinate of point 12' can be recognized. A general method for determining a spline function can be used to obtain a spline function that passes through points 9', 10', and 12'.

[0234] 203: A spline function that passes through the plurality of interpolation points is determined, where the spline function is smoothly connected to the initial tone mapping curve at at least one of the plurality of interpolation points.

[0235] A spline function is a piecewise polynomial function. The spline function S(x) in the interval [a, b] is defined as being continuous in the interval [a, b], and both the first derivative S'(x) and the second derivative S''(x) of the spline function are continuous in the interval [a, b]. Figure 4 shows spline function 1 passing through points 1', 2', and 4', spline function 2 passing through points 5', 6', and 8', and spline function 3 passing through points 9', 10', and 12'.

[0236] After a plurality of interpolation points are determined, the spline function passing through the plurality of interpolation points can be obtained based on the coordinates of each of the plurality of interpolation points. For spline function 1 in Figure 4, the plurality of interpolation points are points 1', 2', and 4'. For spline function 2 in Figure 4, the plurality of interpolation points are points 5', 6', and 8'. For spline function 3 in Figure 4, the plurality of interpolation points are points 9', 10', and 12'.

[0237] The interval [a, b] includes n nodes x1,..., x n where a < x1 <... x n . When the spline function is an Nth-degree spline function, the function in the interval [x k , x k+1 between any two adjacent nodes is an Nth-degree polynomial function, k < n - 1, and k is a positive integer.

[0238] When the spline function includes spline function 1 and spline function 2, spline function 1 can be an Mth-degree spline function in the interval [x M1 , x M2 , and spline function 2 can be an Lth-degree spline function in the interval [x L1 , x L2 . The common part of the interval [x M1 , x M2 and the interval [x L1 , x L2 is empty, or the interval [xM1 ,x M2 ] and the interval [x L1 ,x L2 ] contains only one value. M may be the same as or different from L.

[0239] When the spline function includes a linear spline function, at least two known conditions are required to determine the linear spline function. For example, the coordinates of a point passing through the linear spline function and the slope of the spline function are known in advance. In another example, the coordinates of two points passing through the linear spline function are known in advance.

[0240] When the spline function includes a quadratic spline, at least three known conditions are required to determine the quadratic spline. For example, the functional form of a quadratic spline is y=ax 2 +bx+c, a≠0. To determine the quadratic spline function, the coordinates of three points passing through the quadratic spline function need to be known in advance. Alternatively, the coordinates of two points and the derivative of the quadratic spline function at one point are known in advance.

[0241] When the spline function includes a cubic spline function, at least four known conditions are required to determine the cubic spline function. For example, the functional form of a cubic spline function is y=ax 3 +bx 2 +cx+d, a≠0. To determine the cubic spline function, the coordinates of four points passing through the cubic spline function need to be known in advance. Alternatively, the coordinates of two points and the derivative of the cubic spline function at the two points are known in advance.

[0242] When the spline function includes an Nth order spline function, at least N+1 known conditions are required to determine the Nth order spline function.

[0243] Assume that the initial tone mapping curve is T(x) and the spline function S(x) is defined in the interval [a,b].

[0244] In an example, the spline function S(x) can be smoothly connected to the initial tone mapping curve T(x) when x = a, or can be smoothly connected to the initial tone mapping curve T(x) when x = b, or can be smoothly connected to the initial tone mapping curve T(x) when x = a and when x = b.

[0245] In an example, the plurality of interpolation points includes a point whose first coordinate value is x0, where a < x0 < b, and the spline function S(x) can be smoothly connected to the initial tone mapping curve T(x) when x = x0.

[0246] The spline function S(x) is smoothly connected to the initial tone mapping curve T(x) when x = a, that is, when S(a) = T(a) and S'(a) = T'(a).

[0247] The spline function S(x) is smoothly connected to the initial tone mapping curve T(x) when x = b, that is, when S(b) = T(b) and S'(b) = T'(b).

[0248] The spline function S(x) is smoothly connected to the initial tone mapping curve T(x) when x = x0, that is, when S(x0) = T(x0) and S'(x0) = T'(x0).

[0249] The spline function is smoothly connected to the initial tone mapping curve at the interpolation points. The purpose of the smooth connection is to replace a part of the initial tone mapping curve with the spline function, and as a result, a modified tone mapping curve can be obtained. Since the spline function is smoothly connected to the initial tone mapping curve at the interpolation points, the modified tone mapping curve can be a smooth function.

[0250] Optionally, the plurality of interpolation points includes a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, and the initial tone mapping curve passes through the ninth interpolation point, and the step of determining a spline function passing through the plurality of interpolation points includes the steps of obtaining a first spline function that passes through the seventh interpolation point and the eighth interpolation point, and determining a second spline function that passes through the eighth interpolation point and the ninth interpolation point, wherein the second spline function is smoothly connected to the first spline function at the eighth interpolation point, and the second spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point.

[0251] In the above-mentioned scenarios 1, 2 and 3, the spline function may include a first spline function and a second spline function, where the first spline function is a strong modified spline function and the second spline function is used to implement a smooth transition between the first spline function and the initial luminance mapping curve.

[0252] Obtaining the first spline function may be determining the first spline function by receiving parameter input by a user or by using preset parameters, for example, the first spline function is preset as a linear spline function, and the slope of the linear spline function is 1.

[0253] Optionally, the first spline function may further be determined with reference to the coordinates of the seventh and / or eighth interpolation point.

[0254] The first spline function may alternatively be a quadratic or cubic spline function. The first spline function is determined with reference to the above-mentioned conditions necessary for determining a quadratic or cubic spline function.

[0255] In other words, when the first spline function is determined, in addition to the acquired data, some empirical values ​​or user indications may be introduced to determine the first spline function.

[0256] Then, a second spline function is determined, which is used to smoothly connect the first spline function to the initial tone mapping curve. The first spline function is determined before the second spline function is determined. Thus, the function value of the first spline function at the eighth interpolation point is known, and the derivative value of the first spline function at the eighth interpolation point is known. The second spline function smoothly connects to the first spline function at the eighth interpolation point means that the function value of the first spline function at the eighth interpolation point is the same as the function value of the second spline function at the eighth interpolation point, and the derivative value of the first spline function at the eighth interpolation point is the same as the derivative value of the second spline function at the eighth interpolation point. In addition, the function value of the initial tone mapping function at the ninth interpolation point is known, and the derivative value of the initial tone mapping function at the ninth interpolation point is known. The second spline function being smoothly connected to the initial tone mapping curve at the 9th interpolation point means that the function value of the second spline function at the 9th interpolation point is identical to the function value of the initial tone mapping function at the 9th interpolation point, and the derivative value of the second spline function at the 9th interpolation point is identical to the derivative value of the initial tone mapping function at the 9th interpolation point.

[0257] For ease of explanation, in this application, unless otherwise specified, a "derivative value" may be understood as a first derivative value.

[0258] Optionally, the first spline function is a linear spline function and the second spline function is a cubic spline function.

[0259] Optionally, the derivative of the spline function is always greater than zero.

[0260] In other words, the derivative of the spline function at every point is always greater than 0. This restriction can avoid excessively large differences between the final spline function obtained and the initial tone mapping curve, thus avoiding excessive corrections.

[0261] Arbitrarily, the derivative of the first spline function is always greater than zero.

[0262] In other words, the derivative of the first spline function at every point is always greater than 0. This restriction can avoid an excessively large difference between the finally obtained first spline function and the initial tone mapping curve, and avoids excessive correction.

[0263] Optionally, the derivative of the second spline function is always greater than zero.

[0264] In other words, the derivative of the second spline function at every point is always greater than 0. This restriction can avoid excessively large differences between the final obtained second spline function and the initial tone mapping curve, and avoids excessive corrections.

[0265] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the first spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0266] The points where the second derivative is 0 may be inflection points of the first spline function. For example, the first spline function increases quickly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point.

[0267] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the second spline function passes through the eleventh interpolation point, and a second derivative of the second spline function at the eleventh interpolation point is zero.

[0268] The point where the second derivative is zero may be an inflection point of the second spline function. For example, the second spline function increases quickly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. As shown in FIG. 4, the eleventh interpolation point may be point 3'.

[0269] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, the first spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0270] On one side of the 15th interpolation point, the first spline function is located above the initial tone mapping function, and on the other side of the 15th interpolation point, the first spline function is located below the initial tone mapping function. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0271] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, the second spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0272] On one side of the 15th interpolation point, the second spline function is placed above the initial tone mapping function, and on the other side of the 15th interpolation point, the second spline function is placed below the initial tone mapping function. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0273] Scenario 1

[0274] When the image is dark, a portion of the initial tone mapping curve close to the first end point may be modified. The seventh interpolation point may be the first interpolation point mentioned above, the eighth interpolation point may be the second interpolation point mentioned above, and the ninth interpolation point may be the third interpolation point mentioned above. In this scenario, the spline function includes a first spline function passing through the first and second interpolation points, and a second spline function passing through the second and third interpolation points, where the first spline function is a strongly modified spline function, and the second spline function is used to implement a smooth transition between the first spline function and the initial luminance mapping curve. The locations of the specific interpolation points and spline functions are shown in FIG. 5.

[0275] The above describes how to obtain the first coordinate values ​​of the first, second, and third interpolation points. In other words, the first coordinate value of the seventh, eighth, and ninth interpolation points can all be obtained.

[0276] First, a first spline function f1(x) that passes through the seventh and eighth interpolation points (ie, passes through points 1' and 2' in FIG. 4) is obtained.

[0277] For example, the first spline function is preset as a linear spline function, and the slope of the linear spline function is 1. Since the first coordinate value of the seventh interpolation point is identical to the first coordinate value of the first end point of the initial tone mapping function, the first spline function may be determined to be a function that passes through the origin, i.e., the coordinates of the seventh interpolation point are (0, 0). Therefore, the first spline function may be expressed as f1(x)=x.

[0278] Then, a second spline function f2(x) that passes through the eighth and ninth interpolation points (in this embodiment, the eighth interpolation point is point 2' in Figure 4, and the ninth interpolation point is point 4' in Figure 4) is determined.

[0279] Based on the first spline function f1(x) and the first coordinate value x2 of the eighth interpolation point, it can be determined that the function of the first spline function at the eighth interpolation point is f1(x2) and the derivative of the first spline function at the eighth interpolation point is f1'(x2).

[0280] Based on the initial tone mapping function f0(x) and the first coordinate value x4 of the 9th interpolation point, it can be determined that the function of the second spline function at the 9th interpolation point is f0(x4) and the derivative of the first spline function at the 9th interpolation point is f0'(x4).

[0281] A second spline function f2(x) is determined based on f1(x2)=f2(x2), f1'(x2)=f2'(x2), f0(x4)=f2(x4), and f0'(x4)=f2'(x4).

[0282] Optionally, the second spline function is a cubic spline function.

[0283] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the first spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0284] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, where the second spline function passes through the eleventh interpolation point, and a second derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be point 3′ shown in FIG.

[0285] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the first spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the first spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0286] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, where the second spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point. A derivative value of the second spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be point 3' shown in FIG.

[0287] Scenario 2

[0288] When the image is bright, a portion of the initial tone mapping curve close to the second end point may be modified. In this case, the seventh interpolation point may be the fourth interpolation point mentioned above, the eighth interpolation point may be the fifth interpolation point mentioned above, and the ninth interpolation point may be the sixth interpolation point mentioned above. In this scenario, the spline function includes a first spline function passing through the fourth and fifth interpolation points, and a second spline function passing through the fifth and sixth interpolation points, where the first spline function is a strong modified spline function, and the second spline function is used to implement a smooth transition between the first spline function and the initial brightness mapping curve. The locations of the specific interpolation points and the spline function are shown in FIG. 6.

[0289] The above describes how to obtain the first coordinate values ​​of the fourth, fifth, and sixth interpolation points. In other words, the first coordinate value of the seventh, eighth, and ninth interpolation points are all known.

[0290] First, a first spline function f1(x) that passes through the seventh and eighth interpolation points (ie, passes through points 5' and 6' in FIG. 4) is obtained.

[0291] For example, the first spline function is preset as a linear spline function, and the slope of the linear spline function is 1. Since the first coordinate value of the seventh interpolation point is identical to the first coordinate value of the first end point of the initial tone mapping function, the first spline function may be determined to be a function that passes through (1,1), i.e., the coordinate of the seventh interpolation point is (1,1). Therefore, the first spline function may be expressed as f1(x)=x.

[0292] Then, a second spline function f2(x) that passes through the eighth and ninth interpolation points (ie, passes through points 6' and 8' in FIG. 4) is determined.

[0293] Based on the first spline function f1(x) and the first coordinate value x6 of the eighth interpolation point, it can be determined that the function of the first spline function at the eighth interpolation point is f1(x6) and the derivative of the first spline function at the eighth interpolation point is f1'(x6).

[0294] Based on the initial tone mapping function f0(x) and the first coordinate value x8 of the 9th interpolation point, it can be determined that the function of the second spline function at the 9th interpolation point is f0(x8) and the derivative of the first spline function at the 9th interpolation point is f0'(x8).

[0295] A second spline function f2(x) is determined based on f1(x6)=f2(x6), f1'(x6)=f2'(x6), f0(x8)=f2(x8), and f0'(x8)=f2'(x8).

[0296] Optionally, the second spline function is a cubic spline function.

[0297] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the first spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0298] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, where the second spline function passes through the eleventh interpolation point, and a second derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be point 7′ shown in FIG.

[0299] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the first spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the first spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0300] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, where the second spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point. A derivative value of the second spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be point 7' shown in FIG.

[0301] Scenario 3

[0302] The seventh interpolation point may be the twelfth interpolation point mentioned above, the eighth interpolation point may be the thirteenth interpolation point mentioned above, and the ninth interpolation point may be the fourteenth interpolation point mentioned above. In this scenario, the spline function includes a first spline function that passes through the twelfth and thirteenth interpolation points, and a second spline function that passes through the thirteenth and fourteenth interpolation points, where the first spline function is a strong modified spline function, and the second spline function is used to implement a smooth transition between the first spline function and the initial luminance mapping curve. The locations of the particular interpolation points and the spline functions are shown in FIG. 8.

[0303] The above describes how to obtain the first coordinate values ​​of the 12th, 13th, and 14th interpolation points. In other words, the first coordinate value of the 7th, 8th, and 9th interpolation points are all known.

[0304] First, a first spline function f1(x) that passes through the seventh and eighth interpolation points (ie, passes through points 10' and 11' in FIG. 4) is obtained.

[0305] For example, the first spline function is preset as a linear spline function, and the slope of the linear spline function is 1. The seventh interpolation point is located below the initial tone mapping function, and the eighth interpolation point is located above the initial tone mapping function, so the first coordinate value of the seventh interpolation point is x 10 The first coordinate value of the eighth interpolation point is x 11 and the first spline function is

number

number

[0306] Then, a second spline function f2(x) that passes through the eighth and ninth interpolation points (i.e., passing through points 11' and 12' in FIG. 4, or passing through points 10' and 9' in FIG. 4) is determined.

[0307] The first spline function f1(x) and the first coordinate value x of the eighth interpolation point 11 Based on this, the function of the first spline function at the 8th interpolation point is f1(x 11 ), and the derivative of the first spline function at the eighth interpolation point is f1'(x 11 ) can be determined.

[0308] The initial tone mapping function f0(x) and the first coordinate value x of the 9th interpolation point 12 Based on this, the function of the second spline function at the 9th interpolation point is f0(x 12 ), and the derivative of the first spline function at the 9th interpolation point is f0'(x 12 ) can be determined.

[0309] f1(x 11 )=f2(x 11 ), f1'(x 11 )=f2'(x 11 ), f0(x 12 )=f2(x 12 ), and f0'(x 12 )=f2'(x 12 ), the second spline function f2(x) is determined.

[0310] Optionally, the second spline function is a cubic spline function.

[0311] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the first spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0312] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, where the second spline function passes through the eleventh interpolation point, and where a second derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be point 13′ shown in FIG.

[0313] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the first spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the first spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0314] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, where the second spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point. A derivative value of the second spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be point 13′ shown in FIG.

[0315] Optionally, the plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, a first coordinate value of the eighth interpolation point is between a first coordinate value of the seventh interpolation point and a first coordinate value of the ninth interpolation point, the first coordinate value of the eighth interpolation point being a first modified position, and before the step of determining the plurality of interpolation points, the method further includes the step of obtaining a first offset, the first offset being used to represent a modified range of the initial tone mapping curve at the first modified position, The position is a first coordinate value of an eighth interpolation point, the determining the plurality of interpolation points includes determining a second coordinate value of the eighth interpolation point based on a function value of the initial tone mapping curve at the first modified position and the first offset, and the determining the spline function passing through the plurality of interpolation points includes determining a third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, the third spline function being smoothly connected to the initial tone mapping curve at the seventh and / or ninth interpolation points. Obtaining the first offset may be receiving a parameter input by a user or may be using a preset parameter. For example, the first offset is preset to 0.01.

[0316] The first coordinate value of the eighth interpolation point corresponds to the first modified position. Specifically, the eighth interpolation point that deviates from the initial tone mapping function is determined at the first modified position, i.e., the second coordinate value of the eighth interpolation point is not equal to the function value of the initial tone mapping function at the first modified position.

[0317] The first offset may be an offset degree of the eighth interpolation point relative to the initial tone mapping function. When the first offset is positive, the eighth interpolation point is located above the initial tone mapping function. When the first offset is negative, the eighth interpolation point is located below the initial tone mapping function.

[0318] For example, the second coordinate value of the eighth interpolation point=the first offset+the function value of the initial tone mapping function at the first modified position.

[0319] For example, the shortest distance from the 8th interpolation point to the initial tone mapping function = the 1st offset.

[0320] Thus, the coordinates of the eighth interpolation point may be determined based on the first offset.

[0321] Then, the third spline function is determined.

[0322] In the example, the third spline function is smoothly connected to the initial tone mapping curve at the seventh and ninth interpolation points. Both the seventh and ninth interpolation points can be regarded as points on the initial tone mapping function. Thus, the coordinates of the seventh and ninth interpolation points can be obtained. In addition, the derivative of the third spline function at the seventh interpolation point is equal to the derivative of the initial tone mapping function at the seventh interpolation point, and the derivative of the third spline function at the ninth interpolation point is equal to the derivative of the initial tone mapping function at the ninth interpolation point. Due to the attributes of the spline function itself, the third spline function at the eighth interpolation point is continuous, the first derivative of the third spline function at the eighth interpolation point is continuous, and the second derivative of the third spline function at the eighth interpolation point is continuous. Thus, the third spline function can be divided into a fourth spline function that passes through the seventh and eighth interpolation points, and a fifth spline function that passes through the eighth and ninth interpolation points. The function value of the fourth spline function at the eighth interpolation point is the same as the function value of the fifth spline function at the eighth interpolation point. The first derivative value of the fourth spline function at the eighth interpolation point is the same as the first derivative value of the fifth spline function at the eighth interpolation point. The second derivative value of the fourth spline function at the eighth interpolation point is the same as the second derivative value of the fifth spline function at the eighth interpolation point.

[0323] In the above-mentioned scenarios 1, 2, and 3, the spline functions that pass through the multiple interpolation points are collectively referred to as the third spline function, and the multiple interpolation points in the three scenarios include the seventh, eighth, and ninth interpolation points. The spline function that passes through the seventh and eighth interpolation points is the fourth spline function, and the spline function that passes through the eighth and ninth interpolation points is the fifth spline function.

[0324] Optionally, the derivative of the 4th spline function is always greater than zero.

[0325] In other words, the derivative of the fourth spline function at every point is always greater than 0. This restriction can avoid an excessively large difference between the finally obtained first spline function and the initial tone mapping curve, thus avoiding excessive correction.

[0326] Optionally, the derivative of the fifth spline function is always greater than zero.

[0327] In other words, the derivative of the fifth spline function at every point is always greater than 0. This restriction can avoid an excessively large difference between the final obtained second spline function and the initial tone mapping curve, thus avoiding excessive correction.

[0328] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the fourth spline function passes through the eleventh interpolation point, and a second derivative of the fourth spline function at the eleventh interpolation point is zero.

[0329] The point where the second derivative is zero may be an inflection point of the fourth spline function. For example, the fourth spline function increases quickly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. As shown in FIG. 4, the eleventh interpolation point may be point 3'.

[0330] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the fifth spline function passes through the eleventh interpolation point, and a second derivative of the second spline function at the eleventh interpolation point is zero.

[0331] The point where the second derivative is zero may be an inflection point of the fifth spline function. For example, the fifth spline function increases quickly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. As shown in FIG. 4, the eleventh interpolation point may be point 3'.

[0332] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, the fourth spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0333] On one side of the 15th interpolation point, the fourth spline function is located above the initial tone mapping function, and on the other side of the 15th interpolation point, the fourth spline function is located below the initial tone mapping function. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0334] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, the fifth spline function passes through the fifteenth interpolation point, and the initial tone mapping function passes through the fifteenth interpolation point.

[0335] On one side of the 15th interpolation point, the fifth spline function is located above the initial tone mapping function, and on the other side of the 15th interpolation point, the fifth spline function is located below the initial tone mapping function. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0336] Optionally, the plurality of interpolation points further includes a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point, and before the step of determining the plurality of interpolation points, the method further comprises: obtaining a second offset, the second offset being used to represent a modification range of the initial tone mapping curve at a second modified position, the second modified position being the first coordinate value of the tenth interpolation point, and the step of determining the plurality of interpolation points further comprises: obtaining a second offset at the second modified position. The step of determining a second coordinate value of the tenth interpolation point based on a function value of the initial tone mapping curve and a second offset, and the step of determining a third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point includes a step of determining a third spline function based on the first and second coordinate values ​​of the eighth interpolation point and the first and second coordinate values ​​of the tenth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at both the seventh and ninth interpolation points.

[0337] Obtaining the second offset may be receiving a parameter input by a user, or may be using a preset parameter, for example, the second offset is preset to 0.01.

[0338] The first coordinate value of the tenth interpolation point corresponds to the second modified position. Specifically, the tenth interpolation point that deviates from the initial tone mapping function is determined at the second modified position, i.e., the second coordinate value of the tenth interpolation point is not equal to the function value of the initial tone mapping function at the second modified position.

[0339] The second offset may be a degree of offset of the tenth interpolation point relative to the initial tone mapping function. When the second offset is positive, the tenth interpolation point is located above the initial tone mapping function. When the second offset is negative, the tenth interpolation point is located below the initial tone mapping function.

[0340] For example, the second coordinate value of the tenth interpolation point=the second offset+the function value of the initial tone mapping function at the second modified position.

[0341] For example, the shortest distance from the 10th interpolation point to the initial tone mapping function = 2nd offset.

[0342] Optionally, the first offset and the second offset are mutually inverse numbers.

[0343] Thus, the coordinates of the tenth interpolation point may be determined based on the second offset.

[0344] Then, a third spline function is further determined. Due to the attributes of the spline function itself, the third spline function at the tenth interpolation point is continuous, the first derivative of the third spline function at the tenth interpolation point is continuous, and the second derivative of the third spline function at the tenth interpolation point is continuous. Therefore, the third spline function can be divided into a sixth spline function passing through the seventh and tenth interpolation points, and a seventh spline function passing through the tenth and ninth interpolation points. The function value of the sixth spline function at the tenth interpolation point is identical to the function value of the seventh spline function at the tenth interpolation point. The first derivative value of the sixth spline function at the tenth interpolation point is identical to the first derivative value of the seventh spline function at the tenth interpolation point. The second derivative value of the sixth spline function at the tenth interpolation point is identical to the second derivative value of the seventh spline function at the tenth interpolation point.

[0345] Arbitrarily, the derivative of the 6th spline function is always greater than zero.

[0346] In other words, the derivative of the sixth spline function at every point is always greater than 0. This restriction can avoid an excessively large difference between the finally obtained first spline function and the initial tone mapping curve, thus avoiding excessive correction.

[0347] Optionally, the derivative of the 7th spline function is always greater than zero.

[0348] In other words, the derivative of the seventh spline function at every point is always greater than 0. This restriction can avoid an excessively large difference between the final obtained second spline function and the initial tone mapping curve, thus avoiding excessive correction.

[0349] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the sixth spline function passes through the eleventh interpolation point, and a second derivative of the second spline function at the eleventh interpolation point is zero.

[0350] The point where the second derivative is zero may be an inflection point of the sixth spline function. For example, the sixth spline function increases quickly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. As shown in FIG. 4, the eleventh interpolation point may be point 3'.

[0351] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the seventh spline function passes through the eleventh interpolation point, and a second derivative of the second spline function at the eleventh interpolation point is zero.

[0352] The point where the second derivative is zero may be an inflection point of the seventh spline function. For example, the seventh spline function increases quickly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point. As shown in FIG. 4, the eleventh interpolation point may be point 3'.

[0353] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the fourth spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the fourth spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0354] On one side of the 15th interpolation point, the fourth spline function is located above the initial tone mapping function, and on the other side of the 15th interpolation point, the fourth spline function is located below the initial tone mapping function. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0355] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the fifth spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the fifth spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0356] On one side of the 15th interpolation point, the fifth spline function is located above the initial tone mapping function, and on the other side of the 15th interpolation point, the fifth spline function is located below the initial tone mapping function. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0357] In the example, the third spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point. The ninth interpolation point can be regarded as a point on the initial tone mapping function. Thus, the coordinates of the ninth interpolation point can be obtained. In addition, the derivative of the third spline function at the ninth interpolation point is equal to the derivative of the initial tone mapping function at the ninth interpolation point. Due to the attributes of the spline function itself, the first derivative of the third spline function at the eighth interpolation point is continuous, and the second derivative of the third spline function at the eighth interpolation point is continuous. Thus, the third spline function can be divided into a fourth spline function passing through the seventh and eighth interpolation points, and a fifth spline function passing through the eighth and ninth interpolation points. The first derivative value of the fourth spline function at the eighth interpolation point is identical to the first derivative value of the fifth spline function at the eighth interpolation point. The second derivative value of the fourth spline function at the eighth interpolation point is identical to the second derivative value of the fifth spline function at the eighth interpolation point. Therefore, the coordinates of the seventh interpolation point still need to be determined.

[0358] The manner of determining the coordinates of the seventh interpolation point may be determined based on experience, for example, the coordinates of the seventh interpolation point may be determined to be (0,0) or (1,1).

[0359] When the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point, it is the same as when the third spline function is smoothly connected to the initial tone mapping curve at the ninth interpolation point, and the details are not described here.

[0360] Optionally, the derivative of the third spline function is always greater than zero.

[0361] In other words, the derivative of the spline function at every point is always greater than 0. This restriction can avoid excessively large differences between the final spline function obtained and the initial tone mapping curve, thus avoiding excessive corrections.

[0362] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the third spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0363] The points where the second derivative is 0 may be inflection points of the first spline function. For example, the third spline function increases quickly on one side of the eleventh interpolation point and increases slowly on the other side of the eleventh interpolation point.

[0364] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the third spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the third spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0365] On one side of the 15th interpolation point, the third spline function is located above the initial tone mapping function, and on the other side of the 15th interpolation point, the third spline function is located below the initial tone mapping function. By defining the location of the 15th interpolation point, a modification manner of the initial tone mapping function can be defined.

[0366] Scenario 1

[0367] When the image is dark, a portion of the initial tone mapping curve close to the first end point may be modified. The seventh interpolation point may be the first interpolation point mentioned above, the eighth interpolation point may be the second interpolation point mentioned above, and the ninth interpolation point may be the third interpolation point mentioned above. The locations of the specific interpolation points and the spline function are shown in FIG.

[0368] The above describes how to obtain the first coordinate values ​​of the first, second, and third interpolation points. In other words, the first coordinate value of the seventh, eighth, and ninth interpolation points can all be obtained.

[0369] First, a first offset is obtained, for example, the first offset is preset to 0.01.

[0370] And the second coordinate value of the eighth interpolation point (point 2' in FIG. 4) is determined based on the first offset, for example, f3(x2)=f0(x2)+0.01, where f0(x) represents the initial tone mapping function.

[0371] Since the first coordinate value of the seventh interpolation point is identical to the first coordinate value of the first endpoint of the initial tone mapping function, it can be determined that the third spline function is a function that passes through the origin, i.e., the coordinates of the seventh interpolation point are (0,0).

[0372] Then, a third spline function f3(x) that passes through the seventh, eighth, and ninth interpolation points (i.e., passing through points 1', 2', and 4' in FIG. 4) is determined. The third spline function f3(x) can be divided into a fourth spline function f4(x) that passes through the seventh and eighth interpolation points, and a fifth spline function f5(x) that passes through the eighth and ninth interpolation points.

[0373] f4(x1)=0, f4(x2)=f0(x2)+0.01, and f5(x2)=f0(x2)+0.01 can be determined based on the coordinates of the seventh interpolation point and the coordinates of the eighth interpolation point.

[0374] Based on the initial tone mapping function f0(x) and the first coordinate value x4 of the 9th interpolation point, it can be determined that the function of the 5th spline function at the 9th interpolation point is f5(x4) = f0(x4), and the derivative of the 5th spline function at the 9th interpolation point is f5'(x4) = f0'(x4).

[0375] Based on the attributes of the spline functions, it can be determined that the first derivative of the fourth spline function at the eighth interpolation point is identical to the first derivative of the fifth spline function at the eighth interpolation point, and that the second derivative of the fourth spline function at the eighth interpolation point is identical to the second derivative of the fifth spline function at the eighth interpolation point, i.e., f5'(x2)=f4'(x2) and f5"(x2)=f4"(x2).

[0376] Optionally, the third spline function is a cubic spline function. Specifically, the fourth and fifth spline functions are cubic spline functions.

[0377] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the fourth spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0378] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, where the fifth spline function passes through the eleventh interpolation point, and where a second derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be point 3′ shown in FIG.

[0379] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the first spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the fourth spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0380] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, where the fifth spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point. A derivative value of the fifth spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be point 3' shown in FIG.

[0381] Scenario 2

[0382] When the image is bright, a portion of the initial tone mapping curve close to the second end point may be modified. The seventh interpolation point may be the fourth interpolation point mentioned above, the eighth interpolation point may be the fifth interpolation point mentioned above, and the ninth interpolation point may be the sixth interpolation point mentioned above. The locations of the specific interpolation points and the spline function are shown in FIG. 6.

[0383] The above describes how to obtain the first coordinate values ​​of the fourth, fifth, and sixth interpolation points. In other words, the first coordinate value of the seventh, eighth, and ninth interpolation points are all known.

[0384] First, the first offset is obtained, for example, the first offset is preset to -0.01.

[0385] And the second coordinate value of the eighth interpolation point (point 6' in FIG. 4) is determined based on the first offset, for example, f3(x6)=f0(x6)-0.01, where f0(x) represents the initial tone mapping function.

[0386] Since the first coordinate value of the seventh interpolation point is identical to the first coordinate value of the first end point of the initial tone mapping function, it can be determined that the third spline function is a function that passes through (1,1), i.e., the coordinate of the seventh interpolation point is (1,1).

[0387] Then, a third spline function f3(x) that passes through the seventh, eighth, and ninth interpolation points (i.e., passing through points 5', 6', and 8' in FIG. 4) is determined. The third spline function f3(x) can be divided into a fourth spline function f4(x) that passes through the seventh and eighth interpolation points, and a fifth spline function f5(x) that passes through the eighth and ninth interpolation points.

[0388] f4(x5)=1, f4(x6)=f0(x6)-0.01 and f5(x6)=f0(x6)-0.01 can be determined based on the coordinates of the seventh interpolation point and the coordinates of the eighth interpolation point.

[0389] Based on the initial tone mapping function f0(x) and the first coordinate value x8 of the 9th interpolation point, it can be determined that the function of the 5th spline function at the 9th interpolation point is f5(x8)=f0(x8), and the derivative of the 5th spline function at the 9th interpolation point is f5'(x8)=f0'(x8).

[0390] Based on the attributes of the spline functions, it can be determined that the first derivative of the fourth spline function at the eighth interpolation point is identical to the first derivative of the fifth spline function at the eighth interpolation point, and that the second derivative of the fourth spline function at the eighth interpolation point is identical to the second derivative of the fifth spline function at the eighth interpolation point, i.e., f5'(x6)=f4'(x6) and f5"(x6)=f4"(x6).

[0391] Optionally, the third spline function is a cubic spline function. Specifically, the fourth and fifth spline functions are cubic spline functions.

[0392] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the fourth spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0393] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, where the fifth spline function passes through the eleventh interpolation point, and where a second derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be point 7′ shown in FIG.

[0394] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the first spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the fourth spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0395] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, where the second spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point. A derivative value of the fifth spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be point 7' shown in FIG.

[0396] Scenario 3

[0397] Method 1

[0398] The eighth interpolation point may be the twelfth interpolation point mentioned above. The seventh interpolation point may be the fourteenth interpolation point mentioned above, whose first coordinate value is less than the first coordinate value of the twelfth interpolation point and less than the first coordinate value of the thirteenth interpolation point mentioned above. The ninth interpolation point may be the fourteenth interpolation point mentioned above, whose first coordinate value is greater than the first coordinate value of the twelfth interpolation point and greater than the first coordinate value of the thirteenth interpolation point. In this scenario, the spline function includes a fourth spline function that passes through the twelfth and thirteenth interpolation points, and a fifth spline function that passes through the thirteenth and fourteenth interpolation points. The locations of the particular interpolation points and the spline functions are shown in FIG. 7.

[0399] The above describes how to obtain the first coordinate values ​​of the 12th, 13th, and 14th interpolation points. In other words, the first coordinate value of the 7th, 8th, and 9th interpolation points are all known.

[0400] First, a first offset is obtained, for example, the first offset is preset to 0.01.

[0401] Then, the second coordinate value of the eighth interpolation point (point 10' in FIG. 4) is determined based on the first offset. For example, f3(x 10 )=f0(x 10 )+0.01, and f0(x) represents the initial tone mapping function.

[0402] Since the third spline function is smoothly connected to the initial tone mapping function at the seventh interpolation point, it can be determined that the coordinates of the seventh interpolation point are (x9, f0(x9)), where f3(x9) = f0(x9), and f3'(x9) = f0'(x9).

[0403] The third spline function is smoothly connected to the initial tone mapping function at the ninth interpolation point, so the coordinates of the ninth interpolation point are (x 12 ,f0(x 12 )) and f3(x 12 )=f0(x 12 ), f3'(x 12 )=f0'(x12 ) can be determined.

[0404] Then, a third spline function f3(x) that passes through the seventh, eighth, and ninth interpolation points (i.e., passing through points 9', 10', and 12' in FIG. 4) is determined. The third spline function f3(x) can be divided into a fourth spline function f4(x) that passes through the seventh and eighth interpolation points, and a fifth spline function f5(x) that passes through the eighth and ninth interpolation points.

[0405] f4(x9)=f0(x9), f4'(x9)=f0'(x9), f4(x 10 )=f0(x 10 )+0.01 may be determined based on the initial tone mapping function f0(x), the coordinates of the seventh interpolation point, and the coordinates of the eighth interpolation point.

[0406] f5(x 10 )=f0(x 10 )+0.01, f5(x 12 )=f0(x 12 ), f5'(x 12 )=f0'(x 12 ) may be determined based on the initial tone mapping function f0(x), the coordinates of the eighth interpolation point, and the coordinates of the ninth interpolation point.

[0407] Based on the properties of the spline function, the first derivative of the fourth spline function at the eighth interpolation point is identical to the first derivative of the fifth spline function at the eighth interpolation point, and the second derivative of the fourth spline function at the eighth interpolation point is identical to the second derivative of the fifth spline function at the eighth interpolation point, i.e., f5'(x 10 )=f4'(x 10 ) and f5"(x 10 )=f4"(x 10 ) can be determined.

[0408] Optionally, the third spline function is a cubic spline function. Specifically, the fourth and fifth spline functions are cubic spline functions.

[0409] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the fourth spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0410] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, where the fifth spline function passes through the eleventh interpolation point, and where a second derivative of the second spline function at the eleventh interpolation point is 0. The eleventh interpolation point may be point 11′ shown in FIG.

[0411] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the first spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the fourth spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0412] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, where the second spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point. A derivative value of the fifth spline function at the fifteenth interpolation point is different from a derivative value of the initial tone mapping function at the fifteenth interpolation point. The fifteenth interpolation point may be point 11' shown in FIG.

[0413] Method 2

[0414] The eighth interpolation point may be the twelfth interpolation point described above. The tenth interpolation point may be the thirteenth interpolation point described above. The seventh interpolation point may be the fourteenth interpolation point described above, whose first coordinate value is less than the first coordinate value of the twelfth interpolation point and less than the first coordinate value of the thirteenth interpolation point. The ninth interpolation point may be the fourteenth interpolation point described above, whose first coordinate value is greater than the first coordinate value of the twelfth interpolation point and greater than the first coordinate value of the thirteenth interpolation point. In this scenario, the spline functions include a fourth spline function that passes through the twelfth and thirteenth interpolation points, and a fifth spline function that passes through the thirteenth and fourteenth interpolation points. The locations of the particular interpolation points and the spline functions are shown in FIG. 7.

[0415] The above describes how to obtain the first coordinate values ​​of the 12th, 13th, and 14th interpolation points. In other words, the first coordinate value of the 7th, 8th, 9th, and 10th interpolation points are all known.

[0416] First, a first offset is obtained, for example, the first offset is preset to −0.01.

[0417] Then, the second coordinate value of the eighth interpolation point (point 10' in FIG. 4) is determined based on the first offset. For example, f3(x 10 )=f0(x 10 )-0.01, and f0(x) represents the initial tone mapping function.

[0418] Then the second offset is obtained, for example, the second offset is preset to +0.01.

[0419] Then, the second coordinate value of the eighth interpolation point (point 11' in FIG. 4) is determined based on the first offset. For example, f3(x 11 )=f0(x 11 )+0.01, and f0(x) represents the initial tone mapping function.

[0420] Since the third spline function is smoothly connected to the initial tone mapping function at the seventh interpolation point, it can be determined that the coordinates of the seventh interpolation point are (x9, f0(x9)), where f3(x9) = f0(x9), and f3'(x9) = f0'(x9).

[0421] The third spline function is smoothly connected to the initial tone mapping function at the ninth interpolation point, so the coordinates of the ninth interpolation point are (x 12 ,f0(x 12 )) and f3(x 12 )=f0(x 12 ), f3'(x 12 )=f0'(x 12 ) can be determined.

[0422] Then, a third spline function f3(x) passing through the seventh, eighth, and ninth interpolation points (i.e., passing through points 9', 10', and 12' in FIG. 4) is determined. The third spline function f3(x) can be divided into a seventh spline function f7(x) passing through the seventh and eighth interpolation points, an eighth spline function f8(x) passing through the eighth and tenth interpolation points, and a ninth spline function f9(x) passing through the tenth and ninth interpolation points.

[0423] f7(x9)=f0(x9), f7'(x9)=f0'(x9), f7(x 10 )=f0(x 10 )-0.01 may be determined based on the initial tone mapping function f0(x), the coordinates of the seventh interpolation point, and the coordinates of the eighth interpolation point.

[0424] f8(x 10 )=f0(x 10 )-0.01 and f8(x 11 )=f0(x 11 )+0.01 may be determined based on the coordinates of the eighth interpolation point and the coordinates of the tenth interpolation point.

[0425] f9(x 12 )=f0(x 12 ), f9'(x 12 )=f0'(x 12 ), and f9(x 11 )=f0(x 11 )+0.01 may be determined based on the initial tone mapping function f0(x), the coordinates of the tenth interpolation point, and the coordinates of the ninth interpolation point.

[0426] Based on the properties of the spline function, the first derivative of the seventh spline function at the eighth interpolation point is identical to the first derivative of the eighth spline function at the eighth interpolation point, and the second derivative of the seventh spline function at the eighth interpolation point is identical to the second derivative of the eighth spline function at the eighth interpolation point, i.e., f7'(x 10 )=f8'(x 10 ) and f7"(x 10)=f8"(x 10 ) can be determined.

[0427] Based on the properties of the spline function, the first derivative of the 8th spline function at the 10th interpolation point is the same as the first derivative of the 9th spline function at the 10th interpolation point, and the second derivative of the 8th spline function at the 10th interpolation point is the same as the second derivative of the 9th spline function at the 10th interpolation point, i.e., f8'(x 11 )=f9'(x 11 ), f8"(x 11 )=f9"(x 11 ) can be determined.

[0428] Based on the above conditions, the third spline function f3(x) can be determined.

[0429] Optionally, the third spline function is a cubic spline function. Specifically, the fourth and fifth spline functions are cubic spline functions.

[0430] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the fourth spline function passes through the eleventh interpolation point, and a second derivative of the first spline function at the eleventh interpolation point is zero.

[0431] Optionally, the plurality of interpolation points further includes an eleventh interpolation point, wherein the fifth spline function passes through the eleventh interpolation point, and a second derivative of the second spline function at the eleventh interpolation point is zero.

[0432] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the first spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the fourth spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0433] Optionally, the plurality of interpolation points further includes a fifteenth interpolation point, wherein the second spline function passes through the fifteenth interpolation point and the initial tone mapping function passes through the fifteenth interpolation point, and a derivative value of the fifth spline function at the fifteenth interpolation point differs from a derivative value of the initial tone mapping function at the fifteenth interpolation point.

[0434] 204: Modify the initial tone mapping curve based on a spline function to obtain a modified tone mapping curve for the target picture.

[0435] Modifying the initial tone mapping curve may be replacing a part of the initial tone mapping curve with a spline function. The finally obtained modified tone mapping curve includes a part of the data content originally existing in the initial tone mapping curve and the data content of the spline function. The initial spline function is usually smooth, and the spline function itself has a smooth attribute, and the spline function can be smoothly connected to the initial spline function at the interpolation points. Therefore, the modified tone mapping curve also has a smooth attribute.

[0436] 205: Perform tone mapping on the target picture based on the modified tone mapping curve.

[0437] The electronic device may perform tone mapping for the target picture based on a modified tone mapping curve with reference to the prior art, which is not described in detail here.

[0438] The electronic device implementing steps 201-205 may be the photographing device in FIG. 1, or may be the image processing device in FIG. 1, or may be a display device including an image processor.

[0439] The present application provides a method for modifying the acquired initial tone mapping curve, so that the manner of acquiring the tone mapping curve is more flexible. For example, the form of the tone mapping curve can be flexibly determined based on the device attributes of the display device and the shooting device, so that the image display is more vivid and natural. In another example, in a video, the tone mapping curve of a frame can be determined, and for the frame adjacent to the frame, the parameters for modifying the tone mapping curve are determined, so that the video display effect can be improved without additional excessive data.

[0440] 8 is a schematic diagram of the architecture of an image processing apparatus according to an embodiment of the present application. As shown in FIG. 8, the apparatus 500 includes an acquisition module 510, a processing module 520, and a tone mapping module 530.

[0441] The obtaining module 510 is configured to obtain a target picture and an initial tone mapping curve for the target picture.

[0442] The processing module 520 is configured to determine a number of interpolation points based on luminance information and / or dynamic metadata of the target picture.

[0443] The processing module 520 is further configured to determine a spline function that passes through the plurality of interpolation points, the spline function smoothly connecting to the initial tone mapping curve at at least one of the plurality of interpolation points.

[0444] The processing module 520 is further configured to modify the initial tone mapping curve based on a spline function to obtain a modified tone mapping curve for the target picture.

[0445] The tone mapping module 530 is configured to perform tone mapping on the target picture based on the modified tone mapping curve.

[0446] 9 is a schematic diagram of the architecture of an image processing device according to another embodiment of the present application. As shown in FIG. 9, the device 600 may include a transmission interface 610, a processor 620, and a memory 630.

[0447] FIG. 9 shows only one memory and one processor. In an actual image processing device product, there may be one or more processors and one or more memories. The memory may also be referred to as a storage medium or a storage device. The memory may be located independently of the processor or may be integrated with the processor. This embodiment of the present application is not limited thereto.

[0448] The transmission interface 610, the processor 620, and the memory 630 communicate with each other by using internal connection paths to transfer control and / or data signals.

[0449] Specifically, the transmission interface 610 is configured to receive or transmit image data. The processor 620 is configured to obtain an image to be processed, process the image to be processed by using a first branch of a pre-trained neural network model to obtain a first type of parameter (the first type of parameter is used to perform global color processing on the image), process the image to be processed by using a second branch of the neural network model to obtain a second type of parameter (the second type of parameter is used to perform local color processing on the image), and perform color processing on the image to be processed based on the first type of parameter and the second type of parameter to obtain a color-processed image.

[0450] The memory 630 described in the embodiments of the present application is configured to store computer instructions and parameters executed by the processor.

[0451] For the specific operation process and beneficial effects of the device 600, please refer to the description in the embodiment shown in Figure 2. Details will not be described again here.

[0452] An embodiment of the present application further provides an electronic device, which may be a terminal device, which may be configured to perform the functions / steps of the method embodiments described above.

[0453] 10 is a schematic diagram of the mechanism of an electronic device according to an embodiment of the present application. As shown in FIG. 10, the electronic device 700 includes a processor 710 and a transceiver 720. Optionally, the electronic device 700 may further include a memory 730. The processor 710, the transceiver 720, and the memory 730 may communicate with each other through an internal connection path and transfer control signals and / or data signals. The memory 730 is configured to store a computer program. The processor 710 is configured to call the computer program from the memory 730 and execute the computer program.

[0454] Optionally, the electronic device 700 may further include an antenna 740 configured to transmit the radio signals output by the transceiver 720.

[0455] The processor 710 and the memory 730 may be integrated into one processing unit or, more generally, into separate components. The processor 710 is configured to execute program code stored in the memory 730 to implement the above-mentioned functions. In a particular implementation, the memory 730 may alternatively be integrated into the processor 710 or may be separate from the processor 710. The processor 710 may correspond to the processor 620 in the device 600 in FIG. 9.

[0456] In addition, to improve the functionality of the electronic device 700, the electronic device 700 may further include one or more of an input unit 760, a display unit 770, an audio circuit 780, a camera 790, a sensor 701, etc. The audio circuit may further include a speaker 782, a microphone 784, etc. The display unit 770 may include a display screen.

[0457] Optionally, the electronic device 700 may further include a power source 750 configured to provide power to various devices or circuits in the terminal device.

[0458] It should be understood that the electronic device 700 shown in FIG. 10 may implement the process of the method embodiment shown in FIG. 2. The operation and / or function of the modules in the electronic device 700 are intended to separately implement the corresponding procedures in the above-mentioned method embodiment. For details, please refer to the description in the above-mentioned method embodiment. In order to avoid repetition, detailed description is appropriately omitted here. Those skilled in the art will recognize that the units and algorithm steps can be implemented by electronic hardware, or a combination of computer software and electronic hardware, in combination with the examples described in the embodiments disclosed herein. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered as going beyond the scope of the present application.

[0459] For the purpose of simple and easy description, the detailed operation processes of the above-mentioned systems, devices, and units may be clearly understood by those skilled in the art to refer to the corresponding processes in the above-mentioned method embodiments, and the details will not be described again here.

[0460] In some embodiments provided in the present application, it should be understood that the disclosed system, device and method may be implemented in other ways. For example, the described device embodiment is merely an example. For example, the division into multiple units is merely a logical division of functions, and may be other divisions in actual implementation. For example, multiple units or components may be combined or integrated into another system, and some features may be ignored or not performed. Furthermore, the shown or described mutual couplings or direct couplings or communication connections may be implemented by some interfaces. Indirect couplings or communication connections between devices or units may be implemented in electronic, mechanical, or other forms.

[0461] The units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units, located in one place or distributed over multiple network units. Some or all of the units may be selected based on the actual requirements for realizing the objectives of the solutions in the embodiments.

[0462] Furthermore, the functional units in the embodiments of the present application may be integrated into one processing unit, each of these units may exist physically alone, or two or more units may be integrated into one unit.

[0463] When these functions are implemented in the form of software functional units and sold or used as independent products, these functions may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application may be essentially implemented in the form of a software product, or the part that contributes to the prior art, or some of these technical solutions may be implemented in the form of a software product. The computer software product is stored in a storage medium and includes some instructions for instructing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or some of the steps of the method described in the embodiments of the present application. The above-mentioned storage medium may include any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0464] The above description is merely a specific implementation of the present application, but is not intended to limit the scope of protection of the present application. Any variations or replacements that a person skilled in the art can easily think of within the technical scope disclosed in this application shall be included in the scope of protection of the present application. Therefore, the scope of protection of the present application is subject to the scope of protection of the claims. [Other possible items] (Item 1) 1. An image processing method, comprising: obtaining a target picture and an initial tone mapping curve for the target picture; determining a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture; determining a spline function that passes through the plurality of interpolation points, the spline function smoothly connecting to the initial tone mapping curve at at least one of the plurality of interpolation points; modifying the initial tone mapping curve based on the spline function to obtain a modified tone mapping curve for the target picture; performing tone mapping on the target picture based on the modified tone mapping curve; A method for providing the above. (Item 2) The initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, a number of dark area pixels included in the target picture is equal to or larger than a second preset threshold, and a luminance value of the dark area pixel is smaller than a first preset threshold, the plurality of interpolation points includes a first interpolation point, and the step of determining a plurality of interpolation points based on luminance information of the target picture includes: 2. The method of claim 1, further comprising determining that a first coordinate value of the first interpolation point is the same as a first coordinate value of the first endpoint. (Item 3) The plurality of interpolation points further includes a second interpolation point, and the step of determining the plurality of interpolation points based on luminance information of the target picture includes: 3. The method of claim 2, further comprising determining a first coordinate value of the second interpolation point based on the number of dark area pixels. (Item 4) The method according to item 2 or 3, wherein the plurality of interpolation points further includes a third interpolation point, the target picture further includes a dark area transition pixel, the luminance value of the dark area transition pixel is greater than or equal to the first preset threshold, and the step of determining a plurality of interpolation points based on luminance information of the target picture includes a step of determining a first coordinate value of the third interpolation point based on a number of the dark area transition pixels. (Item 5) the initial tone mapping curve includes the first end point and the second end point, the first end point and the second end point are respectively located at the two ends of the initial tone mapping curve, the luminance value corresponding to the first end point is smaller than the luminance value corresponding to the second end point, a number of bright area pixels included in the target picture is equal to or greater than a fourth preset threshold, and the luminance value of the bright area pixel is greater than a third preset threshold, the plurality of interpolation points includes a fourth interpolation point, and the step of determining a plurality of interpolation points based on luminance information of the target picture includes: 5. The method of any one of claims 1 to 4, further comprising determining that a first coordinate value of the fourth interpolation point is the same as a first coordinate value of the second end point. (Item 6) The plurality of interpolation points further includes a fifth interpolation point, and the step of determining the plurality of interpolation points based on luminance information of the target picture includes: Item 6. The method of item 5, further comprising determining a first coordinate value of the fifth interpolation point based on the number of the bright region pixels. (Item 7) The method according to item 5 or 6, wherein the plurality of interpolation points further includes a sixth interpolation point, the target picture further includes a light region transition pixel, the luminance value of the light region transition pixel being less than or equal to the third preset threshold, and the step of determining a plurality of interpolation points based on the luminance information of the target picture includes a step of determining a first coordinate value of the sixth interpolation point based on the number of the light region transition pixels. (Item 8) 2. The method of claim 1, wherein the dynamic metadata includes coordinate information for each of the plurality of interpolation points. (Item 9) the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are respectively located at two ends of the initial tone mapping curve, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, and the dynamic metadata includes region luminance indication information; When the region luminance indication information indicates a dark region, the plurality of interpolation points includes a first interpolation point, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point; and / or 2. The method according to claim 1, wherein when the region brightness indication information indicates a bright region, the plurality of interpolation points includes a second interpolation point, and a first coordinate value of the second interpolation point is the same as a first coordinate value of the second end point. (Item 10) The initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are located at two ends of the initial tone mapping curve respectively, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the dynamic metadata includes P coordinate information having a one-to-one correspondence with P key pixels, P is a positive integer, and the step of determining a plurality of interpolation points based on the luminance information and the dynamic metadata of the target picture includes: determining that the P key pixels are dark area pixels or bright area pixels based on the P coordinate information and the luminance information of the target picture, where the luminance value of the dark area pixels is less than a first preset threshold, and the luminance value of the bright area pixels is greater than a third preset threshold; determining that the plurality of interpolation points includes a first interpolation point when the P key pixels are the dark area pixels, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point; or determining that the plurality of interpolation points includes a second interpolation point when the P key pixels are the bright region pixels, and a first coordinate value of the second interpolation point is equal to a first coordinate value of the second end point; The method according to item 1, comprising: (Item 11) the plurality of interpolation points include a seventh interpolation point, an eighth interpolation point and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, and the step of determining a spline function passing through the plurality of interpolation points comprises: obtaining a first spline function passing through the seventh interpolation point and the eighth interpolation point; determining a second spline function passing through the eighth interpolation point and the ninth interpolation point, the second spline function smoothly connected to the first spline function at the eighth interpolation point, and the second spline function smoothly connected to the initial tone mapping curve at the ninth interpolation point; 11. The method according to any one of items 1 to 10, comprising: (Item 12) Item 12. The method according to item 11, wherein the first spline function is a linear spline function and the second spline function is a cubic spline function. (Item 13) the plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and a first coordinate value of the eighth interpolation point is between a first coordinate value of the seventh interpolation point and a first coordinate value of the ninth interpolation point; Prior to the step of determining a plurality of interpolation points, the method further comprises: obtaining a first offset, the first offset being used to represent a modification range of the initial tone mapping curve at a first modified position, the first modified position being the first coordinate value of the eighth interpolation point; The step of determining a plurality of interpolation points comprises: determining a second coordinate value of the eighth interpolation point based on a function value of the initial tone mapping curve at the first modified position and the first offset; determining a spline function passing through the plurality of interpolation points includes determining a third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point. 11. The method according to any one of items 1 to 10. (Item 14) Item 14. The method of item 13, wherein the third spline function includes a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point, and a second derivative value of the fourth spline function at the eighth interpolation point is identical to a second derivative value of the fifth spline function at the eighth interpolation point. (Item 15) the plurality of interpolation points further includes a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; Prior to the step of determining a plurality of interpolation points, the method further comprises: obtaining a second offset, the second offset being used to represent a modification range of the initial tone mapping curve at a second modified position, the second modified position being the first coordinate value of the tenth interpolation point; determining a plurality of interpolation points further comprises determining a second coordinate value of the tenth interpolation point based on the function value of the initial tone mapping curve at the second modified position and the second offset; The step of determining the third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point includes: determining the third spline function based on the first and second coordinate values ​​of the eighth interpolation point and the first and second coordinate values ​​of the tenth interpolation point, wherein the third spline function smoothly connects to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point; 15. The method according to item 13 or 14. (Item 16) 16. The method according to any one of items 1 to 15, wherein the derivative value of the spline function is always greater than 0. (Item 17) The initial tone mapping curve is expressed as follows:

number

number

number

number

Claims

1. 1. An image processing method, comprising: obtaining an initial tone mapping curve for a target picture; obtaining luminance information and / or dynamic metadata of the target picture; determining a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture; determining a spline function that passes through the plurality of interpolation points; obtaining a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; performing tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is less than a luminance value corresponding to the second end point, and the plurality of interpolation points includes a first interpolation point; A method according to claim 1, wherein when a number of dark area pixels in the target picture having a luminance value smaller than a first preset threshold is equal to or greater than a second preset threshold, a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point.

2. The plurality of interpolation points further includes a second interpolation point, and the step of determining the plurality of interpolation points based on luminance information of the target picture includes:

2. The method of claim 1, further comprising determining a first coordinate value of the second interpolation point based on the number of dark area pixels.

3. 3. The method of claim 1, wherein the plurality of interpolation points further includes a third interpolation point, the target picture further includes a dark area transition pixel, the luminance value of the dark area transition pixel being equal to or greater than the first preset threshold, and the step of determining a plurality of interpolation points based on luminance information of the target picture includes determining a first coordinate value of the third interpolation point based on a number of the dark area transition pixels.

4. the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, and the plurality of interpolation points includes a fourth interpolation point, and the step of determining the plurality of interpolation points based on luminance information of the target picture includes:

4. The method according to claim 1, further comprising: determining that a first coordinate value of the fourth interpolation point is equal to a first coordinate value of the second end point when a number of bright area pixels in the target picture having a luminance value greater than a third preset threshold is equal to or greater than a fourth preset threshold.

5. An image processing method comprising the steps of: obtaining an initial tone mapping curve for a target picture; obtaining luminance information and / or dynamic metadata of the target picture; determining a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture; determining a spline function that passes through the plurality of interpolation points; obtaining a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; performing tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is less than a luminance value corresponding to the second end point, and the plurality of interpolation points includes a fourth interpolation point; A method according to claim 1, wherein when the number of bright region pixels in the target picture having a luminance value greater than a third preset threshold is equal to or greater than a fourth preset threshold, the first coordinate value of the fourth interpolation point is equal to the first coordinate value of the second end point.

6. The plurality of interpolation points further includes a fifth interpolation point, and the step of determining the plurality of interpolation points based on luminance information of the target picture further comprises:

6. The method of claim 4, further comprising determining a first coordinate value of the fifth interpolation point based on the number of light area pixels.

7. 7. The method of claim 4, wherein the plurality of interpolation points further includes a sixth interpolation point, the target picture further includes a light region transition pixel, the luminance value of the light region transition pixel being less than or equal to the third preset threshold, and the step of determining a plurality of interpolation points based on luminance information of the target picture includes the step of determining a first coordinate value of the sixth interpolation point based on a number of the light region transition pixels.

8. An image processing method, comprising: obtaining an initial tone mapping curve for a target picture; obtaining luminance information and / or dynamic metadata of the target picture; determining a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture; determining a spline function that passes through the plurality of interpolation points; obtaining a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; performing tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, and the dynamic metadata includes a region luminance indication; When the region luminance indication information indicates a dark region, the plurality of interpolation points includes a first interpolation point, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point; and / or When the region luminance indication indicates a bright region, the plurality of interpolation points includes a second interpolation point, and a first coordinate value of the second interpolation point is the same as a first coordinate value of the second end point.

9. An image processing method, comprising: obtaining an initial tone mapping curve for a target picture; obtaining luminance information and / or dynamic metadata of the target picture; determining a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture; determining a spline function that passes through the plurality of interpolation points; obtaining a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; performing tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are located at two ends of the initial tone mapping curve, respectively, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, the dynamic metadata includes P coordinate information having a one-to-one correspondence with P key pixels, P being a positive integer, and the step of determining a plurality of interpolation points based on the luminance information and the dynamic metadata of the target picture includes: determining that the P key pixels are dark area pixels or bright area pixels based on the P coordinate information and the luminance information of the target picture, where the luminance value of the dark area pixels is less than a first preset threshold and the luminance value of the bright area pixels is greater than a third preset threshold; determining that the plurality of interpolation points includes a first interpolation point when the P key pixels are the dark area pixels, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point; or determining that the plurality of interpolation points includes a second interpolation point when the P key pixels are the bright region pixels, a first coordinate value of the second interpolation point being equal to a first coordinate value of the second end point; A method comprising:

10. the plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the initial tone mapping curve passes through the ninth interpolation point, and the step of determining a spline function passing through the plurality of interpolation points comprises: obtaining a first spline function passing through the seventh interpolation point and the eighth interpolation point; determining a second spline function that passes through the eighth interpolation point and the ninth interpolation point, the second spline function smoothly connected to the first spline function at the eighth interpolation point, and the second spline function smoothly connected to the initial tone mapping curve at the ninth interpolation point; 10. The method of claim 1 , comprising:

11. The method of claim 10 , wherein the first spline function is a linear spline function and the second spline function is a cubic spline function.

12. the plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and a first coordinate value of the eighth interpolation point is between a first coordinate value of the seventh interpolation point and a first coordinate value of the ninth interpolation point; Prior to the step of determining a plurality of interpolation points, the method further comprises: obtaining a first offset, the first offset being used to represent a modification range of the initial tone mapping curve at a first modification position, the first modification position being the first coordinate value of the eighth interpolation point; The step of determining a plurality of interpolation points comprises: determining a second coordinate value of the eighth interpolation point based on a function value of the initial tone mapping curve at the first modified position and the first offset; determining a spline function passing through the plurality of interpolation points includes determining a third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, wherein the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

10. The method according to any one of claims 1 to 9.

13. 13. The method of claim 12, wherein the third spline function includes a fourth spline function passing through the seventh and eighth interpolation points and a fifth spline function passing through the eighth and ninth interpolation points, and a second derivative value of the fourth spline function at the eighth interpolation point is the same as a second derivative value of the fifth spline function at the eighth interpolation point.

14. the plurality of interpolation points further includes a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; The method further comprises: obtaining a second offset, the second offset being used to represent a modification range of the initial tone mapping curve at a second modified position, the second modified position being the first coordinate value of the tenth interpolation point; determining a plurality of interpolation points further comprises determining a second coordinate value of the tenth interpolation point based on a function value of the initial tone mapping curve at the second modified position and the second offset; determining the third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, determining the third spline function based on the first and second coordinate values ​​of the eighth interpolation point and the first and second coordinate values ​​of the tenth interpolation point, wherein the third spline function smoothly connects to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

14. The method according to claim 12 or 13.

15. The method according to claim 1 , wherein the derivative value of the spline function is always greater than zero.

16. The initial tone mapping curve is given by: [0019] [0020] ##EQU00021## and [0022] where L represents a linear signal value, L' represents a nonlinear signal value, and p, a, m, b, c １ , c ２ , c ３ , n, N and ρ are all dynamic range mapping coefficients, with 0≦L≦1; 16. The method according to any one of claims 1 to 15.

17. An image processing device, an acquisition module configured to acquire an initial tone mapping curve for a target picture; the acquisition module configured to acquire luminance information and / or dynamic metadata of the target picture; 11. A processing module configured to determine a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture, the processing module comprising: The processing module is further configured to determine a spline function that passes through the plurality of interpolation points; a processing module configured to obtain a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; a tone mapping module configured to perform tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is less than a luminance value corresponding to the second end point, and the plurality of interpolation points includes a first interpolation point; An image processing device, wherein when the number of dark area pixels included in the target picture having a brightness value smaller than a first preset threshold is equal to or greater than a second preset threshold, the first coordinate value of the first interpolation point is identical to the first coordinate value of the first endpoint.

18. the plurality of interpolation points further includes a second interpolation point; The image processing apparatus of claim 17 , wherein the processing module is configured to determine a first coordinate value of the second interpolation point based on the number of the dark area pixels.

19. the plurality of interpolation points further includes a third interpolation point, the target picture further includes a dark area transition pixel, and the luminance value of the dark area transition pixel is equal to or greater than the first preset threshold; the processing module is configured to determine a first coordinate value of the third interpolation point based on the number of the dark region transition pixels.

19. The image processing device according to claim 17 or 18.

20. the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is less than a luminance value corresponding to the second end point, and the plurality of interpolation points includes a fourth interpolation point; the processing module is configured to determine that a first coordinate value of the fourth interpolation point is equal to a first coordinate value of the second end point when a number of bright region pixels in the target picture having a luminance value greater than a third preset threshold is equal to or greater than a fourth preset threshold; 20. An image processing device according to any one of claims 17 to 19.

21. An image processing device, comprising: an acquisition module configured to acquire an initial tone mapping curve for a target picture; the acquisition module configured to acquire luminance information and / or dynamic metadata of the target picture; 11. A processing module configured to determine a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture, the processing module comprising: The processing module is further configured to determine a spline function that passes through the plurality of interpolation points; a processing module configured to obtain a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; a tone mapping module configured to perform tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is less than a luminance value corresponding to the second end point, and the plurality of interpolation points includes a fourth interpolation point; When a number of bright region pixels included in the target picture having a luminance value greater than a third preset threshold is equal to or greater than a fourth preset threshold, a first coordinate value of the fourth interpolation point is equal to a first coordinate value of the second end point. Image processing device.

22. the plurality of interpolation points further includes a fifth interpolation point; 22. The image processing apparatus of claim 20 or 21, wherein the processing module is configured to determine a first coordinate value of the fifth interpolation point based on the number of the light region pixels.

23. the plurality of interpolation points further includes a sixth interpolation point, the current picture further includes a light region transition pixel, the light region transition pixel having a luminance value less than or equal to the third preset threshold; the processing module is configured to determine a first coordinate value of the sixth interpolation point based on the number of the light region transition pixels.

23. An image processing device according to any one of claims 20 to 22.

24. An image processing device, comprising: an acquisition module configured to acquire an initial tone mapping curve for a target picture; the acquisition module configured to acquire luminance information and / or dynamic metadata of the target picture; 11. A processing module configured to determine a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture, the processing module comprising: The processing module is further configured to determine a spline function that passes through the plurality of interpolation points; a processing module configured to obtain a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; a tone mapping module configured to perform tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point, and the dynamic metadata includes a region luminance indication; When the region luminance indication information indicates a dark region, the plurality of interpolation points includes a first interpolation point, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point; and / or When the region luminance indication information indicates a bright region, the plurality of interpolation points includes a second interpolation point, and a first coordinate value of the second interpolation point is the same as a first coordinate value of the second end point.

25. An image processing device, comprising: an acquisition module configured to acquire an initial tone mapping curve for a target picture; the acquisition module configured to acquire luminance information and / or dynamic metadata of the target picture; 11. A processing module configured to determine a plurality of interpolation points based on luminance information and / or dynamic metadata of the target picture, the processing module comprising: The processing module is further configured to determine a spline function that passes through the plurality of interpolation points; a processing module configured to obtain a modified tone mapping curve for the target picture based on the initial tone mapping curve and the spline function; a tone mapping module configured to perform tone mapping on the target picture based on the modified tone mapping curve; Equipped with the initial tone mapping curve includes a first end point and a second end point, the first end point and the second end point are located at two ends of the initial tone mapping curve, respectively, a luminance value corresponding to the first end point is smaller than a luminance value corresponding to the second end point; the dynamic metadata includes P coordinate information having a one-to-one correspondence with P key pixels, where P is a positive integer; The processing module is configured to determine, based on the P coordinate information and the luminance information of the target picture, that the P key pixels are dark area pixels or bright area pixels, where the luminance value of the dark area pixel is smaller than a first preset threshold, and the luminance value of the bright area pixel is larger than a third preset threshold; When the P key pixels are the dark area pixels, the processing module is configured to determine that the plurality of interpolation points includes a first interpolation point, and a first coordinate value of the first interpolation point is equal to a first coordinate value of the first end point; or When the P key pixels are the bright region pixels, the processing module is configured to determine that the plurality of interpolation points includes a second interpolation point, and a first coordinate value of the second interpolation point is equal to a first coordinate value of the second end point. Image processing device.

26. the plurality of interpolation points includes a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, and the initial tone mapping curve passes through the ninth interpolation point; the processing module is configured to obtain a first spline function passing through the seventh interpolation point and the eighth interpolation point; the processing module is configured to determine a second spline function passing through the eighth interpolation point and the ninth interpolation point, the second spline function smoothly connected to the first spline function at the eighth interpolation point, and the second spline function smoothly connected to the initial tone mapping curve at the ninth interpolation point.

26. An image processing device according to any one of claims 17 to 25.

27. 27. The image processing apparatus of claim 26, wherein the first spline function is a linear spline function and the second spline function is a cubic spline function.

28. the plurality of interpolation points include a seventh interpolation point, an eighth interpolation point, and a ninth interpolation point, the spline function includes a third spline function passing through the seventh interpolation point, the eighth interpolation point, and the ninth interpolation point, and a first coordinate value of the eighth interpolation point is between a first coordinate value of the seventh interpolation point and a first coordinate value of the ninth interpolation point; The obtaining module is further configured to obtain a first offset, the first offset being used to represent a modification range of the initial tone mapping curve at a first modification position, the first modification position being the first coordinate value of the eighth interpolation point; the processing module is configured to determine a second coordinate value of the eighth interpolation point based on a function value of the initial tone mapping curve at the first modified position and the first offset; 26. The image processing device of claim 17, wherein the processing module is configured to determine the third spline function based on the first coordinate value of the eighth interpolation point and the second coordinate value of the eighth interpolation point, and the third spline function is smoothly connected to the initial tone mapping curve at the seventh interpolation point and / or the ninth interpolation point.

29. 29. The image processing device of claim 28, wherein the third spline function includes a fourth spline function passing through the seventh interpolation point and the eighth interpolation point, and a fifth spline function passing through the eighth interpolation point and the ninth interpolation point, and a second derivative value of the fourth spline function at the eighth interpolation point is identical to a second derivative value of the fifth spline function at the eighth interpolation point.

30. the plurality of interpolation points further includes a tenth interpolation point, the third spline function passes through the tenth interpolation point, and a first coordinate value of the tenth interpolation point is between the first coordinate value of the seventh interpolation point and the first coordinate value of the ninth interpolation point; Before the processing module determines the plurality of interpolation points, the obtaining module is further configured to obtain a second offset, the second offset being used to represent a modification range of the initial tone mapping curve at a second modification position, the second modification position being the first coordinate value of the tenth interpolation point; the processing module is configured to determine a second coordinate value of the tenth interpolation point based on a function value of the initial tone mapping curve at the second modified position and the second offset; the processing module is configured to determine the third spline function based on the first and second coordinate values ​​of the eighth interpolation point and the first and second coordinate values ​​of the tenth interpolation point, the third spline function smoothly connecting to the initial tone mapping curve at both the seventh interpolation point and the ninth interpolation point.

30. An image processing device according to claim 28 or 29.

31. 31. An image processing apparatus according to any one of claims 17 to 30, wherein a derivative value of the spline function is always greater than zero.

32. 1. An image processing device comprising a processor and a transmission interface, the transmission interface configured to receive or transmit image data; An image processing apparatus, the processor being configured to access program instructions stored in a memory and to execute the program instructions to perform a method according to any one of claims 1 to 16.

33. 17. A computer-readable storage medium having instructions stored thereon that, when executed on a computer or processor, enable the computer or processor to perform a method according to any one of claims 1 to 16.

34. A computer program product causing a processor to carry out a method according to any one of claims 1 to 16.

Images