A method and apparatus for image processing

Abstract

The application provides a method for image processing, comprising: adaptively determining at least one interpolation point according to feature information or dynamic metadata of a to-be-processed image, and correcting a spline curve through the at least one interpolation point, wherein the spline curve comprises at least one of a first-order spline curve or a third-order spline curve; and performing tone mapping on the to-be-processed image according to a corrected tone mapping curve, wherein the corrected tone mapping curve comprises at least a part of an initial tone mapping curve and the corrected spline curve. The method provided by the application can adaptively modify the tone mapping curve according to the needs of a scene, so that the images in various scenes can be displayed with better effects.

Description

[0001] This application is a divisional application. The original application has the application number 201910996393.9 and the original application date is October 18, 2019. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing, and more specifically, to an image processing method and apparatus. Background Technology

[0003] Dynamic range (DR) can be used to represent the ratio of the maximum to the minimum grayscale value of an image. Dynamic range can also be used to represent the ratio of the maximum to the minimum brightness value of an image. The dynamic range in the real world is between 10... -3 -10 6 The range between these ranges is called High Dynamic Range (HDR), but the display capabilities of display devices are typically limited, far below the brightness range of the real world. For example, if the brightness of an object in the real world is 2000 nits, while the maximum brightness a display device can display is only 500 nits, the display device will be unable to reproduce that object without processing. Conversely, if the brightness of an object in the real world is low, say 10 nits, while the minimum brightness a display device can display is 100 nits, the display device will also be unable to reproduce that object without any processing. This application provides a method aimed at enabling display devices to vividly and realistically reproduce scenes from the real world. Summary of the Invention

[0004] This application provides an image processing method and apparatus, the purpose of which is to improve the tone mapping effect of a display device.

[0005] In a first aspect, an image processing method is provided, comprising: determining first coordinates of at least one interpolation point based on feature information or dynamic metadata of an image to be processed; determining at least one second coordinate and at least one first derivative based on the first coordinates of the at least one interpolation point; determining coefficients of a modified spline curve based on the first coordinates, at least one second coordinate, and at least one first derivative of the at least one interpolation point, the modified spline curve including at least one of a first-order spline curve or a cubic spline curve; and performing tone mapping on the image to be processed based on the modified tone mapping curve, the modified tone mapping curve including at least a portion of an initial tone mapping curve and the modified spline curve.

[0006] The position of the interpolation point is adaptively determined based on the feature information or dynamic metadata of the image to be processed, such as brightness information and pixel distribution information. The most suitable corrected spline curve for the image to be processed is flexibly determined based on these interpolation points. The image to be processed is then mapped to a certain tone based on the corrected tone mapping curve. Since the tone mapping curve is not fixed but adaptively adjusted according to the image to be processed or dynamic metadata, images in various scenarios can achieve good display effects.

[0007] The image processing method provided in this application can adaptively modify curves according to the needs of the scene to achieve specific effects, such as increasing brightness, increasing contrast, and blurring some information. For example, the display effect of dark area details can be improved by increasing the brightness of dark areas, or the contrast can be improved by decreasing the brightness of dark areas; or more bits can be allocated to bright areas to display the details of bright areas.

[0008] In one possible implementation, the at least one interpolation point includes: a first interpolation point; the modified spline curve includes: a first spline curve, the first spline curve being a first-order spline curve passing through the first interpolation point; the coefficients of the modified spline curve include a first coefficient MA0 and a second coefficient MB0; determining at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point includes: determining the second coordinate VA1 of the first interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point, and the first derivative GD1 of the initial tone mapping curve at the first interpolation point; determining the coefficients of the modified spline curve based on the first coordinate, at least one second coordinate, and at least one first derivative of the at least one interpolation point includes: determining GD1 as the second coefficient MB0 of the first spline curve; determining VA1 as the first coefficient MA0 of the first spline curve.

[0009] This application embodiment only needs to determine one interpolation point, and optionally, the first interpolation point is close to the endpoint of the bright area, corresponding to the case where the overall scene of the image is relatively bright. At this time, the bright area is corrected to better show the bright details of the image. Specifically, the pixel area with a tone mapping greater than a preset threshold (TH1) is based on a linear function, and the first linear curve is smoothly connected to the initial tone mapping curve at the first interpolation point to avoid the tone mapping effect being affected by the non-smooth tone mapping curve.

[0010] In one possible implementation, the at least one interpolation point includes: a first interpolation point and a second interpolation point; the modified spline curve includes: a first spline curve, which is a cubic spline curve passing through the first interpolation point and the second interpolation point; the coefficients of the modified spline curve include a first coefficient MA1, a second coefficient MB1, a third coefficient MC1, and a fourth coefficient MD1; determining at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point includes: calculating the first interpolation point and the second interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point and the first coordinate TH2 of the second interpolation point, respectively. The second coordinates VA1 and VA2 are used; the first derivative GD1 and the first derivative GD2 of the initial tone mapping curve at the first interpolation point are calculated based on TH1 and TH2; the coefficients of the modified spline curve are determined based on the first coordinates, at least one second coordinate, and at least one first derivative of the at least one interpolation point, including: determining VA1 as the first coefficient MA1 of the first spline curve; determining GD1 as the second coefficient MB1 of the first spline curve; constructing a system of equations containing two equations based on TH1, TH2, VA2, and GD2; and analyzing the system of equations to obtain the third coefficient MC1 and the fourth coefficient MD1 of the first spline curve.

[0011] This application embodiment requires determining two interpolation points. In one optional case, these two interpolation points are close to the endpoints of the bright area, corresponding to a situation where the overall scene of the image is relatively bright. In this case, the bright area is corrected to better display the bright details of the image. Specifically, the pixel region between the first interpolation point and the second interpolation point is tone mapped based on a cubic spline function, and the cubic spline function is smoothly connected to the initial tone mapping curve at the first interpolation point to avoid the non-smooth tone mapping curve affecting the tone mapping effect.

[0012] In one possible implementation, the system of equations constructed based on TH1, TH2, VA2, and GD2 includes: making the second coordinate of the second interpolation point on the first homogeneous curve the sum of VA2 and S1, thus obtaining the first equation, where S1 is the coordinate offset value included in the dynamic metadata; and making the first derivative of the first homogeneous curve at the second interpolation point the sum of GD2 and S2, thus obtaining the second equation, where S2 is the slope offset value included in the dynamic metadata.

[0013] In one possible implementation, the at least one interpolation point includes: a first interpolation point; the modified spline curve includes: a first homoline curve, the first homoline curve being a first-order spline function passing through the origin and the first interpolation point; the coefficients of the modified spline curve include: the first derivative MB0 of the first homoline curve; before determining at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point, the method further includes: obtaining the first derivative MB0 of the first homoline curve based on at least one of the maximum value maxL of the brightness of the image to be processed and the average value avgL of the brightness of the image to be processed included in the dynamic metadata.

[0014] In one possible implementation, the at least one interpolation point further includes: a second interpolation point and a third interpolation point; the modified spline curve further includes: a second spline curve and a third spline curve, the second spline curve being a cubic spline curve passing through the first interpolation point and the second interpolation point, and the third spline curve being a cubic spline curve passing through the second interpolation point and the third interpolation point; the coefficients of the modified spline curve further include: a first coefficient MA1, a second coefficient MB1, a third coefficient MC1, and a fourth coefficient MD1 of the second spline curve, and a first coefficient MA2, a second coefficient MB2, a third coefficient MC2, and a fourth coefficient MD2 of the third spline curve; the formula is as follows: The determination of at least one second coordinate and at least one first derivative by the first coordinate of at least one interpolation point includes: calculating the second coordinate VA1 of the first interpolation point on the first line curve based on the first coordinates TH1 and MB0 of the first interpolation point; calculating the second coordinate VA3 of the third interpolation point on the initial tone mapping curve based on the first coordinate TH3 of the third interpolation point; calculating the second coordinate VA2 of the second interpolation point on a first straight line based on the first coordinate TH2 of the second interpolation point, the first straight line being the straight line containing the first interpolation point and the third interpolation point; and calculating the first derivative GD3 of the initial tone mapping curve at the third interpolation point based on TH3.

[0015] In one possible implementation, determining the coefficients of the modified spline curve based on the first coordinates of the at least one interpolation point, at least one second coordinate, and at least one first derivative includes: determining VA1 as the first coefficient MA1 of the second spline curve; determining MB0 as the second coefficient MB1 of the second spline curve; determining VA2 as the first coefficient MA2 of the third spline curve; constructing a system of equations containing five equations based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the second spline curve and the third spline curve at the second interpolation point; and analyzing the system of equations containing five equations to obtain MC1, MD1, MB2, MC2, and MD2.

[0016] This application embodiment addresses a scenario where the overall scene of the image is relatively dark. In this case, the dark areas are corrected to better showcase the details in the darker parts of the image. This allows pixels with brightness below the display capability of the display device to have a better display effect. Specifically, for pixels below a first threshold (TH1), tone mapping is performed based on a linear spline function formula, which better preserves the dark detail information of this part. Then, a cubic spline function is used to achieve a smooth transition between the linear spline function and the initial tone mapping curve, avoiding the influence of an uneven tone mapping curve on the tone mapping effect. It should be understood that there are two cubic spline curves in this application embodiment. In an optional case, there may be only one cubic spline curve. The first cubic spline curve is smoothly connected to the initial tone mapping curve through a cubic spline function. In this case, only two interpolation points need to be determined.

[0017] Specifically, the first sample curve increases the brightness of the dark areas compared to the base curve, which is equivalent to allocating more bits to the dark areas. The two cubic spline functions achieve a smooth transition between the first sample curve and the initial tone mapping curve, and the enhancement or reduction of the correction area can be achieved by controlling the points and slopes of the curves.

[0018] In one possible implementation, the system of equations comprising five equations is constructed based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the second spline curve and the third spline curve at the second interpolation point. The equations include: setting the second coordinate of the second interpolation point on the second spline curve to VA2 to obtain a first equation; setting the second coordinate of the third interpolation point on the third spline curve to VA3 to obtain a second equation; setting the first derivative of the third spline curve at the third interpolation point to GD3 to obtain a third equation; setting the first derivatives of the second spline curve and the third spline curve at the second interpolation point to be equal to obtain a fourth equation; and setting the second derivatives of the second spline curve and the third spline curve at the second interpolation point to be equal to obtain a fifth equation.

[0019] In one possible implementation, determining the first coordinates of at least one interpolation point based on feature information or dynamic metadata of the image to be processed includes: determining the first coordinates TH1 of the first interpolation point based on at least one of the maximum value maxL and the average value avgL of the brightness of the image to be processed included in the dynamic metadata; TH2 = TH1 + B; TH3 = TH2 + C TH2-D CH2; where B, C, and D are constants. Optionally, B, C, and D are constants, and in one optional case, these constants can be included in the dynamic metadata. Alternatively, B, C, and D can also be directly defined in the relevant program code without needing to be obtained from the dynamic metadata.

[0020] In one possible implementation, the at least one interpolation point includes: a first interpolation point, a second interpolation point, and a third interpolation point; the modified spline curve includes: a first spline curve and a second spline curve; the first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a cubic spline curve passing through the second interpolation point and the third interpolation point; the coefficients of the modified spline curve include: a first coefficient MA1, a second coefficient MB1, a third coefficient MC1, and a fourth coefficient MD1 of the first spline curve, and a first coefficient MA2, a second coefficient MB2, a third coefficient MC2, and a fourth coefficient MD2 of the second spline curve. The determination of at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point includes: calculating the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point and the first coordinate TH3 of the third interpolation point, respectively; calculating the second coordinate VA2 of the second interpolation point on a first straight line based on the first coordinate TH2 of the second interpolation point, the first straight line being the straight line where the first interpolation point and the third interpolation point are located; and calculating the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 of the initial tone mapping curve at the third interpolation point.

[0021] The embodiments of this application correspond to images containing pixels located within a specific brightness range. Therefore, this specific brightness range is targeted for correction to better display the brightness details of the image within that range. Specifically, tone mapping is performed on the pixels between the first and second interpolation points, as well as between the second and third interpolation points, based on a cubic spline function, improving the display effect of these pixels. Furthermore, the transitions between the first and second spline curves, and between the first and second spline curves and the initial tone mapping curve, are smooth to avoid the influence of uneven tone mapping curves on the tone mapping effect.

[0022] In one possible implementation, determining the coefficients of the modified spline curve based on the first coordinates of the at least one interpolation point, at least one second coordinate, and at least one first derivative includes: determining VA1 as the first coefficient MA1 of the first spline curve; determining GD1 as the second coefficient MB1 of the first spline curve; determining VA2 as the sum of the offset value and the first coefficient MA2 of the second spline curve, wherein the offset value is 0, positive, or negative; constructing a system of equations containing five equations based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point; and analyzing the system of equations containing five equations to obtain MC1, MD1, MB2, MC2, and MD2.

[0023] Optionally, this offset value is included in the dynamic metadata.

[0024] In one possible implementation, the system of equations comprising five equations is constructed based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point. The equations include: setting the second coordinate of the second interpolation point on the first spline curve to VA2 to obtain the first equation; setting the second coordinate of the third interpolation point on the second spline curve to VA3 to obtain the second equation; setting the first derivative of the second spline curve at the third interpolation point to GD3 to obtain the third equation; setting the first derivatives of the first spline curve and the second spline curve at the second interpolation point to be equal to obtain the fourth equation; and setting the second derivatives of the first spline curve and the second spline curve at the second interpolation point to be equal to obtain the fifth equation.

[0025] In one possible implementation, the at least one interpolation point includes: a first interpolation point, a second interpolation point, and a third interpolation point; the modified spline curve includes: a first spline curve and a second spline curve; the first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point; the second spline curve is a linear spline curve passing through the second interpolation point and the third interpolation point; the coefficients of the modified spline curve include: a first coefficient MA1, a second coefficient MB1, a third coefficient MC1, and a fourth coefficient MD1 of the first spline curve; and a first coefficient MA2 and a second coefficient MB1 of the second spline curve. 2; Determining at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point includes: calculating the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point and the first coordinate TH3 of the third interpolation point, respectively; calculating the second coordinate VA2 of the second interpolation point on a first straight line based on the first coordinate TH2 of the second interpolation point, the first straight line being the straight line where the first interpolation point and the third interpolation point are located; and calculating the first derivative GD1 of the initial tone mapping curve at the first interpolation point.

[0026] In one possible implementation, determining the coefficients of the modified spline curve based on the first coordinates of the at least one interpolation point, at least one second coordinate, and at least one first derivative includes: determining VA1 as the first coefficient MA1 of the first spline curve; determining GD1 as the second coefficient MB1 of the first spline curve; determining VA2 and the offset value as the first coefficient MA2 of the second spline curve, wherein the offset value is 0, positive, or negative; constructing a system of equations containing three equations based on TH1, TH2, TH3, VA2, VA3, and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point; and analyzing the system of equations containing three equations to obtain MC1, MD1, and MB2.

[0027] It should be understood that this offset value can be included in dynamic metadata.

[0028] In this embodiment of the application, the overall scene of the image is relatively bright. In this case, the bright areas are corrected to better show the bright details of the image. Specifically, the pixel area between the second interpolation point and the third interpolation point is tone mapped based on the first spline curve, and the first spline function and the initial tone mapping curve are smoothly connected based on the cubic spline function between the first interpolation point and the second interpolation point to avoid the tone mapping effect being affected by the non-smooth tone mapping curve.

[0029] In one possible implementation, the system of equations comprising three equations is constructed based on TH1, TH2, TH3, VA2, VA3 and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point. This includes: setting the second coordinate of the second interpolation point on the first spline curve to VA2 to obtain the first equation; setting the second coordinate of the third interpolation point on the second spline curve to VA3 to obtain the second equation; and setting the first derivatives of the first spline curve and the second spline curve at the second interpolation point to be equal to obtain the third equation.

[0030] In one possible implementation, the at least one interpolation point constitutes at least one correction interval, wherein each correction interval corresponds to a correction spline curve, and the tone mapping of the image to be processed according to the correction tone mapping curve includes: when the pixel to be processed is located within the at least one correction interval, the pixel to be processed is tone mapped based on the correction spline curve; when the pixel to be processed is outside the at least one correction interval, the pixel to be processed is tone mapped based on the initial tone mapping curve.

[0031] In one possible implementation, before determining the first coordinates of at least one interpolation point based on feature information or dynamic metadata of the image to be processed, the method further includes: determining a target scene mode applicable to the image to be processed based on a mode indication value in the dynamic metadata, wherein the at least one interpolation point and the modified spline curve are related to the target scene mode.

[0032] For example, the number of interpolation points determined, and the number of first-order and cubic spline curves included in the correction spline curve, are related to the scene mode applicable to the image to be processed. When the image to be processed is generally dark, the first interpolation point is located in the region near the endpoint of the dark area, and the pixels between the origin and the first interpolation point are mapped by the first-order spline curve, which can better display the details of the dark area; when the image to be processed is generally bright, multiple interpolation points are located in the region near the endpoint of the bright area.

[0033] In one possible implementation, the initial tone mapping curve includes one of the following formulas:

[0034] ;

[0035] Where p, m, n, a, b, k1, k2, and k3 are all constants;

[0036] ;

[0037] ;

[0038] Where L represents the linear signal value, L' represents the nonlinear signal value, and p, a, m, b, c1, c2, c3, n, N, and ρ are all dynamic range mapping coefficients, with 0 ≤ L ≤ 1.

[0039] In the embodiments of this application, the method provided in the first aspect can be applied to a variety of initial tone mapping curves, and has great flexibility.

[0040] In one possible implementation, the plurality of interpolation points includes a first interpolation point, a second interpolation point, and a third interpolation point. The spline function includes a first spline function passing through the first interpolation point and the second interpolation point, and a second spline function passing through the second interpolation point and the third interpolation point. The first spline function includes a first coefficient MA1, a second coefficient MB1, a third coefficient MC1, and a fourth coefficient MD1. The second spline function includes a first coefficient MA2, a second coefficient MB2, a third coefficient MC2, and a fourth coefficient MD2.

[0041] In one possible implementation, the first interpolation point is not on the initial tone mapping curve, and determining the spline function passing through the plurality of interpolation points includes: determining the second coordinate VA1 of the first interpolation point on a linear spline function, wherein the linear spline function is a linear spline function between the origin and the first interpolation point; and determining the VA1 as the first coefficient MA1 of the first spline function.

[0042] In one possible implementation, the first interpolation point is located on the initial tone mapping curve, and the determination of the spline function passing through the plurality of interpolation points includes: determining the second coordinate VA1 of the first interpolation point on the initial tone mapping curve; and determining the VA1 as the first coefficient MA1 of the first spline function.

[0043] In one possible implementation, determining the spline function passing through the plurality of interpolation points further includes: determining the second coordinate VA3 of the third interpolation point on the initial tone mapping curve; determining the second coordinate VA2 of the second interpolation point on a first straight line, the first straight line being the straight line where the first interpolation point and the third interpolation point are located; and determining VA2 as the first coefficient MA2 of the second spline function.

[0044] In one possible implementation, the dynamic metadata further includes a second coordinate offset of the second interpolation point. The determination of the spline function passing through the plurality of interpolation points further includes: determining the second coordinate VA3 of the third interpolation point on the initial tone mapping curve; determining the second coordinate VA2 of the second interpolation point on a first straight line, the first straight line being the straight line where the first interpolation point and the third interpolation point are located; and determining the sum of VA2 and the second coordinate offset as the first coefficient MA2 of the second spline function.

[0045] In one possible implementation, determining the spline function passing through the plurality of interpolation points further includes: determining the first derivative of the linear spline function as MB1; and determining the first derivative of the initial tone mapping curve at the third interpolation point as MB2.

[0046] In one possible implementation, determining the spline function passing through the plurality of interpolation points further includes: determining the first derivative of the initial tone mapping curve at the first interpolation point as MB1; and determining the first derivative of the initial tone mapping curve at the third interpolation point as MB2.

[0047] In one possible implementation, determining the spline function passing through the plurality of interpolation points further includes: setting the second coordinate of the second interpolation point on the first spline function to the value of VA2, thus obtaining a first equation; setting the second coordinate of the third interpolation point on the second spline function to the value of VA3, thus obtaining a second equation; setting the first derivative of the second spline function at the third interpolation point to be equal to the first derivative of the initial tone mapping curve at the third interpolation point, thus obtaining a third equation; setting the first derivatives of the first spline function and the second spline function at the second interpolation point to be equal, thus obtaining a fourth equation; setting the second derivatives of the first spline function and the second spline function at the second interpolation point to be equal, thus obtaining a fifth equation; and analyzing the first equation, the second equation, the third equation, the fourth equation, and the fifth equation to obtain the third coefficient MC1 and the fourth coefficient MD1 of the first spline function, and the third coefficient MC2 and the fourth coefficient MD2 of the second spline function.

[0048] In one possible implementation, the first interpolation point and the third interpolation point are located on the initial tone mapping curve. Determining the spline function passing through the plurality of interpolation points includes: determining the second coordinate values ​​VA1 and VA3 of the first interpolation point and the third interpolation point respectively on the initial tone mapping curve; determining the second coordinate value VA2 corresponding to the second interpolation point on a first straight line, the first straight line being the straight line containing the first interpolation point and the third interpolation point; determining VA1 as the first coefficient MA1 of the first spline function and VA2 as the first coefficient MA2 of the second spline function; determining the slopes GD1 and GD3 of the initial tone mapping curve at the first interpolation point and the third interpolation point respectively, setting GD1 as the second coefficient MB1 of the first spline function and GD3 as the second coefficient MB2 of the second spline function; and making the second coordinate value of the second interpolation point on the first spline function VA1. 2. Obtain the first equation; make the second coordinate of the third interpolation point on the second spline function equal to VA3, thus obtaining the second equation; make the first derivative of the second spline function at the third interpolation point equal to GD3, thus obtaining the third equation; make the first derivatives of the first and second spline functions at the second interpolation point equal, thus obtaining the fourth equation; make the second derivatives of the first and second spline functions at the second interpolation point equal, thus obtaining the fifth equation; analyze the first, second, third, fourth, and fifth equations to obtain the third coefficient MC1 and fourth coefficient MD1 of the first spline function, and the third coefficient MC2 and fourth coefficient MD2 of the second spline function; determine the first spline function based on MA1, MB1, MC1, and MD1; determine the second spline function based on MA2, MB2, MC2, and MD2.

[0049] In one possible implementation, the first interpolation point is not on the initial tone mapping curve. Determining the spline function passing through the multiple interpolation points includes: determining the second coordinate VA1 of the first interpolation point on a linear spline function as the first coefficient MA1 of the first spline function, where the linear spline function is a straight line passing through the origin and the first interpolation point; determining the second coordinate VA3 of the third interpolation point on the initial tone mapping curve; determining the second coordinate VA2 of the second interpolation point on a first straight line as the first coefficient MA2 of the second spline function, where the first straight line is the straight line containing the first and third interpolation points; determining the first derivative of the linear spline function as MB1; determining the first derivative of the initial tone mapping curve at the third interpolation point as MB2; and making the second coordinate of the second interpolation point on the first spline function VA2, thus obtaining a first equation; and making... The second coordinate value of the third interpolation point on the second spline function is VA3, resulting in the second equation; the first derivative of the second spline function at the third interpolation point is GD3, resulting in the third equation; the first derivatives of the first and second spline functions at the second interpolation point are equal, resulting in the fourth equation; the second derivatives of the first and second spline functions at the second interpolation point are equal, resulting in the fifth equation; the first, second, third, fourth, and fifth equations are analyzed to obtain the third coefficient MC1 and fourth coefficient MD1 of the first spline function, and the third coefficient MC2 and fourth coefficient MD2 of the second spline function; the first spline function is determined based on MA1, MB1, MC1, and MD1; the second spline function is determined based on MA2, MB2, MC2, and MD2.

[0050] In one possible implementation, the dynamic metadata includes the start position L1 and the end position L3 of the spline function interval. The determination of multiple interpolation points based on the dynamic metadata includes: TH1=L1 / 255, TH3=L3 / 255, TH2=(TH1+TH3) / 2; wherein TH1 is the first coordinate of the first interpolation point, TH2 is the first coordinate of the second interpolation point, and TH3 is the first coordinate of the third interpolation point.

[0051] In one possible implementation, determining multiple interpolation points based on the brightness information of the target image includes: determining the first coordinate TH1 of the first interpolation point according to the following formula.

[0052]

[0053] Here, `maxL` is included in the dynamic metadata and represents the maximum brightness value of the image to be processed. When the image to be processed is received, the first coordinate TH1 of the first interpolation point is calculated based on `maxL` contained in the dynamic metadata of the image to be processed, combined with the above formula. `PvalueL` is the first threshold for dark area pixels, and `PvalueH` is the second threshold for the brightness value of dark area pixels, where the first threshold is less than the second threshold. For example, `PvalueL` can be 0.15, and `PvalueH` can be 0.25. `TPH` is the third threshold for bright area pixels, and `TPL` is the fourth threshold for bright area pixels. For example, `TPH` can be 0.6, and `TPL` can be 0.1. In an optional case, `maxL` in the above formula can be replaced with `avgL`, or `maxL` can be replaced with `a`. maxL+b avgL, where a and b are constants, and 0≤a≤1, 0≤b≤1.

[0054] In one possible implementation, before determining the second coordinate VA1 of the first interpolation point on the linear spline function, the method further includes:

[0055] Determine the first derivative MB0 of this linear spline function.

[0056]

[0057] Here, avgL is included in the dynamic metadata, and avgL is the average brightness of the image to be processed. When the image to be processed is received, the slope MB0 of the first line curve is calculated based on avgL contained in the dynamic metadata and the above formula. AvL is the fifth threshold for the average pixel value, and AvH is the sixth threshold for the average pixel value. For example, the value of AvL can be 0.15, and the value of AvH can be 0.65. SvalueL is the first slope threshold for the increase in brightness value of dark area pixels, and SvalueH is the second slope threshold for the increase in brightness value of dark area pixels. The first slope threshold is less than the second slope threshold. For example, the value of SvalueL can be 1, and the value of SvalueH can be 1.2. In an optional case, avgL in the above formula can be replaced with maxL, or replaced with a maxL+b avgL, where a and b are constants, and 0≤a≤1, 0≤b≤1.

[0058] In one possible implementation, the first identical function f1 is:

[0059] ;

[0060] The second spline function f2 is:

[0061] ;

[0062] Where TH1 is the first coordinate of the first interpolation point, TH2 is the first coordinate of the second interpolation point, and L is the dependent variable of the function.

[0063] In one possible implementation, the tone mapping of pixels in the target image located within the spline function interval according to the spline function includes: when the pixel to be processed is located between the first interpolation point and the second interpolation point, tone mapping is performed on the pixel to be processed according to the first spline function; when the pixel to be processed is located between the second interpolation point and the third interpolation point, tone mapping is performed on the pixel to be processed according to the second spline function.

[0064] In a second aspect, an image processing method is provided, comprising: determining a mode indicator value in dynamic metadata, the mode indicator value being used to indicate a target scene mode applicable to the image to be processed; invoking a process for obtaining modified spline curve parameters corresponding to the target scene mode according to the mode indicator value, and obtaining a spline modified curve; and performing tone mapping on the image to be processed according to the modified tone mapping curve, the modified tone mapping curve including at least a portion of an initial tone mapping curve and the modified spline curve.

[0065] The image processing method provided in this application determines the applicable scene mode for the image to be processed based on the mode indication value in the dynamic metadata, and flexibly corrects the initial tone mapping curve. This allows for adjustments to the image display effect for different brightness areas, enhancing the details of brightness variations and providing users with a better visual experience. By fine-tuning the tone mapping curve in various ways, display effects with different styles, such as soft, vivid, bright, and dusky, can be obtained.

[0066] In one possible implementation, the process of invoking the modified spline curve parameters corresponding to the target scene mode based on the mode indication value includes: invoking a first acquisition process when the mode indication value is a first preset value; invoking a second acquisition process when the mode indication value is a second preset value; invoking a third acquisition process when the mode indication value is a third preset value; invoking a fourth acquisition process when the mode indication value is a fourth preset value; and invoking a fifth acquisition process when the mode indication value is a fifth preset value.

[0067] In one possible implementation, the corrected spline curve obtained by the first acquisition process includes: a first spline curve, a second spline curve, and a third spline curve. The first spline curve is a linear spline function passing through the origin and a first interpolation point. The second spline curve is a cubic spline curve passing through the first and second interpolation points. The third spline curve is a cubic spline curve passing through the second and third interpolation points. Calling the first acquisition process includes: determining the first coordinate TH1 of the first interpolation point based on at least one of the maximum brightness value maxL and the average brightness value avgL of the image to be processed included in the dynamic metadata; obtaining the first derivative MB0 of the first spline curve based on at least one of the average brightness value avgL and the maximum brightness value maxL of the image to be processed included in the dynamic metadata; TH2 = TH1 + B; TH3 = TH2 + C. TH2-D CH2; where TH2 is the first coordinate of the second interpolation point, TH3 is the first coordinate of the third interpolation point, and B, C, and D are constants. Optionally, B, C, and D can be carried in the dynamic metadata; calculate the second coordinate VA1 of the first interpolation point on the first line curve based on TH1 and MB0; calculate the second coordinate VA3 of the third interpolation point on the initial tone mapping curve based on TH3; calculate the second coordinate VA2 of the second interpolation point on the first straight line based on TH2, where the first straight line is the line containing the first interpolation point and the third interpolation point; calculate the first derivative GD3 of the initial tone mapping curve at the third interpolation point based on TH3; determine VA1 as the first coordinate of the third interpolation point. The first coefficient MA1 of the second spline curve is determined; MB0 is determined as the second coefficient MB1 of the second spline curve; the sum of VA2 and the offset value is determined as the first coefficient MA2 of the third spline curve, where the offset value can be 0, positive, or negative; a system of equations containing five equations is constructed based on TH1, TH2, TH3, VA2, VA3, GD3 and the derivative relationship between the second spline curve and the third spline curve at the second interpolation point; the system of equations containing five equations is analyzed to obtain MC1, MD1, MB2, MC2 and MD2.

[0068] The first acquisition process corresponds to a relatively dark overall scene in the image. In this case, the dark areas are corrected to better reveal the details in the darker parts of the image, allowing pixels with brightness below the display device's capability to have a better display effect. Specifically, pixels below a first threshold (TH1) are tone-mapped using a linear spline function formula, better preserving the dark detail information in that area. Then, a cubic spline function is used to achieve a smooth transition between the linear spline function and the initial tone-mapped curve, avoiding the influence of an uneven tone-mapped curve on the tone-mapped effect. It should be understood that in this embodiment, there are two cubic spline curves. In an optional case, there may be only one cubic spline curve. The first cubic spline curve is smoothly connected to the initial tone-mapped curve through a cubic spline function. In this case, only two interpolation points need to be determined.

[0069] The first sample curve increases the brightness of the dark areas compared to the base curve, which is equivalent to allocating more bits to the dark areas. The two cubic spline functions achieve a smooth transition between the first sample curve and the initial tone mapping curve, and the enhancement or reduction of the correction area can be achieved by controlling the points and slopes of the curves.

[0070] In one possible implementation, the corrected spline curve obtained by the second acquisition process includes: a first spline curve and a second spline curve. The first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a cubic spline curve passing through the second interpolation point and the third interpolation point. The second acquisition process includes: acquiring the first coordinate TH1 of the first interpolation point, the first coordinate TH2 of the second interpolation point, and the first coordinate TH3 of the third interpolation point based on the dynamic metadata; calculating the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on TH1 and TH3, respectively; calculating the second coordinate VA2 of the second interpolation point on a first straight line based on TH2, and the first straight line... The line is the straight line containing the first interpolation point and the third interpolation point; calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 at the third interpolation point; determine VA1 as the first coefficient MA1 of the first spline curve and determine GD1 as the second coefficient MB1 of the first spline curve; determine VA2 and the offset value as the first coefficient MA2 of the second spline curve, where the offset value is 0, positive, or negative; construct a system of equations containing five equations based on TH1, TH2, TH3, VA2, VA3, GD3 and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point; parse the system of equations containing five equations to obtain MC1, MD1, MB2, MC2 and MD2.

[0071] The second acquisition process corresponds to images containing pixels located within a specific brightness range. At this point, a focused correction is applied to this specific brightness range to better showcase the brightness details of the image within that range. Specifically, tone mapping is performed on the pixels between the first and second interpolation points, as well as between the second and third interpolation points, based on a cubic spline function, improving the display effect of these pixels. Furthermore, the transitions between the first and second spline curves, and between the first and second spline curves and the initial tone mapping curve, are smooth to avoid the influence of uneven tone mapping curves on the tone mapping effect.

[0072] Specifically, this correction method can correct any range, and the correction curve of the corrected range is smoothly connected to the initial tone mapping curve. It can also improve image contrast or increase or decrease the brightness value of some pixels by controlling the interpolation points and offset values, thereby achieving the desired effect.

[0073] In one possible implementation, the corrected spline curve obtained by the third acquisition process includes: a first spline curve, which is a first-order spline curve passing through the first interpolation point. Calling the third acquisition process includes: acquiring the first coordinate TH1 of the first interpolation point based on the dynamic metadata; determining the second coordinate VA1 of the first interpolation point on the initial tone mapping curve based on TH1, and the first derivative GD1 of the initial tone mapping curve at the first interpolation point; determining GD1 as the second coefficient MB0 of the first spline curve; and determining VA1 as the first coefficient MA0 of the first spline curve.

[0074] The third acquisition process corresponds to a situation where the overall scene of the image is relatively bright. At this time, the bright areas are corrected to better show the bright details of the image. Specifically, the pixel areas with a value greater than a preset threshold (TH1) are tone mapped based on a linear function, and the first linear curve is smoothly connected to the initial tone mapping curve at the first interpolation point to avoid the tone mapping effect being affected by an unsmooth tone mapping curve.

[0075] Specifically, a linear spline curve is used to perform tone mapping on pixels larger than the first interpolation point, meaning that all points after TH1 are compressed using the compression rate of the first interpolation point, avoiding over-compression of bright pixels and thus improving the image brightness value.

[0076] In one possible implementation, the corrected spline curve obtained by the fourth acquisition process includes: a first spline curve, which is a cubic spline curve passing through the first interpolation point and the second interpolation point; the fourth acquisition process includes: acquiring the first coordinate TH1 of the first interpolation point and the first coordinate TH2 of the second interpolation point based on the dynamic metadata; calculating the second coordinates VA1 and VA2 of the first interpolation point and the second interpolation point on the initial tone mapping curve based on TH1 and TH2 respectively; and calculating the first coordinate VA1 and VA2 of the initial tone mapping curve on the first interpolation point and the second interpolation point based on TH1 and TH2 respectively. The first derivative GD1 at the first interpolation point and the first derivative GD2 at the second interpolation point are used to make the second coordinate of the second interpolation point on the first homogeneous curve the sum of VA2 and S1, thus obtaining the first equation, where S1 is the coordinate offset value included in the dynamic metadata; the first derivative of the first homogeneous curve at the second interpolation point is the sum of GD2 and S2, thus obtaining the second equation, where S2 is the slope offset value included in the dynamic metadata; the equation system consisting of the first equation and the second equation is analyzed to obtain the third coefficient MC1 and the fourth coefficient MD1 of the first homogeneous curve.

[0077] The fourth acquisition process corresponds to a relatively bright overall scene in the image. In this case, the bright areas are corrected to better show the bright details of the image. Specifically, the pixel area between the first interpolation point and the second interpolation point is tone mapped based on the cubic spline function. The cubic spline function is smoothly connected to the initial tone mapping curve at the first interpolation point to avoid the tone mapping effect being affected by the non-smooth tone mapping curve.

[0078] Specifically, the brightness value of the pixel between the first interpolation point and the endpoint of the bright area pixel can be increased or decreased. This can satisfy the needs of increasing brightness to achieve a more dizzying effect, or decreasing brightness to enhance details in bright areas. Furthermore, increasing and decreasing brightness is a non-linear process, which can improve contrast.

[0079] In one possible implementation, the corrected spline curve obtained by the fifth acquisition process includes: a first spline curve and a second spline curve. The first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a linear spline curve passing through the second interpolation point and the third interpolation point. Calling the fifth acquisition process includes: acquiring the first coordinate TH1 of the first interpolation point, the first coordinate TH2 of the second interpolation point, and the first coordinate TH3 of the third interpolation point based on the dynamic metadata; calculating the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on TH1 and TH3, respectively; and calculating the first coordinates VA1 and VA3 of the second interpolation point on the first linear spline curve based on TH2. The second coordinate VA2 on the line is given, and the first straight line is the line containing the first interpolation point and the third interpolation point. The first derivative GD1 of the initial tone mapping curve at the first interpolation point is calculated. VA1 is determined as the first coefficient MA1 of the first spline curve. GD1 is determined as the second coefficient MB1 of the first spline curve. The sum of VA2 and the coordinate offset value contained in the dynamic metadata is determined as the first coefficient MA2 of the second spline curve. Based on TH1, TH2, TH3, VA2, VA3 and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point, a system of equations containing three equations is constructed. The system of equations containing three equations is analyzed to obtain MC1, MD1 and MB2.

[0080] In one optional case, the brightness value of the first interpolation point > the brightness value of the second interpolation point > the brightness value of the third interpolation point.

[0081] In one optional case, the brightness value of the first interpolation point < the brightness value of the second interpolation point < the brightness value of the third interpolation point.

[0082] The fifth acquisition process corresponds to a relatively bright overall scene in the image. At this time, the bright areas are corrected to better show the bright details of the image. Specifically, tone mapping is performed on the pixel area between the second and third interpolation points based on the first spline curve, and the first spline function is smoothly connected to the initial tone mapping curve based on the cubic spline function between the first and second interpolation points to avoid the tone mapping effect being affected by the non-smooth tone mapping curve.

[0083] Specifically, in this scenario, a cubic spline function can be used between the first and second interpolation points to increase and decrease brightness on the initial tone mapping curve, and this increase and decrease are non-linear. Tone mapping between the second and third interpolation points is based on a straight line, which allows for the allocation of more bits to pixels between these points, thereby improving detail in bright areas.

[0084] In one possible implementation, the initial tone mapping curve includes one of the following formulas:

[0085] ;

[0086] ;

[0087] ;

[0088] Where L represents the linear signal value, L' represents the nonlinear signal value, and p, a, m, b, c1, c2, c3, n, N, and ρ are all dynamic range mapping coefficients, with 0 ≤ L ≤ 1.

[0089] Thirdly, this application provides an image processing apparatus, including modules for performing the first aspect or any possible implementation thereof.

[0090] Fourthly, this application provides an image processing apparatus, comprising: a determining module, configured to determine a mode indication value in dynamic metadata, the mode indication value being used to indicate a target scene mode applicable to the image to be processed; the determining module is further configured to invoke a process for obtaining modified spline curve parameters corresponding to the target scene mode according to the mode indication value, thereby obtaining a modified spline curve; and a tone mapping module, configured to perform tone mapping on the image to be processed according to the modified tone mapping curve, the modified tone mapping curve including at least a portion of an initial tone mapping curve and the modified spline curve.

[0091] In one possible implementation, the determining module is specifically used to: call a first acquisition process when the mode indication value is a first preset value; call a second acquisition process when the mode indication value is a second preset value; call a third acquisition process when the mode indication value is a third preset value; call a fourth acquisition process when the mode indication value is a fourth preset value; and call a fifth acquisition process when the mode indication value is a fifth preset value.

[0092] In one possible implementation, the corrected spline curve obtained by the first acquisition process includes: a first spline curve, a second spline curve, and a third spline curve. The first spline curve is a linear spline function passing through the origin and a first interpolation point. The second spline curve is a cubic spline curve passing through the first and second interpolation points. The third spline curve is a cubic spline curve passing through the second and third interpolation points. The determining module is specifically used to: determine the first coordinate TH1 of the first interpolation point based on at least one of the maximum brightness value maxL and the average brightness value avgL of the image to be processed included in the dynamic metadata; obtain the first derivative MB0 of the first spline curve based on at least one of the maximum brightness value maxL and the average brightness value avgL of the image to be processed included in the dynamic metadata; TH2 = TH1 + B; TH3 = TH2 + C TH2-D CH2; where TH2 is the first coordinate of the second interpolation point, TH3 is the first coordinate of the third interpolation point, and B, C, and D are constants, optionally, B, C, and D are constants contained in the dynamic metadata; calculate the second coordinate VA1 of the first interpolation point on the first line curve based on TH1 and MB0; calculate the second coordinate VA3 of the third interpolation point on the initial tone mapping curve based on TH3; calculate the second coordinate VA2 of the second interpolation point on the first straight line based on TH2, the first straight line being the line containing the first interpolation point and the third interpolation point; calculate the initial tone mapping curve based on TH3. The first derivative GD3 at the third interpolation point is determined; VA1 is determined as the first coefficient MA1 of the second spline curve; MB0 is determined as the second coefficient MB1 of the second spline curve; the sum of VA2 and the offset value is determined as the first coefficient MA2 of the third spline curve, where the offset value can be 0, positive, or negative; a system of equations containing five equations is constructed based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the second and third spline curves at the second interpolation point; the system of equations containing five equations is analyzed to obtain MC1, MD1, MB2, MC2, and MD2.

[0093] In one possible implementation, the corrected spline curve obtained by the second acquisition process includes: a first spline curve and a second spline curve. The first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a cubic spline curve passing through the second interpolation point and the third interpolation point. The determining module is specifically used to: obtain the first coordinate TH1 of the first interpolation point, the first coordinate TH2 of the second interpolation point, and the first coordinate TH3 of the third interpolation point according to the dynamic metadata; calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve according to TH1 and TH3, respectively; calculate the second coordinate VA2 of the second interpolation point on the first straight line according to TH2, and the first straight line... The line is the straight line containing the first interpolation point and the third interpolation point; calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 at the third interpolation point; determine VA1 as the first coefficient MA1 of the first spline curve and determine GD1 as the second coefficient MB1 of the first spline curve; determine VA2 and the offset value as the first coefficient MA2 of the second spline curve, where the offset value is 0, positive, or negative; construct a system of equations containing five equations based on TH1, TH2, TH3, VA2, VA3, GD3 and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point; parse the system of equations containing five equations to obtain MC1, MD1, MB2, MC2 and MD2.

[0094] In one possible implementation, the corrected spline curve obtained by the third acquisition process includes: a first spline curve, which is a first-order spline curve passing through the first interpolation point. The determining module is specifically used to: obtain the first coordinate TH1 of the first interpolation point according to the dynamic metadata; determine the second coordinate VA1 of the first interpolation point on the initial tone mapping curve according to TH1, and the first derivative GD1 of the initial tone mapping curve at the first interpolation point; determine GD1 as the second coefficient MB0 of the first spline curve; and determine VA1 as the first coefficient MA0 of the first spline curve.

[0095] In one possible implementation, the corrected spline curve obtained by the fourth acquisition process includes: a first spline curve, which is a cubic spline curve passing through the first interpolation point and the second interpolation point. The determining module is specifically configured to: acquire the first coordinate TH1 of the first interpolation point and the first coordinate TH2 of the second interpolation point based on the dynamic metadata; calculate the second coordinates VA1 and VA2 of the first interpolation point and the second interpolation point on the initial tone mapping curve based on TH1 and TH2, respectively; and calculate the second coordinates VA1 and VA2 of the initial tone mapping curve on the first... The first derivative GD1 at the interpolation point and the first derivative GD2 at the second interpolation point are used to make the second coordinate of the second interpolation point on the first homogeneous curve the sum of VA2 and S1, thus obtaining the first equation, where S1 is the coordinate offset value included in the dynamic metadata; the first derivative of the first homogeneous curve at the second interpolation point is the sum of GD2 and S2, thus obtaining the second equation, where S2 is the slope offset value included in the dynamic metadata; the equation system consisting of the first equation and the second equation is analyzed to obtain the third coefficient MC1 and the fourth coefficient MD1 of the first homogeneous curve.

[0096] In one possible implementation, the corrected spline curve obtained in the fifth acquisition process includes: a first spline curve and a second spline curve. The first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a linear spline curve passing through the second interpolation point and the third interpolation point. The determining module is specifically used to: acquire the first coordinate TH1 of the first interpolation point, the first coordinate TH2 of the second interpolation point, and the first coordinate TH3 of the third interpolation point according to the dynamic metadata; calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve according to TH1 and TH3, respectively; and calculate the second interpolation point on the first straight line according to TH2. The second coordinate VA2 on the first line is the line where the first interpolation point and the third interpolation point are located; the first derivative GD1 of the initial tone mapping curve at the first interpolation point is calculated; VA1 is determined as the first coefficient MA1 of the first spline curve; GD1 is determined as the second coefficient MB1 of the first spline curve; the sum of VA2 and the offset value is determined as the first coefficient MA2 of the second spline curve, where the offset value is 0, positive, or negative; a system of equations containing three equations is constructed based on TH1, TH2, TH3, VA2, VA3 and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point; the system of equations containing three equations is analyzed to obtain MC1, MD1, and MB2. In a fourth aspect, this application provides an image processing apparatus, including a processor and a transmission interface, the transmission interface being used to receive or transmit image data; the processor being used to call program instructions stored in a memory and execute the program instructions to perform the method of the first aspect or any implementation thereof.

[0097] For example, the image processing device can be a chip.

[0098] Fifthly, this application provides an image processing apparatus, including a processor and a transmission interface, wherein the transmission interface is used to receive or transmit image data; and the processor is used to call program instructions stored in a memory and execute the program instructions to perform the method of the first aspect or any possible implementation of the first aspect.

[0099] In one possible implementation, the image processing apparatus may also include a memory.

[0100] In a sixth aspect, this application provides an image processing apparatus, including a processor and a transmission interface, the transmission interface being used to receive or transmit image data; the processor being used to call program instructions stored in a memory and execute the program instructions to perform the method of the second aspect or any possible implementation of the second aspect.

[0101] In a seventh aspect, this application provides an image processing apparatus, including a processor and a memory, for performing the method described in the first aspect or any of the methods described in the first aspect.

[0102] Eighthly, this application provides an image processing apparatus, including a processor and a memory, for performing the method described in the second aspect or any of the methods described in the second aspect.

[0103] Ninthly, this application provides a computer-readable storage medium storing program instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in the first aspect or any implementation thereof.

[0104] In a tenth aspect, this application provides a computer-readable storage medium storing program instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in the second aspect or any implementation thereof.

[0105] In one aspect, this application provides a computer program product, including a computer program that, when executed on a computer or processor, will cause the computer or processor to perform the method described in the first aspect or any implementation thereof.

[0106] In a twelfth aspect, this application provides a computer program product, including a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method described in the second aspect or any implementation thereof. Attached Figure Description

[0107] Figure 1 This is a schematic block diagram of an image display system provided in an embodiment of this application;

[0108] Figure 2 This is a hardware architecture diagram of an exemplary image processing apparatus provided in an embodiment of this application;

[0109] Figure 3 This is a schematic flowchart of an exemplary image processing method provided in an embodiment of this application;

[0110] Figure 4a This is a schematic diagram of an exemplary modified spline curve provided in an embodiment of this application;

[0111] Figure 4b This is a schematic diagram of an exemplary modified spline curve provided in an embodiment of this application;

[0112] Figure 4c This is a schematic diagram of an exemplary modified spline curve provided in an embodiment of this application;

[0113] Figure 4d This is a schematic diagram of an exemplary modified spline curve provided in an embodiment of this application;

[0114] Figure 4e This is a schematic diagram of an exemplary modified spline curve provided in an embodiment of this application;

[0115] Figure 4f These are schematic diagrams of several exemplary initial tone mapping curves provided in the embodiments of this application;

[0116] Figure 5 This is a flowchart illustrating an exemplary method for obtaining parameters of a modified spline curve, provided in an embodiment of this application.

[0117] Figure 6a This is a flowchart illustrating an exemplary method for obtaining parameters of a modified spline curve, provided in an embodiment of this application.

[0118] Figure 6b This application provides an exemplary flowchart of a method for obtaining parameters of a modified spline curve;

[0119] Figure 7 This is a flowchart illustrating an exemplary method for obtaining parameters of a modified spline curve, provided in an embodiment of this application.

[0120] Figure 8 This is a flowchart illustrating an exemplary method for obtaining parameters of a modified spline curve, provided in an embodiment of this application.

[0121] Figure 9 This is a flowchart illustrating an exemplary method for obtaining parameters of a modified spline curve, provided in an embodiment of this application.

[0122] Figure 10 This is a flowchart illustrating an exemplary method for obtaining parameters of a modified spline curve, provided in an embodiment of this application.

[0123] Figure 11 This is a structural block diagram of an exemplary image processing apparatus provided in an embodiment of this application. Detailed Implementation

[0124] The terms "first," "second," etc., in the specification, embodiments, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0125] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0126] Image acquisition devices can generate images or videos containing natural scenes by receiving light signals from them. To facilitate image or video transmission, the light signals need to be converted into electrical signals, and the image information of each pixel is recorded using fixed-range brightness or chromaticity values ​​(e.g., grayscale values ​​between 0 and 255). The curve that converts light signals into electrical signals is called the photoelectric transfer function. Common photoelectric transfer functions include the PQ (perceptual quantizer) photoelectric transfer function, the HLG (hybrid log-gamma) photoelectric transfer function, and the scene luminance fidelity (SLF) photoelectric transfer function. Display devices can reproduce natural scenes by displaying images or videos. In one optional scenario, the image acquisition device is called the front-end device, and the display device is called the back-end device. The display device can determine the brightness of an object at the time of capture based on the photoelectric transfer function and the brightness or grayscale values ​​of each pixel in the image; that is, the electronic device can convert YUV or RGB information into brightness in nits. However, the brightness of the object may exceed the display capability of the display device. Because the brightness information acquired by the image acquisition device does not match the brightness display capability of the display device, there are situations where a high-brightness image is displayed on a display device with low brightness display capability, and vice versa. Both situations require tone mapping of the image acquired by the image acquisition device to ensure the image matches the display capability of the display device. For example, a high dynamic range image can be tone-mapped onto a display device with low dynamic range display capability based on a tone mapping curve. It should be understood that this tone mapping curve can be determined by the display device or by a processing device external to the display device.

[0127] However, in some cases, the display quality of high dynamic range (HDR) images is poor when the tone mapping curve is inaccurate. For example, during video playback, if only one tone mapping curve is determined for the video, the differences in image content displayed between different video frames may be significant. Some video frames might display very dark images, while others might display very bright images. If a fixed tone mapping curve is used for the entire video, it will be impossible to perfectly reproduce every frame on the display device. Furthermore, tone mapping curves carry a large amount of data; if a tone mapping curve is determined for each image or frame, it may introduce data redundancy.

[0128] This application provides an image processing method and apparatus that flexibly corrects the initial tone mapping curve according to the characteristics of the scene, so that the image tone mapping in different scenes can achieve a better or more subjective display effect.

[0129] like Figure 1 The diagram shown is a schematic block diagram of an exemplary image display system 10 provided in an embodiment of this application. The image display system 10 may include an image acquisition device 12, an image processing device 14, and a display device 34. For example, the image acquisition device 12 may acquire HDR images or Standard Dynamic Range (SDR) images. The image processing device 14 may process the HDR images or SDR images acquired by the image acquisition device 12.

[0130] Image acquisition device 12 and image processing device 14 can be communicatively connected via link 13, and image processing device 14 can receive image data from image acquisition device 12 via link 13. Link 13 may include one or more communication media or devices. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. Optionally, the one or more communication media may form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from image acquisition device 12 to image processing device 14. In an optional case, link 13 may be a Bluetooth wireless link.

[0131] For example, the image acquisition device 12 includes an image source 16 and a communication interface 22, which may also be called a transmission interface. Alternatively, the image acquisition device 12 may also include an image processor 20. In specific implementations, the image processor 20, the image source 16, and the communication interface 22 may be hardware components of the image acquisition device 12, or they may be software programs within the image acquisition device 12; alternatively, the communication interface 22 may be a combination of a hardware module and a software program.

[0132] Image source 16 may include or be any type of image capture device for, for example, capturing real-world images, and / or any type of image or commentary (for screen content encoding, some text on the screen is also considered part of the image or picture to be encoded) generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). For example, image source 16 may be a camera for capturing images or a memory for storing images. Image source 16 may also include any type of (internal or external) interface for storing previously captured or generated images and / or acquiring or receiving images. When image source 16 is a camera, image source 16 may be, for example, a local or integrated camera within an image acquisition device; when image source 16 is a memory, image source 16 may be a local or integrated memory within an image acquisition device. When the image source 16 includes an interface, the interface may be, for example, an external interface for receiving images from an external video source. The external video source may be, for example, an external image capture device, such as a camera, external storage, or an external image generation device. The external image generation device may be, for example, an external computer graphics processor, a computer, or a server.

[0133] An image can be viewed as a two-dimensional array or matrix of pixels. The pixels in the array can also be called sampling points. In one alternative case, to represent color, each pixel includes three color components. For example, in RGB format or color space, an image includes corresponding red, green, and blue sampling arrays. However, in video encoding, each pixel is typically represented in a luma / chroma format or color space. For example, for a YUV format image, it includes a luma component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The luma component Y represents the brightness or grayscale level intensity (e.g., both are the same in a grayscale image), while the two chroma components U and V represent chroma or color information components. Accordingly, a YUV format image includes a luma sampling array of luma sample values ​​(Y) and two chroma sampling arrays of chroma values ​​(U and V). RGB format images can be converted or transformed to YUV format, and vice versa; this process is also called color conversion or color format conversion. If the image is black and white, it may only include a luma sampling array. In this embodiment of the application, the image transmitted from the image source 16 to the image processor can also be referred to as the original image data 17.

[0134] Image processor 20 is used to perform image processing, such as brightness mapping, tone mapping, color format conversion (e.g., from RGB to YUV format), color gamut conversion, saturation adjustment, color correction, resolution adjustment, or noise reduction.

[0135] The communication interface 22 can be used to receive image data 19 after image processing, and can transmit the image data 19 to the image processing device 14 for further image processing via the link 13, or to the memory for storage. For example, the communication interface 22 can be used to encapsulate the image data 19 into a suitable format, such as a data packet, for transmission over the link 13.

[0136] The image processing device 14 includes a communication interface 28, an image processor 32, and a display device 34. These are described below:

[0137] Communication interface 28 can be used to receive image-processed image data 19 from image acquisition device 12 or any other source, such as a storage device. Specific examples of communication interfaces 22 and 28 can be found in the foregoing description of the interfaces, and will not be repeated here. Communication interface 28 can be used to transmit or receive image-processed image data 19 via link 13 between image acquisition device 12 and image processing device 14 or any other type of network. Communication interface 28 can, for example, be used to decapsulate data packets transmitted by communication interface 22 to obtain image-processed image data 19.

[0138] Both communication interfaces 28 and 22 can be configured as unidirectional or bidirectional communication interfaces, and can be used, for example, to send and receive messages to establish connections, acknowledge and exchange any other information related to the communication link and / or, for example, image data processed by image processing and / or data transmission. For example, communication interfaces 28 and 22 can be any type of interface according to any proprietary or standardized interface protocol, such as High Definition Multimedia Interface (HDMI), Mobile Industry Processor Interface (MIPI), MIPI-standardized Display Serial Interface (DSI), Video Electronics Standards Association (VESA) standardized Embedded Display Port (eDP), Display Port (DP), or V-By-One interface, a digital interface standard developed for image transmission, as well as various wired or wireless interfaces, optical interfaces, etc.

[0139] Image processor 32 is used to perform tone mapping processing on the image data 19 after image processing to obtain tone-mapped image data 33. The processing performed by image processor 32 may also include: super-resolution, color format conversion (e.g., from YUV format to RGB format), noise reduction, color gamut conversion, saturation adjustment, brightness mapping, upsampling, downsampling, and image sharpening, and may also be used to transmit the tone-mapped image data 33 to display device 34. It should be understood that image processors 20 and 32 can be general-purpose central processing units (CPUs), systems on chips (SOCs), processors integrated on SOCs, stand-alone processor chips, or controllers, etc.; image processors 20 and 32 can also be dedicated processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), dedicated video or graphics processors, graphics processing units (GPUs), and neural-network processing units (NPUs), etc. Image processors 20 and 32 can also be processor groups consisting of multiple processors, with the multiple processors coupled to each other through one or more buses.

[0140] Display device 34 is configured to receive tone-mapped image data 33 for displaying the image to a user or viewer. Display device 34 can be or may include any class of display for presenting a reconstructed image, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other class of display. Alternatively, display device 34 may have its own image processing capabilities, and tone mapping of the image may also be performed at the display device.

[0141] It should be understood that Figure 1Image acquisition device 12, image processing device 14, and display device 34 are illustrated as separate devices. In one optional case, a single image processing device may simultaneously possess the functions of both image acquisition device 12 and image processing device 14. In another optional case, a single image processing device may simultaneously possess the functions of both image processing device 14 and display device 34. In yet another optional case, a single image processing device may simultaneously possess the functions of image acquisition device 12, image processing device 14, and display device 34. For example, a smartphone may have a camera, an image processor, and a display screen, where the camera corresponds to image acquisition device 12, the image processor corresponds to image processing device 14, and the display screen corresponds to display device 34. Similarly, a smart TV may have a webcam, an image processor, and a display screen, where the webcam corresponds to image acquisition device 12, the image processor corresponds to image processing device 14, and the display screen corresponds to display device 34.

[0142] In one optional embodiment, the image processing device includes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a CPU, memory management unit (MMU), and memory (also known as main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software. Furthermore, this application embodiment does not specifically limit the specific structure of the execution subject of the method provided in this application embodiment, as long as it can perform image processing by running relevant code. For example, the execution subject of the method provided in this application embodiment can be an electronic device, or a functional module in an electronic device capable of calling and executing programs, such as a processor in an electronic device.

[0143] It should be understood that the image acquisition device 12 and the image processing device 14 may include any type of handheld or stationary device, such as a laptop or laptop computer, mobile phone, smartphone, tablet or tablet computer, camera, desktop computer, set-top box, television, camera, in-vehicle device, display device, digital media player, video game console, video streaming device (e.g., content service server or content distribution server), broadcast receiver device, broadcast transmitter device, etc., and may or may not use any type of operating system.

[0144] like Figure 2The diagram shown is a hardware architecture diagram of an exemplary image processing device provided in an embodiment of this application. The image processing device 200 can be, for example, a processor chip. Figure 2 The hardware architecture diagram shown may be Figure 1 An exemplary architecture diagram of the image processor 20 or image processor 32 is provided. The image processing method provided in this application embodiment can be applied to the processor chip.

[0145] refer to Figure 2 The device 200 includes: at least one CPU, a microcontroller unit (MCU), a GPU, an NPU, a memory bus, a receive interface, and a transmit interface, etc. Although Figure 2 As not shown, the device 200 may further include an application processor (AP), memory, decoder, and dedicated video or image processor. The various components of the device 200 are coupled together via connectors. Exemplarily, connectors include various interfaces, transmission lines, or buses, which are typically electrical communication interfaces, but may also be mechanical interfaces or other forms of interfaces; this embodiment does not limit the specific type of interface.

[0146] Optionally, the CPU can be a single-core or multi-core processor; alternatively, the CPU can be a processor group consisting of multiple processors, which are coupled to each other through one or more buses. The receiving interface can be a data input interface for the processor chip. In an optional case, the types of the receiving and transmitting interfaces can be referred to the descriptions in sections 28 and 22 of the communication interface, which will not be repeated here.

[0147] The memory is coupled to the CPU, GPU, MCU and NPU through the memory bus. The memory can be used to store computer program instructions, including the computer operating system (OS) and various user applications. The memory is also used to store image data or video data. The memory can be non-volatile memory, such as an embedded multi-media card (EMMC), universal flash storage (UFS), or read-only memory (ROM), or other types of static storage devices capable of storing static information and instructions. It can also be volatile memory, such as random access memory (RAM), or other types of dynamic storage devices capable of storing information and instructions. Furthermore, it can be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), disk storage media, or other magnetic storage devices, or any other computer-readable storage medium capable of carrying or storing program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. It should be understood that the memory can be independent of the image processing device 200.

[0148] In one optional scenario, the aforementioned components are integrated onto the same chip; in another optional scenario, the CPU, GPU, decoder, receiving interface, and transmitting interface are integrated onto a single chip, with each component within the chip accessing external memory via a bus. A dedicated video / graphics processor can be integrated onto the same chip as the CPU, or it can exist as a separate processor chip; for example, a dedicated video / graphics processor can be a dedicated ISP. In one optional scenario, the NPU can also be a separate processor chip. This NPU is used to implement various neural network or deep learning related operations. Optionally, the image processing method and image processing framework provided in this application embodiment can be implemented by a GPU or NPU, or by a dedicated graphics processor.

[0149] The chip involved in this application embodiment is a system manufactured on the same semiconductor substrate using integrated circuit technology, also called a semiconductor chip. It can be a collection of integrated circuits formed on a substrate (usually a semiconductor material such as silicon) using integrated circuit technology, and its outer layer is typically encapsulated by semiconductor packaging materials. The integrated circuit can include various functional devices, each including logic gates, metal-oxide-semiconductor (MOS) transistors, bipolar transistors, or diodes, and may also include other components such as capacitors, resistors, or inductors. Each functional device can operate independently or under the action of necessary driving software, and can realize various functions such as communication, computation, or storage.

[0150] like Figure 3 The diagram shown is a schematic flowchart of an image processing method provided in an embodiment of this application. It should be understood that, for ease of description, Figure 3 The method is described in steps, although in the method flow Figure 3 The order of the methods is shown, but in some cases, the steps described may be performed in a different order than that shown here.

[0151] The image processing method includes:

[0152] 301. Determine the mode indication value in the dynamic metadata, which is used to indicate the target scene mode applicable to the image to be processed;

[0153] It should be understood that dynamic metadata is generated by the video front-end device when generating image or video data. This dynamic metadata includes a mode indicator value; for example, this mode indicator value can be obtained by the front-end device through analysis of the acquired or generated image data. The video front-end device can be, for example... Figure 1The image acquisition device 12 shown. Exemplarily, when acquiring or generating image data, the image acquisition device 12 performs image frame analysis on each frame of image data to obtain feature information for each frame. Further, it can determine the tone mapping mode suitable for the image data based on the obtained feature information. For example, if the overall scene of the current image is relatively dark, it is confirmed that the image frame is suitable for a dark scene tone mapping mode; if the overall scene of the current image is relatively bright, it is confirmed that the image frame is suitable for a bright scene tone mapping mode; if the current image contains pixels located within a specific brightness range, it is confirmed that the image frame is suitable for a specific scene tone mapping mode. Further, after receiving the image to be processed and its dynamic metadata, the image processing device 14 or display device 34 adaptively corrects the initial tone mapping curve according to the mode indication value in the dynamic metadata, and performs tone mapping on the image to be processed based on the corrected tone mapping curve. In an optional case, the set-top box can be referred to as a playback device. It should be understood that the execution entity of step 301 can be... Figure 1 The image processing device 14 or display device 34 shown in the figure can be, for example, a set-top box or an image processor in a set-top box, or an image processor or image processing unit in a smart mobile terminal with playback and display capabilities.

[0154] 302. Based on the mode indication value, call the process to obtain the modified spline curve parameters corresponding to the target scene mode, and obtain the spline curve;

[0155] Specifically,

[0156] When the mode indicator value is the first preset value, the first acquisition process is invoked;

[0157] When the mode indicator value is the second preset value, the second acquisition process is invoked;

[0158] When the mode indicator value is the third preset value, the third acquisition process is invoked;

[0159] When the mode indicator value is the fourth preset value, the fourth acquisition process is invoked;

[0160] When the mode indicator value is the fifth preset value, the fifth acquisition process is invoked.

[0161] For example, the mode indicator value included in the dynamic metadata is set to 3Spline_num.

[0162] If 3Spline_num equals 0, then call spline curve parameter acquisition process 0 and obtain the corrected spline curve parameters corresponding to the first scene mode;

[0163] If 3Spline_num equals 1, then call spline curve parameter acquisition process 1 and obtain the corrected spline curve parameters corresponding to the second scene mode;

[0164] If 3Spline_num equals 2, then call spline curve parameter acquisition process 2 and obtain the corrected spline curve parameters corresponding to the third scene mode;

[0165] If 3Spline_num equals 3, then call spline curve parameter acquisition process 3 and obtain the corrected spline curve parameters corresponding to the fourth scene mode;

[0166] If 3Spline_num equals 4, then call spline curve parameter acquisition process 4 and obtain the corrected spline curve parameters corresponding to the fifth scene mode.

[0167] In an optional case, the value of 3Spline_num can also be any other integer greater than or equal to 5, corresponding to other spline curve parameter acquisition processes; this embodiment of the application does not limit this. Optionally, the value of 3Spline_num can also be an English letter, a Greek letter, or other preset indicator value, and is not necessarily a Roman numeral.

[0168] Different spline curve parameters result in different parameters for the modified spline curve. The parameters of the modified spline curve include the first coordinate of at least one interpolation point and multiple coefficients of the modified spline curve.

[0169] The at least one interpolation point constitutes a correction interval, and each pair of adjacent interpolation points constitutes a sub-correction interval. The correction spline curve passes through the at least one interpolation point. For example, the correction spline curve may include multiple spline curves. When the spline curve includes spline curve 1 and spline curve 2, spline curve 1 may be located at [x...]. M1 , x M2 The M-order spline function within the interval [x], the spline curve 2 can be in [x] L1 , x L2 An L-order spline function within the interval [x], where the interval [x] is defined as follows: M1 , x M2 ] and interval [x L1 , x L2 The intersection of [x] is empty, or the interval [x] is empty. M1 , x M2 ] and interval [x L1 , x L2They share a common endpoint value. The degree M of spline curve 1 and the degree L of spline curve 2 can be the same or different. For example, the multiple modified spline curves may include at least one of linear spline curves or cubic spline curves. The number of cubic spline curves included in the modified spline curve can be one, two, three, or more; this application embodiment does not limit the number of cubic spline curves. The cubic spline curves are smoothly connected to the initial tone mapping curve. For example, when the modified spline curve includes both linear and cubic spline curves, the cubic spline curves are smoothly connected to both the linear spline curve and the initial tone mapping curve.

[0170] A linear spline curve has the function form y = ax + b, which includes two coefficients. Determining a linear spline curve requires at least two known conditions.

[0171] The functional form of a quadratic spline curve is y = ax 2 +bx+c, and a≠0, including three coefficients, requires at least three known conditions to determine the quadratic spline curve, such as the coordinates of three points passing through the quadratic spline curve; or the coordinates of two points passing through the quadratic spline curve and the derivative of the quadratic spline curve at one of the points.

[0172] The functional form of a cubic spline curve is y=ax. 3 +bx 2 +cx+d, and a≠0, including four coefficients, the cubic spline curve requires at least four known conditions, such as knowing the coordinates of four points passing through the cubic spline curve in advance; or knowing the coordinates of two points in advance and the derivative of the cubic spline curve at those two points.

[0173] An Nth-degree spline curve consists of N+1 coefficients, and at least N+1 known conditions are required to determine the Nth-degree spline curve.

[0174] For example, such as Figure 4a The diagram shown is an exemplary modified spline curve diagram corresponding to the first scene mode described above, provided in an embodiment of this application. The first scene mode corresponds to a situation where the overall scene of the image is relatively dark.

[0175] The multiple interpolation points include: a first interpolation point, a second interpolation point, and a third interpolation point. The corrected spline curves include: a first spline curve, a second spline curve, and a third spline curve. The first spline curve is a linear spline curve, while the second and third spline curves are cubic spline curves. The first spline curve is a straight line between the origin and the first interpolation point. The second spline curve is a spline curve between the first and second interpolation points. The third spline curve is a spline curve between the second and third interpolation points. The first and second spline curves are smoothly connected at the first interpolation point, the second and third spline curves are smoothly connected at the second interpolation point, and the third spline curve is smoothly connected to the initial tone mapping curve at the third interpolation point. The parameters of the output corrected spline curves include: TH1, TH2, TH3, the slope MB0 of the first spline curve, the four coefficients MA1, MB1, MC1, and MD1 of the second spline curve, and the four coefficients MA2, MB2, MC2, and MD2 of the third spline curve.

[0176] The first curve: (1)

[0177] Second spline curve: (2)

[0178] Third spline curve: (3)

[0179] Where TH1 is the first coordinate of the first interpolation point, TH2 is the first coordinate of the second interpolation point, TH3 is the first coordinate of the third interpolation point, and the first coordinate can be, for example, the x-coordinate, and L is the independent variable of the function.

[0180] The first scene mode corresponds to a relatively dark overall scene in the image. In this case, the dark areas are corrected to better display the dark details of the image, so that pixels with brightness lower than the display capability of the display device can have a better display effect on the display device. Specifically, for pixels below the first threshold (TH1), tone mapping is performed based on the first spline function formula (1), which better preserves the dark details of this part. Then, a smooth transition between the first spline function and the initial tone mapping curve is achieved based on the cubic spline function, avoiding the influence of an unsmooth tone mapping curve on the tone mapping effect. It should be understood that there are two cubic spline curves in the first scene mode. In one optional case, there can be only one cubic spline curve. The first spline curve is smoothly connected to the initial tone mapping curve through a cubic spline function. In this case, only two interpolation points need to be determined. The acquisition process in the first scene mode is the first acquisition process. For details of the first acquisition process, refer to Figure 6a and Figure 6bThe method shown or case 3.1 below. Specifically, in this scenario, the first sample curve increases the brightness of the dark area compared to the base curve, which is equivalent to allocating more bits to the dark area. Two cubic spline functions achieve a smooth transition between the first sample curve and the initial tone mapping curve, and the enhancement or reduction of the correction area can be achieved by controlling the points and slopes of the curves.

[0181] like Figure 4b The diagram shown is an exemplary modified spline curve diagram corresponding to the second scene mode described above, provided in an embodiment of this application. The image corresponding to the second scene mode includes pixels located within a specific brightness range.

[0182] Multiple interpolation points include: a first interpolation point, a second interpolation point, and a third interpolation point. The first and third interpolation points lie on the initial tone mapping curve. The second interpolation point may or may not lie on the initial tone mapping curve. When the second interpolation point is not on the initial tone mapping curve, there is a ordinate offset between the second interpolation point and the initial tone mapping curve. The corrected spline curves include: a first spline curve and a second spline curve. Both the first and second spline curves are cubic spline curves. The first spline curve is the spline curve between the first and second interpolation points, and the second spline curve is the spline curve between the second and third interpolation points. For example, the first spline curve is smoothly connected to the initial tone mapping curve at the first interpolation point, the second spline curve is smoothly connected to the initial tone mapping curve at the third interpolation point, and the first and second spline curves are smoothly connected at the second interpolation point. The parameters of the output modified spline curve include: TH1, TH2, TH3, the four coefficients of the first spline curve MA1, MB1, MC1 and MD1, and the four coefficients of the second spline curve MA2, MB2, MC2 and MD2.

[0183] The first curve: (4)

[0184] Second spline curve: (5)

[0185] Where TH1 is the first coordinate of the first interpolation point, TH2 is the first coordinate of the second interpolation point, TH3 is the first coordinate of the third interpolation point, and the first coordinate can be, for example, the x-coordinate, and L is the independent variable of the function.

[0186] The second scene mode corresponds to images containing pixels within a specific brightness range. In this mode, the focus is on correcting this specific brightness range to better showcase the brightness details within that range. Specifically, tone mapping is performed on the pixels between the first and second interpolation points, as well as between the second and third interpolation points, based on a cubic spline function, improving the display effect of these pixels. Furthermore, the transitions between the first and second spline curves, and between the first and second spline curves and the initial tone mapping curve, are smooth to avoid the influence of uneven tone mapping curves on the tone mapping effect. The acquisition process in the second scene mode is the second acquisition process; for details of the second acquisition process, please refer to [link to relevant documentation]. Figure 7 The method shown or the case described in 3.3 below. Specifically, this correction method can correct any interval, and the correction curve of the correction interval is smoothly connected to the initial tone mapping curve. It can also improve image contrast or increase or decrease the brightness value of some pixels by controlling the interpolation points and offset values, thereby achieving the desired effect.

[0187] like Figure 4c The diagram shown is an exemplary modified spline curve diagram corresponding to the third scene mode described above, provided in an embodiment of this application. The third scene mode corresponds to a situation where the overall scene of the image is relatively bright.

[0188] Multiple interpolation points include: a first interpolation point; the corrected spline curve includes a first spline curve, which is a first-order spline curve passing through the first interpolation point; the first spline curve is smoothly connected to the initial tone mapping curve at the first interpolation point; the parameters of the output corrected spline curve include: TH1, the first coefficient MA0 of the first spline curve, and the second coefficient MB0.

[0189] The first curve: (6)

[0190] Wherein, TH1 is the first coordinate of the first interpolation point, and the first coordinate can be the x-coordinate.

[0191] The third scene mode corresponds to a relatively bright overall scene in the image. In this mode, bright areas are corrected to better reveal the details of the bright parts of the image. Specifically, a linear function (the first spline curve) is used to tone map pixels larger than a preset threshold (TH1). This first spline curve is smoothly connected to the initial tone mapping curve at the first interpolation point to avoid the influence of an uneven tone mapping curve on the tone mapping effect. Specifically, a linear spline curve is used to tone map pixels larger than the first interpolation point, meaning that points after TH1 are compressed using the compression rate of the first interpolation point, avoiding over-compression of bright pixels and thus improving the image brightness. The acquisition process in the third scene mode is the third acquisition process. For details of the third acquisition process, please refer to [link to relevant documentation]. Figure 8The method shown or the following situation 1.

[0192] like Figure 4d The diagram shown is an exemplary modified spline curve diagram corresponding to the fourth scene mode described above, provided in an embodiment of this application. The fourth scene mode corresponds to a situation where the overall scene of the image is relatively bright.

[0193] Multiple interpolation points include: a first interpolation point and a second interpolation point. In an optional case, the second interpolation point has an offset value Spline_strength from the initial tone mapping curve in the vertical direction. The corrected spline curve includes: a first spline curve, which is a cubic spline curve passing through the first interpolation point and the second interpolation point. The first spline curve is smoothly connected to the initial tone mapping curve at the first interpolation point. The parameters of the output corrected spline curve include: TH1, TH2, and the four coefficients MA1, MB1, MC1 and MD1 of the first spline curve.

[0194] The first curve: (7)

[0195] Where TH1 is the first coordinate of the first interpolation point, TH2 is the first coordinate of the second interpolation point, and the first coordinate can be, for example, the x-coordinate, and L is the independent variable of the function.

[0196] The fourth scene mode corresponds to a relatively bright overall scene in the image. In this case, the bright areas are corrected to better display the details of the bright parts of the image. Specifically, the pixel area between the first interpolation point and the second interpolation point is tone-mapped based on the cubic spline function (formula (7)). The cubic spline function is smoothly connected to the initial tone-mapped curve at the first interpolation point to avoid the unsmooth tone-mapped curve affecting the tone-mapped effect. Specifically, the brightness value of the pixel between the endpoints of the first interpolation point and the bright area pixel can be increased, or the brightness value between the endpoints of the first interpolation point and the bright area pixel can be decreased. This can satisfy the functions of increasing brightness to achieve a more dizzying effect, or decreasing brightness to enhance the details of the bright area. Moreover, increasing and decreasing is a non-linear process, which can improve the contrast. The acquisition process in the fourth scene mode is the fourth acquisition process. For the detailed process of the fourth acquisition process, please refer to the following. Figure 9 The method shown or the following situation 2.

[0197] like Figure 4e The diagram shown is an exemplary modified spline curve diagram corresponding to the fifth scene mode described above, provided in an embodiment of this application. The fifth scene mode corresponds to a situation where the overall scene of the image is relatively bright.

[0198] The multiple interpolation points include: a first interpolation point, a second interpolation point, and a third interpolation point. The corrected spline curves include: a first spline curve and a second spline curve. The first spline curve is a cubic spline curve between the first and second interpolation points, and the second spline curve is a linear spline curve between the second and third interpolation points. The first spline curve is smoothly connected to the initial tone mapping curve at the first interpolation point, the second spline curve is smoothly connected to the initial tone mapping curve at the third interpolation point, and the first spline curve and the second spline curve are smoothly connected at the second interpolation point. The parameters of the output corrected spline curve include: TH1, TH2, TH3, the four coefficients of the first spline curve MA1, MB1, MC1, and MD1, and the two coefficients of the second spline curve MA2 and MB2. TH1 is the first coordinate of the first interpolation point, TH2 is the first coordinate of the second interpolation point, and TH3 is the first coordinate of the third interpolation point. The first coordinate can be, for example, the x-axis.

[0199] The first curve: (8)

[0200] Second spline curve: (9)

[0201] Where L is the independent variable of the function.

[0202] It should be understood that the corrected spline curve in the fifth scenario mode includes one linear spline curve and one cubic spline curve, and the positions of the two spline curves are not restricted. Although in Figure 4e In this case, the brightness of the pixels in the pixel interval corresponding to the linear spline curve is greater than the brightness of the pixels in the pixel interval corresponding to the cubic spline curve, that is, the brightness value of the first interpolation point < the brightness value of the second interpolation point < the brightness value of the third interpolation point; in an optional case, the brightness of the pixels in the pixel interval corresponding to the linear spline curve can also be less than the brightness of the pixels in the pixel interval corresponding to the cubic spline curve, in which case the brightness value of the first interpolation point > the brightness value of the second interpolation point > the brightness value of the third interpolation point.

[0203] The fifth scene mode corresponds to a relatively bright overall scene in the image. In this case, the bright areas are corrected to better show the details of the bright parts of the image. Specifically, the pixel area between the second interpolation point and the third interpolation point is tone-mapped based on the first spline curve (formula (9)). The first spline function and the initial tone-mapped curve are smoothly connected based on the cubic spline function between the first interpolation point and the second interpolation point to avoid the tone-mapped effect being affected by the non-smooth tone-mapped curve. Specifically, in this scene, the first interpolation point and the second interpolation point can use the cubic spline function to satisfy the increase and decrease of brightness of the initial tone-mapped curve, and this increase and decrease is non-linear. The tone mapping between the second interpolation point and the third interpolation point is based on a straight line, which can allocate more bits to the pixels between the second interpolation point and the third interpolation point, thereby improving the details of the bright areas. The acquisition process in the fifth scene mode is the fifth acquisition process. For the detailed process of the first acquisition process, please refer to the following. Figure 10 The method shown or the following situation 3.2.

[0204] 303. Tone map the image to be processed according to the modified tone mapping curve, which includes at least a portion of the initial tone mapping curve and the modified spline curve.

[0205] It should be understood that the initial tone mapping curve can be any form of curve that can be used in the tone mapping process. Specifically, Figure 4f Several possible forms of initial tone mapping curves are shown. For example, an initial tone mapping curve can be a normalized curve, where the coordinates of any point in the initial tone mapping curve are (first coordinate value, second coordinate value), where the first coordinate value is between 0 and 1, and the second coordinate value is also between 0 and 1. The following explanation uses a normalized initial tone mapping curve; it should be understood that this application does not limit the scope of the application.

[0206] Figure 4f The initial tone mapping curve shown by the solid line is also called the SLF tone mapping curve, which satisfies:

[0207] (10)

[0208] Where L represents the signal value before mapping, and L' represents the signal value after mapping. In one optional case, the signal value before mapping is a linear signal value, and the signal value after mapping is a non-linear signal value. p, a, m, and b are all dynamic range mapping coefficients, where 0 ≤ L ≤ 1 and 0 ≤ L' ≤ 1. a is directly proportional to the dynamic range display capability of the display device and inversely proportional to the dynamic range of the target image. b is directly proportional to the minimum brightness display capability of the display device.

[0209] Figure 4fThe initial tone mapping curve shown by the single-dotted line is a first-order polynomial tone mapping function, which satisfies:

[0210] (11)

[0211] Where L represents the signal value before mapping, and L' represents the signal value after mapping. In one optional case, the signal value before mapping is a linear signal value, and the signal value after mapping is a non-linear signal value. a and b are dynamic range mapping coefficients, 0≤L≤1, 0≤L'≤1. a is directly proportional to the dynamic range display capability of the display device and inversely proportional to the dynamic range of the target image. b is directly proportional to the minimum brightness display capability of the display device.

[0212] Figure 4f The initial tone mapping curve shown by the dashed line is also called the sigmoidal tone mapping curve, which satisfies:

[0213] (12)

[0214] Where L represents the signal value before mapping, and L' represents the signal value after mapping. In one optional case, the signal value before mapping is a linear signal value, and the signal value after mapping is a non-linear signal value. c1, c2, c3, and n are dynamic range mapping coefficients, where 0 ≤ L ≤ 1 and 0 ≤ L' ≤ 1. c1, c2, c3, and n are all related to the average brightness, maximum brightness, and minimum brightness of the target image, and are also related to the minimum and maximum display brightness of the display device.

[0215] Figure 4f The initial tone mapping curve shown by the double-dotted line is also called the Bézier tone mapping curve, which satisfies:

[0216] (13)

[0217] Where L represents the signal value before mapping, and L' represents the signal value after mapping. In one optional case, the signal value before mapping is a linear signal value, and the signal value after mapping is a non-linear signal value. n, N, and ρ are dynamic range mapping coefficients, where 0 ≤ L ≤ 1 and 0 ≤ L' ≤ 1. n, N, and ρ are related to the brightness distribution of the target image and also to the average brightness of the target image.

[0218] Taking the SLF formula as an example, p and m can be determined by the overall brightness of the image. When the image is generally bright, p is smaller and m is larger; conversely, when the image is generally dark, p is larger and m is smaller. The value of p can range from 3 to 6, and the value of m can range from 2 to 4, with m typically set to 2.4.

[0219] It should be understood that the modified tone mapping curve can be a piecewise polynomial function, where, for example, multiple piecewise functions of the modified tone mapping curve are smoothly connected. Specifically, when the pixel to be processed is within the modified interval, tone mapping is performed on the pixel according to the modified spline curve; when the pixel to be processed is not within the modified interval, tone mapping is performed on the pixel according to the initial tone mapping curve. It should be understood that when determining the interval in which the pixel to be processed is located, the pixel value of the pixel to be processed can be compared with the first coordinate values ​​of multiple interpolation points. For example, when the pixel to be processed is in RGB format, the maximum value among the three RGB components can be used as the pixel value of the pixel to be processed; when the pixel to be processed is in YUV format, the Y component can be used as the pixel value of the pixel to be processed.

[0220] For example, the correction intervals include a first correction interval, a second correction interval, and a third correction interval. The first correction interval corresponds to a first spline function, the second correction interval corresponds to a second spline function, and the third correction interval corresponds to a third spline function. When the pixel to be processed is located in the first correction interval, tone mapping is performed on the pixel to be processed based on the first spline function; when the pixel to be processed is located in the second correction interval, tone mapping is performed on the pixel to be processed based on the second spline function; when the pixel to be processed is located in the third correction interval, tone mapping is performed on the pixel to be processed based on the third spline function; when the pixel to be processed is not located in any of these three correction intervals, tone mapping is performed on the pixel to be processed based on the initial tone mapping curve.

[0221] For example, corresponding to the first scenario mode mentioned above:

[0222] When the pixel to be processed is located between the origin and the first interpolation point, tone mapping is performed on the pixel to be processed based on formula (1);

[0223] When the pixel to be processed is located between the first interpolation point and the second interpolation point, tone mapping is performed on the pixel to be processed based on formula (2);

[0224] When the pixel to be processed is located between the second interpolation point and the third interpolation point, tone mapping is performed on the pixel to be processed based on formula (3);

[0225] When the pixel value of the pixel to be processed is greater than the first coordinate TH3 of the third interpolation point, tone mapping is performed on the pixel to be processed based on the initial tone mapping curve.

[0226] Corresponding to the second scenario mode mentioned above:

[0227] When the pixel to be processed is located between the first interpolation point and the second interpolation point, tone mapping is performed on the pixel to be processed based on formula (4);

[0228] When the pixel to be processed is located between the second interpolation point and the third interpolation point, tone mapping is performed on the pixel to be processed based on formula (5);

[0229] When the pixel value of the pixel to be processed is greater than the first coordinate TH3 of the third interpolation point, tone mapping is performed on the pixel to be processed based on the initial tone mapping curve.

[0230] Corresponding to the third scenario mode mentioned above:

[0231] When the pixel value of the pixel to be processed is less than the first coordinate TH1 of the first interpolation point, tone mapping is performed on the pixel to be processed based on the initial tone mapping curve;

[0232] When the pixel value of the pixel to be processed is greater than the first coordinate TH1 of the first interpolation point, the pixel to be processed is tone-mapped based on formula (6).

[0233] Corresponding to the fourth scenario mode mentioned above:

[0234] When the pixel value of the pixel to be processed is less than the first coordinate TH1 of the first interpolation point, tone mapping is performed on the pixel to be processed based on the initial tone mapping curve;

[0235] When the pixel to be processed is located between the first interpolation point and the second interpolation point, tone mapping is performed on the pixel to be processed based on formula (7).

[0236] Corresponding to the fifth scenario mode mentioned above:

[0237] When the pixel value of the pixel to be processed is less than the first coordinate TH1 of the first interpolation point, tone mapping is performed on the pixel to be processed based on the initial tone mapping curve;

[0238] When the pixel to be processed is located between the first interpolation point and the second interpolation point, tone mapping is performed on the pixel to be processed based on formula (8);

[0239] When the pixel to be processed is located between the second interpolation point and the third interpolation point, tone mapping is performed on the pixel to be processed based on formula (9).

[0240] The image processing method provided in this application determines the applicable scene mode for the image to be processed based on the mode indication value in the dynamic metadata, and flexibly corrects the initial tone mapping curve. This allows for adjustments to the image display effect for different brightness areas, enhancing the details of brightness variations and providing users with a better visual experience. By fine-tuning the tone mapping curve in various ways, display effects with different styles, such as soft, vivid, bright, and dusky, can be obtained.

[0241] The image processing method provided in this application can adaptively modify any part of the curve according to the needs of the scene, thereby improving brightness, improving contrast, and blurring some information. For example, the display effect of dark area details can be improved by increasing the brightness of dark areas, or the contrast can be improved by decreasing the brightness of dark areas; or more bits can be allocated to bright areas to display the details of bright areas, and any part of the curve can be modified to achieve the desired specific effect.

[0242] like Figure 5 The diagram shown is a flowchart of a method for obtaining parameters of a modified spline curve according to an embodiment of this application. The method includes:

[0243] 501. Determine the first coordinates of at least one interpolation point based on the feature information or dynamic metadata of the image to be processed;

[0244] For example, the at least one interpolation point may include only the first interpolation point, or include the first difference point and the second interpolation point, or include the first interpolation point, the second interpolation point, and the third interpolation point. For example, in the third scenario mode shown in this application, only one interpolation point needs to be determined; in the fourth scenario mode shown in this application, two interpolation points need to be determined; and in the first, second, and fifth scenario modes of this application, three interpolation points need to be determined.

[0245] 502. Determine at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point;

[0246] 503. Determine the coefficients of the modified spline curve based on the first coordinate of the at least one interpolation point, at least one second coordinate, and at least one first derivative, wherein the modified spline curve includes at least one of a linear spline curve or a cubic spline curve.

[0247] This application provides several exemplary cases for determining modified spline curves:

[0248] Case 1: The case involves one interpolation point;

[0249] At least one interpolation point includes a first interpolation point, and the corrected spline curve includes a first linear spline curve, which is a first-order spline curve passing through the first interpolation point. The coefficients of the corrected spline curve include a first coefficient MA0 and a second coefficient MB0. The first-order spline curve is smoothly connected to the initial tone mapping curve at the first interpolation point. Specifically, the first derivative of the first-order spline curve at the first interpolation point is equal to the first derivative of the initial tone mapping curve at the first interpolation point. Correspondingly,

[0250] Specifically, 501 includes: determining the first coordinate TH1 of the first interpolation point based on the data contained in the dynamic metadata;

[0251] 502 specifically includes:

[0252] Calculate the second coordinate VA1 of the first interpolation point on the initial tone mapping curve and the first derivative GD1 of the initial tone mapping curve at the first interpolation point based on TH1;

[0253] 503 specifically includes:

[0254] Determine GD1 as the second coefficient MB0 of the first identical curve, and VA1 as the first coefficient MA0 of the first identical curve;

[0255] The first identical curve is obtained based on MB0 and MA0.

[0256] The specific solution for scenario 1 is as follows: Figure 8 The scheme is shown.

[0257] Case 2: Case involving two interpolation points;

[0258] At least one interpolation point includes a first interpolation point and a second interpolation point. The modified spline curve includes a first spline curve, which is a cubic spline curve. The second interpolation point can be one of the endpoints of the initial tone mapping curve. As shown in the fourth scene mode of the embodiments of this application, the second interpolation point is the endpoint of the bright area of ​​the initial tone mapping curve. The modified spline curve and the initial tone mapping curve are smoothly connected at the first interpolation point. Correspondingly,

[0259] 501 specifically includes:

[0260] The first coordinates TH1 of the first interpolation point and the first coordinates TH2 of the second interpolation point are determined based on the data contained in the dynamic metadata. For example, TH1=A and TH2=B, where A and B are constants contained in the dynamic metadata; or A and B are calculated based on the data contained in the dynamic metadata.

[0261] 502 specifically includes:

[0262] Calculate the second coordinates VA1 and VA2 of the first interpolation point and the second interpolation point on the initial tone mapping curve based on TH1 and TH2 respectively;

[0263] Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD2 at the second interpolation point based on TH1 and TH2.

[0264] 503 specifically includes:

[0265] Determine VA1 as the first coefficient MA1 of the first identical curve, and determine GD1 as the second coefficient MB1 of the first identical curve;

[0266] Based on TH1, TH2, VA2, and GD2, a system of equations containing two equations is constructed;

[0267] The third coefficient MC1 and the fourth coefficient MD1 of the first identical curve are obtained by analyzing the system of equations.

[0268] The first identical curve is obtained based on MA1, MB1, MC1, and MD1.

[0269] Among them, a system of equations containing two equations is constructed based on TH1, TH2, VA2, and GD2, specifically including:

[0270] The second coordinate of the second interpolation point on the first linear curve is the sum of VA2 and S1, thus obtaining the first equation, where S1 is the coordinate offset value included in the dynamic metadata.

[0271] The first derivative of the first sample curve at the second interpolation point is the sum of GD2 and S2, resulting in the second equation, where S2 is the slope offset value included in the dynamic metadata.

[0272] The specific solutions for scenario 2 are as follows: Figure 9 The scheme is shown.

[0273] For cases involving two interpolation points, an optional scenario is also included. The first interpolation point is located near the endpoint of the dark area. The corrected spline curves consist of a first spline curve and a second spline curve. The first spline curve is a linear spline curve, and the second spline curve is a cubic spline curve. The first spline curve is a straight line passing through the origin and the first interpolation point, while the second spline curve passes through both the first and second interpolation points. The linear and cubic spline curves are smoothly connected at the first interpolation point, and the cubic spline curve is smoothly connected to the initial tone mapping curve at the second interpolation point. This scenario allows for the correction of dark areas.

[0274] Case 3: Case involving three interpolation points;

[0275] At least one interpolation point includes a first interpolation point, a second interpolation point, and a third interpolation point.

[0276] Case 3.1: The modified spline curve includes: one linear spline curve and two cubic spline curves (as shown in the first scenario mode of the embodiments of this application); correspondingly,

[0277] 501 specifically includes:

[0278] The first coordinate TH1 of the first interpolation point is determined based on at least one of the maximum brightness maxL and the average brightness avgL of the image to be processed included in the dynamic metadata.

[0279] TH2 = TH1 + B;

[0280] TH3 = TH2 + C TH2-D CH2;

[0281] Wherein, TH2 is the first coordinate of the second interpolation point, TH3 is the first coordinate of the third interpolation point, and B, C, and D are constants. In an optional case, B, C, and D are constants contained in the dynamic metadata.

[0282] The method also includes:

[0283] The first derivative MB0 of the first line curve is obtained based on the average brightness avgL of the image to be processed included in the dynamic metadata.

[0284] 502 specifically includes:

[0285] Calculate the second coordinate VA1 of the first interpolation point on the first striped curve based on TH1 and MB0;

[0286] Calculate the second coordinate VA3 of the third interpolation point on the initial tone mapping curve based on TH3;

[0287] Calculate the second coordinate VA2 of the second interpolation point on the first straight line, where the first straight line is the line containing the first interpolation point and the third interpolation point;

[0288] Calculate the first derivative GD3 of the initial tone mapping curve at the third interpolation point;

[0289] 503 specifically includes:

[0290] VA1 is determined to be the first coefficient MA1 of the second spline curve;

[0291] MB0 is determined to be the second coefficient MB1 of the second spline curve;

[0292] The sum of VA2 and the offset value is determined to be the first coefficient MA2 of the third spline curve, where the offset value can be 0, a positive number, or a negative number;

[0293] It should be understood that this offset value can be passed to the tone-mapped equipment via dynamic metadata or defined in the program code.

[0294] Based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the second spline curve and the third spline curve at the second interpolation point, construct a system of equations containing five equations;

[0295] Solving the system of equations containing five equations yields MC1, MD1, MB2, MC2, and MD2.

[0296] Specifically, a system of equations comprising five equations is constructed based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the second spline curve and the third spline curve at the second interpolation point. This system includes:

[0297] The second coordinate of the second interpolation point on the second spline curve is VA2, thus obtaining the first equation;

[0298] This gives the second coordinate of the third interpolation point on the third spline curve VA3, thus obtaining the second equation.

[0299] The third spline curve is made to have a first derivative of GD3 at the third interpolation point, thus obtaining a third-party program.

[0300] By making the first derivatives of the second spline curve and the third spline curve equal at the second interpolation point, we obtain the fourth equation;

[0301] By making the second derivatives of the second spline curve and the third spline curve equal at the second interpolation point, we obtain the fifth equation.

[0302] The specific solution corresponding to situation 3.1 is as follows: Figure 6a and Figure 6b The scheme is shown.

[0303] Case 3.2: The corrected spline curve includes: a first spline curve and a second spline curve. The first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a linear spline curve passing through the second interpolation point and the third interpolation point. The coefficients of the corrected spline curve include: the first coefficient MA1, the second coefficient MB1, the third coefficient MC1, and the fourth coefficient MD1 of the first spline curve, and the first coefficient MA2 and the second coefficient MB2 of the second spline curve (as shown in the fifth scenario mode of the embodiments of this application); correspondingly,

[0304] 501 specifically includes:

[0305] The first coordinates TH1 of the first interpolation point, the first coordinates TH2 of the second interpolation point, and the first coordinates TH3 of the third interpolation point are determined based on the data contained in the dynamic metadata. For example, TH1=A, TH2=B, and TH3=C, where A, B, and C are constants contained in the dynamic metadata; or A, B, and C are calculated based on the data contained in the dynamic metadata.

[0306] 502 specifically includes:

[0307] Calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on TH1 and TH3 respectively. VA1 is the first coefficient MA1 of the first parallel curve.

[0308] Calculate the second coordinate VA2 of the second interpolation point on the first straight line, which is the line containing the first interpolation point (TH1, VA1) and the third interpolation point (TH3, VA3);

[0309] Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point.

[0310] 503 specifically includes:

[0311] VA1 is determined to be the first coefficient MA1 of the first identical curve.

[0312] GD1 is determined to be the second coefficient MB1 of the first identical curve;

[0313] In an optional case, the dynamic metadata includes coordinate offset values, and the sum of VA2 and the coordinate offset values ​​contained in the dynamic metadata is determined to be the first coefficient MA2 of the second spline curve;

[0314] Based on TH1, TH2, TH3, VA2, VA3 and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point, construct a system of equations containing three equations;

[0315] The system of equations, consisting of three equations, is analyzed to obtain MC1, MD1, and MB2.

[0316] Specifically, a system of equations containing three equations is constructed based on TH1, TH2, TH3, VA2, VA3, and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point. This system includes:

[0317] The second coordinate of the second interpolation point on the first linear curve is VA2, thus obtaining the first equation;

[0318] This makes the second coordinate of the third interpolation point on the second spline curve VA3, thus obtaining the second equation;

[0319] The first and second spline curves are made to have the same first derivative at the second interpolation point, thus obtaining a third-party program.

[0320] The specific solution corresponding to situation 3.2 is as follows: Figure 10 The scheme is shown.

[0321] Case 3.3: The corrected spline curve includes: a first spline curve and a second spline curve. The first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a cubic spline curve passing through the second interpolation point and the third interpolation point. The coefficients of the corrected spline curve include: the first coefficient MA1, the second coefficient MB1, the third coefficient MC1, and the fourth coefficient MD1 of the first spline curve, and the first coefficient MA2, the second coefficient MB2, the third coefficient MC2, and the fourth coefficient MD2 of the second spline curve (as shown in the second scenario mode of this application embodiment). Correspondingly,

[0322] 501 specifically includes:

[0323] The first coordinates TH1 of the first interpolation point, the first coordinates TH2 of the second interpolation point, and the first coordinates TH3 of the third interpolation point are determined based on the data contained in the dynamic metadata. For example, TH1=A, TH2=B, and TH3=C, where A, B, and C are constants contained in the dynamic metadata; or A, B, and C are calculated based on the data contained in the dynamic metadata.

[0324] 502 specifically includes:

[0325] Calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on TH1 and TH3 respectively;

[0326] Calculate the second coordinate VA2 of the second interpolation point on the first straight line, where the first straight line is the line containing the first interpolation point and the third interpolation point;

[0327] Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 at the third interpolation point;

[0328] 503 specifically includes:

[0329] VA1 is determined to be the first coefficient MA1 of the first identical curve.

[0330] GD1 is determined to be the second coefficient MB1 of the first identical curve;

[0331] The sum of VA2 and the offset value is determined to be the first coefficient MA2 of the second spline curve, wherein the offset value is 0, positive, or negative; optionally, the offset value may be included in dynamic metadata.

[0332] Based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point, construct a system of equations containing five equations;

[0333] Solving the system of equations containing five equations yields MC1, MD1, MB2, MC2, and MD2.

[0334] Specifically, based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the first and second spline curves at the second interpolation point, a system of five equations is constructed, including:

[0335] The second coordinate of the second interpolation point on the first linear curve is VA2, thus obtaining the first equation;

[0336] This makes the second coordinate of the third interpolation point on the second spline curve VA3, thus obtaining the second equation;

[0337] This makes the first derivative of the second spline curve at the third interpolation point GD3, thus obtaining a third-party program;

[0338] By making the first derivative of the first spline curve and the second spline curve equal at the second interpolation point, we obtain the fourth equation;

[0339] By making the second derivatives of the first spline curve and the second spline curve equal at the second interpolation point, we obtain the fifth equation.

[0340] The specific solutions corresponding to situation 3.3 are as follows: Figure 7 The scheme is shown.

[0341] 504. Tone map the image to be processed according to the modified tone mapping curve, which includes at least a portion of the initial tone mapping curve and the modified spline curve.

[0342] For 504, please refer to the description in section 303; it will not be repeated here.

[0343] In an alternative case, prior to 501, the method further includes:

[0344] The target scene mode applicable to the image to be processed is determined based on the mode indicator value in the dynamic metadata. At least one interpolation point and the correction spline curve are related to the target scene mode. Specifically, the number of at least one interpolation point, and the determination of the first-order and cubic spline curves included in the correction spline curve, are related to the mode indicator value. In other words, the parameters of the correction spline curve for which of the following scenarios (case 1, case 2, case 3.1, case 3.2, and case 3.3) are specifically obtained are determined based on the mode indicator value.

[0345] like Figure 6a The diagram shown is a flowchart illustrating a method for obtaining parameters of a modified spline curve according to an embodiment of this application. Figure 6aThe flowchart illustrates the acquisition of modified spline curve parameters in the first scenario mode and case 3.1. In this case, the modified spline curve includes one linear spline curve (the first spline curve) and two cubic spline curves: the second spline curve and the third spline curve.

[0346] 601. Obtain the first coordinate TH1 of the first interpolation point and the first derivative MB0 of the first curve;

[0347] (14)

[0348] (15)

[0349] Wherein, maxL and avgL are contained in the dynamic metadata, maxL is the maximum value of the brightness of the image to be processed, and avgL is the average value of the brightness of the image to be processed; when the image to be processed is received, the first coordinate TH1 of the first interpolation point is calculated according to the maxL contained in the dynamic metadata of the image to be processed and the formula (14), and the slope MB0 of the first parallel curve is calculated according to the avgL contained in the dynamic metadata and the formula (15). The first parallel curve is, for example, the curve shown in formula (1). In an optional case, the first coordinate of the first interpolation point can be calculated according to the average value avgL of the brightness of the image to be processed. Correspondingly, maxL in formula (14) can be replaced with avgL. In another optional case, the first coordinate of the first interpolation point can be calculated according to the maximum value maxL and the average value avgL of the brightness of the image to be processed. Correspondingly, maxL in formula (14) can be replaced with avgL. maxL+b avgL, where a and b are constants, and 0≤a≤1, 0≤b≤1.

[0350] Correspondingly, in one optional case, the first derivative MB0 of the first linear curve can be calculated based on the maximum brightness value maxL of the image to be processed. Correspondingly, avgL in formula (15) can be replaced with maxL. In another optional case, the first coordinate of the first interpolation point can be calculated based on the maximum brightness value maxL and the average brightness value avgL of the image to be processed. Correspondingly, avgL in formula (15) can be replaced with a maxL+b avgL, where a and b are constants, and 0≤a≤1, 0≤b≤1.

[0351] PvalueL is the first threshold for dark area pixels, and PvalueH is the second threshold for the brightness value of dark area pixels, where the first threshold is less than the second threshold. For example, PvalueL can be 0.15 and PvalueH can be 0.25. TPH is the third threshold for bright area pixels, and TPL is the fourth threshold for bright area pixels, where TPH can be 0.6 and TPL can be 0.1. AvL is the fifth threshold for average pixel values, and AvH is the sixth threshold for average pixel values, where AvL can be 0.15 and AvH can be 0.65. SvalueL is the first slope threshold for the increase in brightness value of dark area pixels, and SvalueH is the second slope threshold for the increase in brightness value of dark area pixels, where the first slope threshold is less than the second slope threshold. For example, SvalueL can be 1 and SvalueH can be 1.2.

[0352] 602. Obtain the first coordinates TH2 and TH3 of the second and third interpolation points;

[0353] TH2 = TH1 + B (16)

[0354] TH3 = TH2 + C TH2-D TH2 (17)

[0355] Where B is the first offset value of the pixel in the dark transition zone, C is the second offset value of the pixel in the bright zone, and D is the third offset value of the pixel in the bright zone; B, C, and D are constants. In an optional case, the dynamic metadata includes the above B, C, and D; B, C, and D can also be directly defined in the relevant program code without needing to be obtained from the dynamic metadata. For example, B can be 0.15, and C and D can be 0.5.

[0356] 603. Determine the four coefficients MA1, MB1, MC1 and MD1 of the second spline curve and the four coefficients MA2, MB2, MC2 and MD2 of the third spline curve based on TH1, TH2 and TH3.

[0357] It should be understood that, in this case, the modified spline curve includes two cubic spline curves: a second spline curve and a third spline curve. In an alternative case, the modified spline curve may include only one cubic spline curve.

[0358] Specifically, such as Figure 6b As shown, step 603 includes:

[0359] 6031. Calculate the second coordinates VA1, VA2, and VA3 of the first interpolation point, the second interpolation point, and the third interpolation point, respectively;

[0360] Specifically, 6031 includes:

[0361] 1) Calculate the second coordinate VA1 of the first interpolation point on the linear spline curve;

[0362] VA1 = MB0 × TH1;

[0363] 2) Calculate the second coordinate VA3 of the third interpolation point on the initial tone mapping curve;

[0364] For example, the initial tone mapping curve is taken as the SLF tone mapping curve, as shown in formula (18):

[0365] (18)

[0366] Substituting TH3 into formula (18) yields VA3=F(TH3); it should be understood that p, m, n, a, b, k1, k2, and k3 in formula (18) are all constants. The parameters of this initial tone mapping curve can be obtained from dynamic metadata or calculated in advance based on the image to be processed.

[0367] 3) Calculate the second coordinate VA2 of the second interpolation point on the first straight line, where the first straight line is the line containing the first interpolation point and the third interpolation point;

[0368] (19)

[0369] 6032. Determine VA1 as the first coefficient MA1 of the second spline curve, and determine the sum of VA2 and the offset value as the first coefficient MA2 of the third spline curve; optionally, the offset value can be 0, a positive number, or a negative number; the offset value can be obtained from dynamic metadata or defined in the relevant program code of the spline curve parameter acquisition process.

[0370] 6033. Determine that the first derivative MB0 of the first spline curve is the second coefficient MB1 of the second spline curve;

[0371] 6034. Calculate the first derivative GD3 of the initial tone mapping curve at the third interpolation point;

[0372] (20)

[0373] 6035. Construct a system of equations based on TH1, TH2, TH3, VA2, VA3, GD3, and the derivative relationship between the second and third spline curves at the second interpolation point;

[0374] For example, the system of equations includes: a first equation, a second equation, a third equation, a fourth equation, and a fifth equation, and 6035 specifically includes:

[0375] 1) Make the second coordinate of the second interpolation point on the second spline curve VA2 as described above. In an optional case, VA2 = formula (19) + offset value, and the first equation is obtained as shown in formula (21):

[0376] (twenty one)

[0377] 2) Make the second coordinate of the third interpolation point on the third spline curve VA3 as described above, and obtain the second equation as shown in formula (22):

[0378] (twenty two)

[0379] 3) Make the first derivative of the third spline curve at the third interpolation point equal to the above GD3, and obtain the third program, as shown in formula (23):

[0380] (twenty three)

[0381] 4) Make the first derivatives of the second spline curve and the third spline curve at the second interpolation point equal, and obtain the fourth equation, as shown in formula (24):

[0382]

[0383] (twenty four)

[0384] 5) Make the second derivatives of the second spline curve and the third spline curve at the second interpolation point equal, and obtain the fifth equation, as shown in formula (25):

[0385] (25)

[0386] 6036. Solve the system of equations consisting of the first equation, the second equation, the third equation, the fourth equation, and the fifth equation to obtain MC1, MD1, MB2, MC2, and MD2.

[0387] (26)

[0388] (27)

[0389] (28)

[0390] (29)

[0391] (30)

[0392] Where h1 = TH2 - TH1, h2 = TH3 - TH2.

[0393] 604. Determine the second spline curve based on MA1, MB1, MC1, and MD1, and determine the third spline curve based on MA2, MB2, MC2, and MD2.

[0394] Specifically, the second spline curve is shown in formula (2), and the third spline curve is shown in formula (3).

[0395] It should be understood that, for ease of description, Figure 6a and Figure 6b Methods are described in steps. Although the order of methods is shown in the method flowchart, in some cases, the steps described may be performed in a different order than that shown here.

[0396] like Figure 7 The diagram shown is a flowchart illustrating a method for obtaining parameters of a modified spline curve according to an embodiment of this application. Figure 7 The flowchart illustrates the acquisition of modified spline curve parameters in the second scenario mode. In this case, the modified spline curve includes two cubic spline curves: a first spline curve and a second spline curve. In an optional scenario, the modified spline curve may include only one cubic spline curve, corresponding to the determination of only two interpolation points.

[0397] The method includes:

[0398] 701. Obtain the first coordinates TH1, TH2, and TH3 of the first interpolation point, the second interpolation point, and the third interpolation point respectively;

[0399] For example, TH1=A, TH2=B, TH3=C, where A, B, and C are all constants. Assume AB constitutes the first adjustment interval, BC constitutes the second adjustment interval, A is the minimum brightness value of a pixel in the first adjustment interval, B is the maximum brightness value of a pixel in the first adjustment interval, and C is the maximum brightness value of a pixel in the second adjustment interval. <B<C。

[0400] In an alternative scenario, A, B, and C can be obtained from dynamic metadata. For example, if the dynamic metadata includes the start position L1 and end position L3 of the modified spline curve interval, then A, B, and C can be calculated based on the data information in the dynamic metadata.

[0401] A=L1 / 255, C=L3 / 255, B=(A+C) / 2

[0402] In one optional scenario, the dynamic metadata directly carries A, B, and C as described above.

[0403] In one alternative case, A, B and C can be calculated using a formula similar to formula (14).

[0404] 702. Calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve, respectively;

[0405] For example, the initial tone mapping curve is shown in formula (18) above. Substituting TH1 into formula (18) yields VA1, and substituting TH3 into formula (18) yields VA3:

[0406] (31)

[0407] (32)

[0408] 703. Calculate the second coordinate VA2 of the second interpolation point on the first straight line, where the first straight line is the line containing the first interpolation point and the third interpolation point;

[0409] In one optional case, the formula for calculating VA2 is the same as formula (19).

[0410] In one optional case, the dynamic metadata includes a coordinate offset value, Spline_strength. It should be understood that this coordinate offset value can be positive, negative, or zero. In this case, VA2 is the sum of the second coordinate of the second interpolation point on the first straight line and the coordinate offset value carried in the dynamic metadata.

[0411] (33)

[0412] 704. Determine VA1 as the first coefficient MA1 of the first spline curve and VA2 as the first coefficient MA2 of the second spline curve;

[0413] 705. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 at the third interpolation point, and determine GD1 as the second coefficient MB1 of the first identical curve.

[0414] (34)

[0415] The formula for calculating GD3 is the same as formula (20).

[0416] 706. Based on TH1, TH2, TH3, VA2, VA3, GD3 and the derivative relationship between the first spline curve and the second spline curve at the second interpolation point, construct a system of equations, which includes 5 equations.

[0417] For example, 706 specifically includes:

[0418] 1) Make the second coordinate of the second interpolation point on the first same curve as the above VA2, and obtain the first equation, which is shown in formula (21);

[0419] 2) Make the second coordinate of the third interpolation point on the second spline curve the above VA3, and obtain the second equation, as shown in formula (22);

[0420] 3) Make the first derivative of the second spline curve at the third interpolation point GD3, and obtain the third program, as shown in formula (23);

[0421] 4) Make the first derivative of the first spline curve and the second spline curve at the second interpolation point equal, and obtain the fourth equation, as shown in formula (24);

[0422] 5) Make the second derivatives of the first spline curve and the second spline curve at the second interpolation point equal, and obtain the fifth equation, as shown in formula (25).

[0423] 707. Solve the system of equations consisting of the first equation, the second equation, the third equation, the fourth equation, and the fifth equation to obtain MC1, MD1, MB2, MC2, and MD2.

[0424] The calculation results are referenced from formulas (26) to (30), and will not be repeated here.

[0425] 708. Determine the first spline curve based on MA1, MB1, MC1, and MD1, and determine the second spline curve based on MA2, MB2, MC2, and MD2.

[0426] The first curve: (35)

[0427] Second spline curve: (36)

[0428] In this scenario, the first coordinates of the first, second, and third interpolation points are obtained based on dynamic metadata. The first and third interpolation points are both on the initial tone mapping curve. The dynamic metadata may also include the ordinate offset value of the second interpolation point, which is used to indicate the offset value of the second interpolation point in the ordinate direction from the straight line defined by the first and third interpolation points.

[0429] like Figure 8 The diagram shown is a flowchart illustrating a method for obtaining parameters of a modified spline curve according to an embodiment of this application. Figure 8The flowchart illustrates the acquisition of parameters for the modified spline curve in the third scenario mode. In this case, the modified spline curve includes a first-order spline curve: the first spline curve. The parameters that need to be determined include: the first coordinate TH1 of the first interpolation point, the first coefficient MA0 and the second coefficient MB0 of the first spline curve.

[0430] The method includes:

[0431] 801. Obtain the first coordinate TH1 of the first interpolation point;

[0432] In one possible solution, TH1 = A;

[0433] Optionally, A can be a constant carried in the dynamic metadata, or A can be calculated based on the data carried in the dynamic metadata. For example, if the dynamic metadata carries the starting value L1 of the correction interval, then A = L1 / 255.

[0434] In an alternative approach, TH1 can be calculated based on the feature information of the image and a piecewise formula similar to formula (14).

[0435] 802. Calculate the second coordinate VA1 of the first interpolation point on the initial tone mapping curve;

[0436] The initial tone mapping curve is shown in formula (18). Substituting TH1 into formula (18) yields VA1:

[0437] (37)

[0438] 803. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point;

[0439] (38)

[0440] 804. Determine GD1 as the second coefficient MB0 of the first linear curve and VA1 as the first coefficient MA0 of the first linear curve.

[0441] Correspondingly, the obtained modified spline curve is as shown in formula (6).

[0442] like Figure 9 The diagram shown is a flowchart illustrating a method for obtaining parameters of a modified spline curve according to an embodiment of this application. Figure 9 The flowchart illustrates the acquisition of modified spline curve parameters in the fourth scenario mode. In this case, the modified spline curve includes a cubic spline curve: the first spline curve.

[0443] The method includes:

[0444] 901. Obtain the first coordinates TH1 and TH2 of the first interpolation point and the second interpolation point respectively based on the dynamic metadata;

[0445] For example, TH1=A, TH2=B; where A and B are constants. For example, B can be 1.0.

[0446] In one optional case, A and B can be obtained directly from the dynamic metadata; in another optional case, A and B can be calculated based on the data information carried in the dynamic metadata. A and B can also be calculated based on the features of the image to be processed and a piecewise formula similar to formula (14).

[0447] 902. Calculate the second coordinates VA1 and VA2 of the first interpolation point and the second interpolation point on the initial tone mapping curve, respectively;

[0448] For example, the initial tone mapping curve is shown in formula (18) above. Substituting TH1 into formula (18) yields VA1, and substituting TH2 into formula (18) yields VA2:

[0449] (39)

[0450] (40)

[0451] 903. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD2 at the second interpolation point;

[0452] (41)

[0453] (42)

[0454] 904. Determine VA1 as the first coefficient MA1 of the first identical curve, and determine GD1 as the second coefficient MB1 of the first identical curve;

[0455] 905. Construct a system of equations based on TH1, TH2, VA2, and GD2;

[0456] For example, 905 specifically includes:

[0457] 1) Make the second coordinate of the second interpolation point on the first straight line VA2'=VA2+S1, and obtain the first equation as shown in formula (43);

[0458] (43)

[0459] S1 is the coordinate offset value included in the dynamic metadata. Optionally, S1 can be a positive value, a negative value, or 0.

[0460] 2) Make the first derivative of the first identical curve at the second interpolation point GD2'=GD2+S2, and obtain the second equation as shown in formula (44).

[0461] (44)

[0462] S2 is the slope offset value included in the dynamic metadata. Optionally, S2 can be a positive value, a negative value, or 0.

[0463] 906. Solve the system of equations consisting of the first equation and the second equation to obtain MC1 and MD1.

[0464] Correspondingly, the obtained modified spline curve is shown in formula (45):

[0465] (45)

[0466] like Figure 10 The diagram shown is a flowchart illustrating a method for obtaining parameters of a modified spline curve according to an embodiment of this application. Figure 10 The flowchart illustrates the acquisition of modified spline curve parameters in the fifth scenario mode. In this case, the modified spline curve includes one cubic spline curve and one linear spline curve. The parameters of the modified spline curve include: four coefficients MA1, MB1, MC1, and MD1 of the first spline curve, and two coefficients MA2 and MB2 of the second spline curve.

[0467] The method includes:

[0468] 1001. Obtain the first coordinates TH1, TH2, and TH3 of the first interpolation point, the second interpolation point, and the third interpolation point respectively;

[0469] For example, TH1=A, TH2=B, TH3=C, where A, B, and C are all constants. Assume AB constitutes the first adjustment interval, BC constitutes the second adjustment interval, A is the minimum brightness value of a pixel in the first adjustment interval, B is the maximum brightness value of a pixel in the first adjustment interval or the minimum brightness value of a pixel in the second adjustment interval, and C is the maximum brightness value of a pixel in the second adjustment interval. <B<C。

[0470] In an alternative scenario, A, B, and C can be obtained from dynamic metadata. For example, if the dynamic metadata includes the start position L1 and end position L3 of the modified spline curve interval, then A, B, and C can be calculated based on the data information in the dynamic metadata.

[0471] A=L1 / 255, C=L3 / 255, B=(A+C) / 2

[0472] In one optional scenario, the dynamic metadata directly carries A, B, and C as described above.

[0473] In one alternative case, A, B and C can be calculated using a formula similar to formula (14).

[0474] 1002. Calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve, respectively;

[0475] Please refer to section 702 for details; they will not be repeated here.

[0476] 1003. Calculate the second coordinate VA2 of the second interpolation point on the first straight line, where the first straight line is the line containing the first interpolation point (TH1, VA1) and the third interpolation point (TH3, VA3);

[0477] In one alternative case, the formula for calculating VA2 is the same as that for formula (19).

[0478] In an optional case, the dynamic metadata includes a coordinate offset value Spline_strength. It should be understood that this coordinate offset value can be positive, negative, or 0. When Spline_strength is 0, the value of VA2 can be calculated according to formula (19). In this case, VA2 is the sum of the second coordinate of the second interpolation point on the first straight line and the coordinate offset value carried in the dynamic metadata:

[0479] (46)

[0480] It should be understood that VA1 <VA2<VA3。

[0481] 1004. Determine VA1 as the first coefficient MA1 of the first spline curve and VA2 as the first coefficient MA2 of the second spline curve;

[0482] 1005. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 at the third interpolation point, and determine GD1 as the second coefficient MB1 of the first identical curve.

[0483] (47)

[0484] (48)

[0485] 1006. Construct a system of equations based on the derivatives of TH1, TH2, TH3, VA2, VA3, the first spline curve, and the second spline curve at the second interpolation point;

[0486] For example, 1006 specifically includes:

[0487] 1) Make the second coordinate of the second interpolation point on the first identical curve VA2, and obtain the first equation, as shown in formula (49);

[0488] (49)

[0489] 2) Make the second coordinate of the third interpolation point on the second spline curve VA3, and obtain the second equation, as shown in formula (50);

[0490] (50)

[0491] 3) Make the first derivative of the first spline curve and the second spline curve at the second interpolation point equal to obtain the third program, as shown in formula (51);

[0492] (51)

[0493] 1007. Solve the system of equations consisting of the first equation, the second equation, and the third equation to obtain MC1, MD1, and MB2.

[0494] The obtained modified spline curves are shown in formulas (52) and (53), where formula (52) is the first spline curve and formula (53) is the second spline curve.

[0495] (52)

[0496] (53)

[0497] It should be understood that, for ease of description, Figures 7 to 10 The method flowchart shown describes the method in the form of steps. Although the order of the methods is shown, in some cases, the steps described may be performed in a different order than that shown here.

[0498] like Figure 11 The diagram shown is a structural block diagram of an exemplary image processing apparatus 1100 provided in an embodiment of this application. The image processing apparatus 1100 includes:

[0499] Determining module 1110: used to execute steps 301, 302, 501, 502 and 503 in the method embodiment.

[0500] Tone mapping module 1120: used to perform steps 303 and 504 in the method embodiment.

[0501] Specifically, this determining module is also used to execute Figure 6a , Figure 6b ,as well as Figures 7 to 10Some or all of the steps in the method embodiments shown.

[0502] In an optional embodiment, the image processing apparatus 1100 further includes an acquisition module 1130 for acquiring the image to be processed and the corresponding dynamic metadata.

[0503] It should be understood that, such as Figure 1 The image processing device 14 shown and as shown Figure 2 The image processing apparatus shown can all be used to implement the embodiments of this application. Figure 3 , Figure 5 , Figure 6a , Figure 6b as well as Figures 7 to 10 Any of the provided method embodiments. Specifically, image processing device 14 and Figure 2 The processor in the memory calls program instructions stored in memory to execute, such as... Figure 3 , Figure 5 , Figure 6a , Figure 6b as well as Figures 7 to 10 Some or all of the steps in the method shown in any embodiment.

[0504] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of any of the above methods. If the constituent modules of the above-described signal processing apparatus are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0505] Based on this understanding, embodiments of this application also provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to execute any of the methods provided in the embodiments of this application. The technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device or its processor to execute all or part of the steps of the methods described in the various embodiments of this application.

[0506] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0507] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, include: Based on the feature information or dynamic metadata of the image to be processed, determine the first coordinates of at least one interpolation point; Determine at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point; The coefficients of the modified spline curve are determined based on the first coordinate of the at least one interpolation point, at least one second coordinate, and at least one first derivative. The modified spline curve includes at least one of a linear spline curve or a cubic spline curve. The image to be processed is tone-mapped according to the modified tone mapping curve, wherein the modified tone mapping curve includes at least a portion of the initial tone mapping curve and the modified spline curve. The at least one interpolation point includes: a first interpolation point and a second interpolation point; the modified spline curve includes: a first spline curve, which is a cubic spline curve passing through the first interpolation point and the second interpolation point; the coefficients of the modified spline curve include a first coefficient MA1, a second coefficient MB1, a third coefficient MC1, and a fourth coefficient MD1. Determining at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point includes: Calculate the second coordinates VA1 and VA2 of the first interpolation point and the second interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point and the first coordinate TH2 of the second interpolation point, respectively. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD2 at the second interpolation point based on TH1 and TH2. Determining the coefficients of the modified spline curve based on the first coordinates of the at least one interpolation point, at least one second coordinate, and at least one first derivative includes: VA1 is determined to be the first coefficient MA1 of the first spline curve; GD1 is determined to be the second coefficient MB1 of the first spline curve; The third coefficient MC1 and the fourth coefficient MD1 of the first spline curve are obtained based on TH1, TH2, VA2 and GD2; The first spline curve is determined according to the following function: L is the independent variable of the function; The TH1 is determined according to the following formula: Wherein, avgL is the average brightness of the image to be processed, PvalueL is the first threshold for dark area pixels, PvalueH is the second threshold for the brightness value of dark area pixels, the first threshold is less than the second threshold, TPH is the third threshold for bright area pixels, and TPL is the fourth threshold for bright area pixels, the third threshold is greater than the fourth threshold.

2. The method according to claim 1, characterized in that: The second coordinate of the second interpolation point on the first spline curve is the sum of VA2 and S1, where S1 is the coordinate offset value included in the dynamic metadata. The first derivative of the first spline curve at the second interpolation point is the sum of GD2 and S2, where S2 is the slope offset value included in the dynamic metadata.

3. An image processing method, characterized in that, include: Based on the feature information or dynamic metadata of the image to be processed, determine the first coordinates of at least one interpolation point; Determine at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point; The coefficients of the modified spline curve are determined based on the first coordinate of the at least one interpolation point, at least one second coordinate, and at least one first derivative. The modified spline curve includes at least one of a linear spline curve or a cubic spline curve. The image to be processed is tone-mapped according to the modified tone mapping curve, wherein the modified tone mapping curve includes at least a portion of the initial tone mapping curve and the modified spline curve. The at least one interpolation point includes: a first interpolation point, a second interpolation point, and a third interpolation point; the modified spline curve includes: a first spline curve and a second spline curve; the first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a cubic spline curve passing through the second interpolation point and the third interpolation point; the coefficients of the modified spline curve include: the first coefficient MA1, the second coefficient MB1, the third coefficient MC1, and the fourth coefficient MD1 of the first spline curve, and the first coefficient MA2, the second coefficient MB2, the third coefficient MC2, and the fourth coefficient MD2 of the second spline curve; Determining at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point includes: Calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point and the first coordinate TH3 of the third interpolation point, respectively. The second coordinate VA2 of the second interpolation point on the first straight line is calculated based on the first coordinate TH2 of the second interpolation point. The first straight line is the straight line where the first interpolation point and the third interpolation point are located. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 at the third interpolation point; The first spline curve is determined according to the following function: L is the independent variable of the function; The TH1 is determined according to the following formula: Wherein, avgL is the average brightness of the image to be processed, PvalueL is the first threshold for dark area pixels, PvalueH is the second threshold for the brightness value of dark area pixels (the first threshold is less than the second threshold), TPH is the third threshold for bright area pixels, and TPL is the fourth threshold for bright area pixels.

4. The method according to claim 3, characterized in that, Determining the coefficients of the modified spline curve based on the first coordinates of the at least one interpolation point, at least one second coordinate, and at least one first derivative includes: VA1 is determined to be the first coefficient MA1 of the first spline curve. GD1 is determined to be the second coefficient MB1 of the first spline curve; The sum of VA2 and the offset value is determined to be the first coefficient MA2 of the second spline curve, wherein the offset value is 0, a positive number, or a negative number; Based on TH1, TH2, TH3, VA2, VA3, GD3, we obtain MC1, MD1, MB2, MC2, and MD2.

5. The method according to claim 4, characterized in that: The second coordinate of the second interpolation point on the first spline curve is VA2; The second coordinate of the third interpolation point on the second spline curve is VA3; The first derivative of the second spline curve at the third interpolation point is GD3; The first derivative of the first spline curve and the second spline curve at the second interpolation point are equal; The second derivatives of the first spline curve and the second spline curve at the second interpolation point are equal.

6. The method according to any one of claims 1 to 5, characterized in that, The at least one interpolation point constitutes at least one correction interval, wherein one correction interval corresponds to one correction spline curve, and the step of performing tone mapping on the image to be processed according to the corrected tone mapping curve includes: When the pixel to be processed is located in the at least one correction interval, tone mapping is performed on the pixel to be processed based on the correction spline curve; When the pixel to be processed is outside the at least one correction interval, the pixel to be processed is tone mapped based on the initial tone mapping curve.

7. An image processing apparatus, characterized in that, include: The determination module is used to determine the first coordinates of at least one interpolation point based on the feature information or dynamic metadata of the image to be processed; The determining module is further configured to determine at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point; The determining module further determines the coefficients of the modified spline curve based on the first coordinates of the at least one interpolation point, at least one second coordinate, and at least one first derivative. The modified spline curve includes at least one of a linear spline curve or a cubic spline curve. A tone mapping module is used to perform tone mapping on the image to be processed according to a modified tone mapping curve, wherein the modified tone mapping curve includes at least a portion of an initial tone mapping curve and the modified spline curve. The at least one interpolation point includes: a first interpolation point and a second interpolation point; the modified spline curve includes: a first spline curve, which is a cubic spline curve passing through the first interpolation point and the second interpolation point; the coefficients of the modified spline curve include a first coefficient MA1, a second coefficient MB1, a third coefficient MC1, and a fourth coefficient MD1. The determining module is specifically used for: Calculate the second coordinates VA1 and VA2 of the first interpolation point and the second interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point and the first coordinate TH2 of the second interpolation point, respectively. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD2 at the second interpolation point based on TH1 and TH2. VA1 is determined to be the first coefficient MA1 of the first spline curve; GD1 is determined to be the second coefficient MB1 of the first spline curve; The third coefficient MC1 and the fourth coefficient MD1 of the first spline curve are obtained based on TH1, TH2, VA2 and GD2; The first spline curve is determined according to the following function: L is the independent variable of the function; The TH1 is determined according to the following formula: Wherein, avgL is the average brightness of the image to be processed, PvalueL is the first threshold for dark area pixels, PvalueH is the second threshold for the brightness value of dark area pixels (the first threshold is less than the second threshold), TPH is the third threshold for bright area pixels, and TPL is the fourth threshold for bright area pixels.

8. The apparatus according to claim 7, characterized in that, The second coordinate of the second interpolation point on the first spline curve is the sum of VA2 and S1, where S1 is the coordinate offset value included in the dynamic metadata. The first derivative of the first spline curve at the second interpolation point is the sum of GD2 and S2, where S2 is the slope offset value included in the dynamic metadata.

9. An image processing apparatus, characterized in that, include: The determination module is used to determine the first coordinates of at least one interpolation point based on the feature information or dynamic metadata of the image to be processed; The determining module is further configured to determine at least one second coordinate and at least one first derivative based on the first coordinate of the at least one interpolation point; The determining module further determines the coefficients of the modified spline curve based on the first coordinates of the at least one interpolation point, at least one second coordinate, and at least one first derivative. The modified spline curve includes at least one of a linear spline curve or a cubic spline curve. A tone mapping module is used to perform tone mapping on the image to be processed according to a modified tone mapping curve, wherein the modified tone mapping curve includes at least a portion of an initial tone mapping curve and the modified spline curve. The at least one interpolation point includes: a first interpolation point, a second interpolation point, and a third interpolation point; the modified spline curve includes: a first spline curve and a second spline curve, wherein the first spline curve is a cubic spline curve passing through the first interpolation point and the second interpolation point, and the second spline curve is a cubic spline curve passing through the second interpolation point and the third interpolation point; the coefficients of the modified spline curve include: the first coefficient MA1, the second coefficient MB1, the third coefficient MC1, and the fourth coefficient MD1 of the first spline curve, and the first coefficient MA2, the second coefficient MB2, the third coefficient MC2, and the fourth coefficient MD2 of the second spline curve; the determining module is specifically used for: Calculate the second coordinates VA1 and VA3 of the first interpolation point and the third interpolation point on the initial tone mapping curve based on the first coordinate TH1 of the first interpolation point and the first coordinate TH3 of the third interpolation point, respectively. The second coordinate VA2 of the second interpolation point on the first straight line is calculated based on the first coordinate TH2 of the second interpolation point. The first straight line is the straight line where the first interpolation point and the third interpolation point are located. Calculate the first derivative GD1 of the initial tone mapping curve at the first interpolation point and the first derivative GD3 at the third interpolation point; The first spline curve is determined according to the following function: L is the independent variable of the function; The TH1 is determined according to the following formula: Wherein, avgL is the average brightness of the image to be processed, PvalueL is the first threshold for dark area pixels, PvalueH is the second threshold for the brightness value of dark area pixels (the first threshold is less than the second threshold), TPH is the third threshold for bright area pixels, and TPL is the fourth threshold for bright area pixels.

10. The apparatus according to claim 9, characterized in that, The determining module is also used for: VA1 is determined to be the first coefficient MA1 of the first spline curve. GD1 is determined to be the second coefficient MB1 of the first spline curve; The sum of VA2 and the offset value is determined to be the first coefficient MA2 of the second spline curve, wherein the offset value is 0, a positive number, or a negative number; Based on TH1, TH2, TH3, VA2, VA3, GD3, we obtain MC1, MD1, MB2, MC2, and MD2.

11. The apparatus according to any one of claims 7 to 10, characterized in that, The at least one interpolation point constitutes at least one correction interval, wherein one correction interval corresponds to one correction spline curve, and the tone mapping module is specifically used for: When the pixel to be processed is located in the at least one correction interval, tone mapping is performed on the pixel to be processed based on the correction spline curve; When the pixel to be processed is outside the at least one correction interval, the pixel to be processed is tone mapped based on the initial tone mapping curve.

12. An image processing apparatus, characterized in that, Including processor and transmission interface, The transmission interface is used to receive or send image data; The processor is configured to call program instructions stored in memory and execute the program instructions to perform the method as described in any one of claims 1 to 6.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed on a computer or processor, cause the computer or processor to perform the method as described in any one of claims 1 to 6.

Images