Data processing method and device, electronic equipment, program product and storage medium

Abstract

Embodiments of the present application disclose a data processing method and device, electronic equipment, program product and storage medium; after obtaining the metadata of a to-be-processed video frame, the original tone mapping curve of the to-be-processed video frame is constructed according to the metadata, a current line corresponding to a preset function is selected from at least two segmented lines, a luminance interval corresponding to the current line is determined, the number of interval divisions of the luminance interval is determined according to the number of lines, the luminance interval is divided into sub-luminance intervals of the interval division number according to the interval division number, the target function corresponding to the sub-luminance interval is constructed according to the boundary point of the sub-luminance interval, the current line is updated in the original tone mapping curve based on the target function, and the luminance of the to-be-processed video frame is adjusted according to the updated tone mapping curve, so that the calculation efficiency and energy efficiency ratio can be improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a data processing method, apparatus, electronic device, program product, and storage medium. Background Technology

[0002] In recent years, with the rapid development of the ultra-high-definition video industry, HDR Vivid (High Dynamic Range Vivid), as a high dynamic range standard, has achieved more realistic image quality on various terminals through dynamic metadata-driven tone mapping technology, and has been widely used in mobile video playback, streaming media clients and other scenarios.

[0003] In the process of researching and practicing current technologies, the inventors of this application discovered that when a GPU (Graphics Processing Unit) performs tone mapping on video according to the HDR Vivid standard, due to the inherent structure of the standard, the GPU needs to frequently call threads with limited throughput, resulting in computationally intensive and computationally complex operations, large single-pixel overhead, and difficulty in meeting the requirements of high frame rate real-time processing. Therefore, there is a problem of low computational efficiency. Summary of the Invention

[0004] This application provides a data processing method, a data processing apparatus, an electronic device, a computer program product, and a computer-readable storage medium, which can improve computing efficiency.

[0005] A data processing method, comprising: Obtain the metadata of the video frame to be processed, and construct the original tone mapping curve of the video frame to be processed based on the metadata. The original tone mapping curve includes at least two segmented lines. Filter out the current line corresponding to the preset function from the at least two segmented lines, and determine the brightness range corresponding to the current line; Based on the number of lines, determine the number of interval divisions for the brightness interval, and based on the number of interval divisions, divide the brightness interval into sub-brightness intervals of the specified number of interval divisions; Based on the boundary points of the sub-brightness interval, construct the objective function corresponding to the sub-brightness interval, wherein the computational complexity of the objective function is less than that of the preset function; Based on the objective function, the current line is updated in the original tone mapping curve, and the brightness of the video frame to be processed is adjusted according to the updated tone mapping curve.

[0006] Accordingly, embodiments of this application provide a data processing apparatus, including: An acquisition unit is used to acquire metadata of a video frame to be processed, and construct an original tone mapping curve of the video frame to be processed based on the metadata. The original tone mapping curve includes at least two segmented lines. A filtering unit is used to filter out the current line corresponding to a preset function from the lines of the at least two segments, and to determine the brightness range corresponding to the current line; A division unit is used to determine the number of interval divisions of the brightness interval based on the number of lines, and to divide the brightness interval into sub-brightness intervals based on the number of interval divisions. A construction unit is used to construct an objective function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness interval, wherein the computational complexity of the objective function is less than the computational complexity of the preset function; An adjustment unit is used to update the current line in the original tone mapping curve based on the objective function, and to adjust the brightness of the video frame to be processed according to the updated tone mapping curve.

[0007] In some embodiments, the construction unit may be specifically used to identify the boundary points of the sub-brightness interval in the brightness interval, and call the preset function to calculate the node data of the boundary points; perform polynomial interpolation on the node data to obtain the offset coefficients corresponding to the sub-brightness interval; and construct the target function corresponding to the sub-brightness interval based on the offset coefficients.

[0008] In some embodiments, the construction unit may be specifically used to obtain the current brightness value of the boundary point in the brightness range, and based on the current brightness value, call the preset function to calculate the function value of the boundary point; perform first-order differentiation on the preset function to obtain the analytical derivative function, and based on the current brightness value, call the analytical derivative function to calculate the derivative value of the boundary point; and use the function value and the derivative value as the node data of the boundary point.

[0009] In some embodiments, the construction unit may be specifically used to determine the size of the sub-brightness interval based on the boundary points of the sub-brightness interval, and to filter out the starting boundary points under the target offset direction from the boundary points to obtain the target boundary points; calculate the difference between the function values ​​of the boundary points in the sub-brightness interval to obtain the function value offset under at least one offset direction, the at least one offset direction including the target offset direction; and determine the offset coefficient corresponding to the sub-brightness interval according to the target boundary points, the interval size and the function value offset.

[0010] In some embodiments, the construction unit may be specifically used to determine the starting height of the target line corresponding to the sub-brightness interval based on the function value of the target boundary point; calculate the starting offset direction of the target line corresponding to the sub-brightness interval based on the derivative value of the target boundary point; determine the bending parameter of the target line corresponding to the sub-brightness interval based on the interval size and the function value offset, wherein the bending parameter indicates the degree of bending and the rate of change of bending of the target line; and use the starting height, the starting offset direction, and the bending parameter as the offset coefficient corresponding to the sub-brightness interval.

[0011] In some embodiments, the partitioning unit may be specifically used to identify the total number of lines in the original tone mapping curve and count the current number of lines; calculate the difference between the total number of lines and the current number to obtain the current number difference, and obtain local computing performance information; determine the line number threshold under the current hardware constraints based on the local computing performance information, and obtain the interval partitioning number based on the difference between the line number threshold and the current number difference.

[0012] In some embodiments, the division unit may be specifically used to calculate the ratio between the size of the brightness interval and the number of interval divisions to obtain the target size of each sub-brightness interval; based on the target size, identify the boundary point of each sub-brightness interval in the brightness interval; and divide the brightness interval into the number of sub-brightness intervals according to the boundary points.

[0013] In some embodiments, the adjustment unit may be specifically used to extract the boundary array of each segment and the offset coefficient of the current function corresponding to the line of the segment from the updated tone mapping curve; write the boundary array and the offset coefficient of the current function into the buffer area of ​​the target computing component; obtain the current brightness value of each pixel in the video frame to be processed, and adjust the current brightness value through the target computing component.

[0014] In some embodiments, the adjustment unit may be specifically used to: filter out the target segment corresponding to the current brightness value in the segment based on the boundary array using the target calculation component; extract the target offset coefficient corresponding to the target segment in the buffer area; construct a brightness adjustment function corresponding to the function type based on the target offset coefficient; call the brightness adjustment function through the target calculation component to calculate the adjusted brightness value corresponding to the current brightness value; and adjust the current brightness value of the corresponding pixel in the video frame to be processed to the adjusted brightness value.

[0015] In some embodiments, the adjustment unit may be specifically used to construct a comparison instruction for the boundary point of each segment based on the boundary array, the comparison instruction being used to compare the current brightness value with the boundary point; execute the comparison instruction in parallel by at least one thread of the target calculation component to obtain a comparison result corresponding to each comparison instruction; accumulate the comparison results to obtain the segment index corresponding to the current brightness value, and filter out the target segment corresponding to the current brightness value in the segment based on the segment index.

[0016] In some embodiments, the adjustment unit may be specifically used to sample at least one candidate brightness value in the sub-brightness interval corresponding to the objective function, and call the objective function to calculate the current function value corresponding to the candidate brightness value; based on the candidate brightness value and the current function value, generate the target line corresponding to the objective function; and replace the current line with the target line in the original tone mapping curve to obtain the updated tone mapping curve.

[0017] Furthermore, this application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the data processing method provided in this application.

[0018] Furthermore, embodiments of this application also provide a computer-readable storage medium storing multiple computers adapted for loading by a processor to execute steps in any of the data processing methods provided in embodiments of this application.

[0019] Furthermore, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps in the data processing method provided in embodiments of this application.

[0020] In this embodiment, after obtaining the metadata of the video frame to be processed, an original tone mapping curve of the video frame to be processed can be constructed based on the metadata. The original tone mapping curve includes lines with at least two segments. Then, the current line corresponding to a preset function is selected from the at least two segments, and the brightness interval corresponding to the current line is determined. Based on the number of lines, the number of interval divisions for the brightness interval is determined, and the brightness interval is divided into sub-brightness intervals based on the number of interval divisions. Then, based on the boundary points of the sub-brightness intervals, an objective function corresponding to the sub-brightness intervals is constructed. The computational complexity of the objective function is less than that of the preset function. Finally, based on the objective function, the original tone mapping curve... The current lines are updated, and the brightness of the video frame to be processed is adjusted according to the updated tone mapping curve. Since this solution can reconstruct the computationally complex preset function in the original tone mapping curve into multiple objective functions with lower computational complexity than the preset function, it can reduce the GPU's calls to throughput-limited threads, thereby significantly improving the GPU's resource utilization and significantly reducing the processing overhead of a single pixel. In addition, the brightness of the video frame to be processed can be adjusted by the updated tone mapping curve, so as to significantly improve the overall computational efficiency and energy efficiency of video tone mapping while ensuring that the output image quality is lossless to the human eye. This makes it suitable for high frame rate real-time processing scenarios with strict limitations on computing and power consumption, such as mobile devices. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of a scenario illustrating the data processing method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the data processing method provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the fitting results between the updated tone mapping curve and the original tone mapping curve in multiple scenarios, provided in the embodiments of this application. Figure 4 This is a schematic diagram illustrating the time consumed when the CPU calculates parameters according to an embodiment of this application; Figure 5 This is a schematic diagram of the worst-case fitting result between the updated tone mapping curve and the original tone mapping curve, provided in the embodiments of this application. Figure 6This is another schematic flowchart of the data processing method provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] This application provides a data processing method, a data processing apparatus, an electronic device, a computer program product, and a computer-readable storage medium. The data processing apparatus can be integrated into an electronic device, which may be a server or a terminal, etc.

[0025] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smart TV, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0026] For example, see Figure 1Taking the integration of a data processing device into an electronic device as an example, the electronic device can acquire the metadata of the video frame to be processed and construct the original tone mapping curve of the video frame to be processed based on the metadata. Then, it selects the current line corresponding to the preset function from at least two segmented lines and determines the brightness range corresponding to the current line. Then, it determines the number of interval divisions of the brightness range based on the number of lines and divides the brightness range into sub-brightness intervals based on the number of interval divisions. Then, it constructs the objective function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness intervals. The computational complexity of the objective function is less than that of the preset function. Then, based on the objective function, it updates the current line in the original tone mapping curve and adjusts the brightness of the video frame to be processed based on the updated tone mapping curve. Therefore, it can improve computational efficiency and energy efficiency ratio.

[0027] It is understood that, in the specific embodiments of this application, the data related to the video frames to be processed and the metadata of the video frames to be processed are involved. When the following embodiments of this application are applied to specific products or technologies, permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0028] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0029] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an optional data processing method provided in an embodiment of the present disclosure. The data processing method can be executed by a server or a terminal alone, or it can be executed by a terminal in cooperation with a server. The specific flow of the data processing method can be as follows: 101. Obtain the metadata of the video frame to be processed, and construct the original tone mapping curve of the video frame to be processed based on the metadata.

[0030] In this context, a video frame to be processed can be understood as a video frame that needs to undergo tone mapping after the video has been decoded. A video to be processed can be understood as an image sequence containing at least one video frame, and each video frame to be processed is encoded in HDR Vivid format, with each video frame carrying its own unique metadata.

[0031] Metadata can be understood as a set of parameters describing the brightness distribution characteristics of the video frame to be processed. Specifically, metadata can include descriptive parameters of the lines in each segment of the original tone mapping curve. This metadata guides the generation of original tone mapping curves that correspond one-to-one with each video frame to be processed. In other words, each video frame to be processed uniquely corresponds to a set of metadata and a unique original tone mapping curve.

[0032] The original tone mapping curve comprises at least two segmented lines. It can be understood as a conversion curve used to map the original luminance values ​​of the video frame to the luminance range of the target display device. Each segment can be understood as dividing the entire luminance range [0, 1] into several consecutive luminance intervals based on the magnitude of the luminance value, with each interval corresponding to a mathematical formula. Each line can be understood as a specific function corresponding to each segment, and the entire original tone mapping curve can be formed by connecting these lines from different segments end-to-end.

[0033] Because the tone mapping relationship defined by the HDR Vivid standard is quite complex, it is impossible to use a unified mathematical formula to accurately express the entire brightness range (the brightness range can be represented by the PQ domain [0, 1], compressing the brightness range from the darkest to the brightest into a numerical representation between 0 and 1). Therefore, the original tone mapping curve can be a non-linear mapping curve composed of multiple segments of different types of mathematical formulas. The original tone mapping curve can include lines corresponding to linear segments, lines corresponding to cubic polynomial spline segments, lines corresponding to basic curve segments, and lines corresponding to linear extension segments.

[0034] For example, the original tone mapping curves can be shown in Table 1: Table 1: Functional Expression of the Original Tone Mapping Curve

[0035] As shown in Table 1, the PQ domain [0, 1] can be divided into 7 segments: [0, TH30), [TH30, TH21), [TH21, TH31), [TH31, TH12), [TH12, TH22), [TH22, TH32), and [TH32, 1]. This means that the brightness gradually increases from the first segment [0, TH30] to the last segment [TH32, 1]. Furthermore, each segment's lines correspond to a specific function expression, for example: (1) In the segment [0, TH30), the line of this segment can be a straight line, then the function expression corresponding to this line can be: The function can be a linear function. In this function, y can represent the output brightness value, x can represent the input brightness value, k can represent the slope, and b can represent the intercept. The segmentation [TH32, 1] is similar and will not be elaborated on here.

[0036] (2) In the segment [TH30, TH21), the line of this segment can be a curve, then the function expression corresponding to this line can be: dx = x - TH21, where the function type can be a cubic polynomial function. In this function, y can represent the output brightness value, x can represent the input brightness value, dx can represent the offset of the input brightness value, MD, MC, MB, and MA are the coefficients of the cubic polynomial function. MD can represent the coefficient of the cubic term, used to characterize the rate of change of the curve's curvature, MC can represent the coefficient of the quadratic term, used to characterize the degree of curvature of the curve, MB can represent the coefficient of the linear term, used to characterize the direction of the curve's tilt, and MA can represent the constant term, used to characterize the height of the curve. Furthermore, the segmentation [TH21, TH31), [TH12, TH22), and [TH22, TH32) is similar, and will not be elaborated further here.

[0037] (3) In the segment [TH31, TH12), the line of this segment can be a curve, then the function expression corresponding to this line can be: , dx=x-TH31.

[0038] The function can be a transcendental function. In this function, y can represent the output brightness value, x can represent the input brightness value, dx can represent the offset of the input brightness value, m_a can represent the amplitude scaling factor of the base curve, which is used to characterize the vertical stretching of the curve, m_p can represent the peak brightness factor of the base curve, which is used to characterize the brightness range of the curve, m_n can represent the power exponent of the base curve, which is used to characterize the contrast distribution characteristics of the curve, K1 can represent the first adjustment factor of the base curve, which is used to indicate the shape adjustment of the curve, K2 can represent the second adjustment factor of the base curve, which is used to adjust the curve shape in conjunction with K1, K3 can represent the third adjustment factor of the base curve, which is used to characterize the denominator offset of the curve, m_m can represent the external power exponent of the base curve, which is used to characterize the overall curvature of the curve, and m_b can represent the offset parameter of the base curve, which is used to characterize the vertical translation of the curve.

[0039] Specifically, the method of obtaining metadata for the video frames to be processed can include: For HDR Vivid format videos, the metadata is usually embedded in the SEI (Supplemental Enhancement Information) of the video stream corresponding to the video to be processed. Therefore, based on the HDR Vivid standard, the metadata corresponding to each video frame to be processed can be parsed or derived from the video stream corresponding to the video to be processed. The metadata can include the segmentation points mentioned above (e.g., TH30, TH21, TH31, TH12, TH22, TH32, etc.) and the relevant parameters corresponding to each segment (e.g., k, b, MD, MC, MB, MA, m_p, m_n, K1, K2, K3, m_m, m_b, etc.).

[0040] It's important to note that in actual encoding, metadata isn't necessarily present in every video frame being processed. Metadata is typically updated when scenes change or significant changes occur in brightness distribution, and multiple consecutive frames may share the same set of metadata. Therefore, when acquiring metadata, it's advisable to cache it after each new metadata acquisition. If subsequent frames don't receive new metadata, the cached metadata is used to directly construct the original tone mapping curve until new metadata is received, at which point the original tone mapping curve is updated. This avoids redundant parsing and transmission overhead, thereby further improving processing efficiency.

[0041] After obtaining the metadata, the original tone mapping curve of the video frame to be processed can be constructed based on the metadata. The specific construction method can include: parsing the segmentation points and the relevant parameters corresponding to each segment from the metadata; constructing the function corresponding to each segment line in the PQ domain [0,1] according to the segmentation function defined by the HDR Vivid standard; and finally, splicing the constructed segment functions in order to form a complete original tone mapping curve, which describes the complete mapping relationship from the input brightness value x to the output brightness value y.

[0042] In this context, the GPU (Graphics Processing Unit) can be understood as a processor designed for parallel computing. It's important to note that transcendental functions in the original tone mapping curve (such as exponentiation in the basic curve segment) require calls to special function units (threads) with limited throughput when executed on the GPU, resulting in significant computational overhead. Therefore, while mathematically accurate, directly constructing and executing the original tone mapping curve suffers from low computational efficiency and poor energy efficiency on the GPU, especially in mobile devices and low-power scenarios, making it difficult to meet the requirements of high frame rate real-time processing. Therefore, updating the original tone mapping curve is necessary to reduce the GPU's calls to throughput-limited threads, thereby significantly improving GPU resource utilization, reducing the processing overhead per pixel, and enhancing the overall computational efficiency and energy efficiency of video tone mapping.

[0043] 102. Filter out the current line corresponding to the preset function from at least two segmented lines, and determine the brightness range corresponding to the current line.

[0044] In this context, a preset function can be understood as a function that is pre-defined and needs to be updated or optimized. For example, in the original tone mapping curve, the preset function can be the transcendental function corresponding to the line segment [TH31, TH12). It can be understood that the functions corresponding to the lines of other segments (linear segments, cubic polynomial spline segments) in the original tone mapping curve can also be preset functions.

[0045] Here, the current line can be understood as a segment of the original tone mapping curve that has been filtered out and needs to be updated or optimized. The brightness range can be understood as the range of input brightness values ​​corresponding to the current line in the PQ domain. For example, the brightness range corresponding to the basic curve segment is [TH31, TH12].

[0046] There are several ways to select the current line corresponding to the preset function from lines that are at least two segments, as follows: For example, one could iterate through the mathematical expressions of all lines for function expression analysis (analyzing whether the expression contains transcendental function calls such as pow(), exp(), log(), etc., and identifying lines containing such functions as preset functions). Alternatively, when constructing the original tone mapping curve, each line segment could be labeled with its type (such as TYPE_LINEAR, TYPE_SPLINE, TYPE_BASE_CURVE). Then, when filtering the current line corresponding to the preset function, one could directly match based on this identifier, such as filtering lines labeled TYPE_BASE_CURVE as the current line. Or, one could estimate the instruction overhead or execution cycle on the GPU for each line segment and set a complexity threshold, identifying lines with estimated overhead exceeding the complexity threshold as preset functions that need optimization. Or, one could directly locate the curve based on known curve structure information. In the original tone mapping curve, if the fourth segment [TH31, TH12) is known as the base curve segment, it can be directly identified as the current line, and so on.

[0047] Optionally, after filtering out the current line, the brightness range corresponding to the current line can be determined. Specific determination methods may include: parsing the interval endpoint parameters corresponding to the segment from the metadata (such as the left endpoint TH31 and right endpoint TH12 of the basic curve segment), and using these two endpoint values ​​as the brightness range corresponding to the line. The interval can be represented as a left-closed, right-open, or closed interval, for example, [TH31, TH12).

[0048] It's important to note that the purpose of filtering the current lines corresponding to preset functions is to identify lines in the original tone mapping curve that require optimization. For segments that are already linear or cubic polynomials (computationally friendly), a unified processing method can be chosen based on actual needs; while for segments containing transcendental functions (computationally intensive), focused optimization is required. This filtering mechanism allows for targeted reduction of overall computational overhead while maintaining accuracy.

[0049] 103. Based on the number of lines, determine the number of intervals to be divided into in the brightness interval, and based on the number of intervals, divide the brightness interval into sub-brightness intervals of the specified number of intervals.

[0050] The number of lines can be understood as the total number of segments in the original tone mapping curve. The number of interval divisions can be understood as the number of sub-brightness intervals to be divided into for the current line's corresponding brightness interval. A sub-brightness interval can be understood as each brightness range obtained after dividing the current line's corresponding brightness interval into sub-brightness intervals.

[0051] Specifically, the method of determining the number of brightness interval divisions based on the number of lines may include: identifying the total number of lines in the original tone mapping curve and counting the current number of lines; calculating the difference between the total number of lines and the current number to obtain the current number difference, and obtaining local computing performance information; determining the line number threshold under the current hardware constraints based on the local computing performance information, and obtaining the number of interval divisions based on the difference between the line number threshold and the current number difference.

[0052] The total number of lines can be understood as the total number of segments in the original tone mapping curve. For example, in this application, the total number of lines in the original tone mapping curve can be 7 segments. The current number of lines can be understood as the number of segments corresponding to the preset function in the original tone mapping curve. For example, when the current line corresponds to a transcendental function, the current number can be 1; when the current line corresponds to a linear function, the current number can be 2; or when the current line corresponds to a cubic polynomial function, the current number can be 4. The current number difference can be understood as the total number of lines minus the current number of lines. In this application, the current number difference can be represented by n_spline_intervals.

[0053] Among these, local computing performance information can be understood as the current GPU's computing power parameters, such as core frequency, number of ALUs (Arithmetic Logic Units), number of SFUs, and memory bandwidth. Current hardware constraints can be understood as the upper limit of processing power determined by the current GPU's local computing performance information; for example, the difference in parallel computing capabilities between mobile GPUs and desktop GPUs. The line count threshold can be understood as the maximum number of segments allowed in the entire tone mapping curve under current hardware constraints.

[0054] Optionally, in some embodiments, there can be multiple ways to obtain local computing performance information, such as: for example, by calling the interface of the operating system or driver to query the current GPU device information and obtain parameters such as the number of cores, frequency, and architecture generation, local computing performance information can be obtained; or, a set of benchmark tests can be run during computer program initialization, and the computing power of the current GPU can be evaluated based on the test results, thereby obtaining local computing performance information; or, by reading system configuration information and combining it with a preset hardware performance database, the performance level of the current device can be determined, thereby obtaining local computing performance information, and so on.

[0055] Optionally, in some embodiments, after obtaining local computing performance information, the line number threshold under the current hardware constraints can be determined based on the local computing performance information. There are various ways to determine the threshold, such as: for example, a mapping table between performance level and line number threshold can be established in advance, with lower-end devices corresponding to smaller line number thresholds and higher-end devices corresponding to larger line number thresholds. Alternatively, the line number threshold can be determined based on the number of SFUs and ALUs of the current GPU. For example, the fewer the number of SFUs, the more segments are needed to approximate the transcendental function (more segments can avoid using SFUs), and the more ALUs, the more segments can be used to approximate the transcendental function. Alternatively, the line number threshold can be set to a certain multiple of the number of GPU cores, or the line number threshold can be dynamically adjusted based on actual test data, and so on.

[0056] It's important to note that determining the number of intervals is a trade-off between accuracy and efficiency. More segments result in higher approximation accuracy but also greater computational overhead; fewer segments result in higher computational efficiency but may increase approximation error. Therefore, by incorporating local computational performance information, the optimal number of intervals can be adaptively determined for different hardware platforms, maximizing computational efficiency while ensuring perceptual quality (ΔE_ITP < 1.0 JND).

[0057] Once the number of intervals for the brightness interval is determined, the brightness interval can be divided into sub-brightness intervals according to that number of intervals. There are several ways to divide the brightness interval, for example: (1) Uniform division: The ratio between the size of the brightness interval and the number of interval divisions can be calculated to obtain the target size of each sub-brightness interval; then, based on the target size, the boundary points of each sub-brightness interval are identified in the brightness interval; according to the boundary points, the brightness interval is divided into the number of sub-brightness intervals of the interval division.

[0058] The interval size can be understood as the total width of the brightness interval corresponding to the current line. Taking the current line as the base curve segment as an example, if the brightness interval corresponding to the base curve segment is [TH31, TH12), then the interval size can be expressed as TH12-TH31. The target size can be understood as the width of each sub-brightness interval. The boundary point of the sub-brightness interval can be understood as the brightness value of the dividing point between each sub-brightness interval.

[0059] Once the target size is obtained, the boundary points of each sub-brightness interval can be identified within the brightness interval based on the target size. There are several ways to identify these boundary points, such as: starting from the left endpoint, increasing the target size by one at a time to obtain each boundary point sequentially until the right endpoint is reached; or, using an arithmetic sequence, the target size can be used to evenly divide the brightness interval into the number of sub-intervals; or, the boundary point array can be pre-calculated and generated in batches by assigning values ​​in a loop, and so on.

[0060] After identifying the boundary points, duplicate boundary points can be removed, and the deduplicated boundary points can be sorted in ascending order of brightness value, serving as the boundaries of each sub-brightness interval, thereby dividing the entire brightness interval into the number of sub-brightness intervals.

[0061] It should be noted that this uniform division method is simple to implement, has low computational cost, and is suitable for scenarios where the curve changes relatively uniformly. For the basic curve segment, since its shape is generally smooth, uniform division can achieve a good approximation effect.

[0062] Furthermore, additional nodes are uniformly placed within the brightness interval corresponding to the basic curve segment, subdividing the brightness interval into multiple sub-brightness intervals. Each sub-brightness interval can be approximated by a cubic polynomial function to approximate its original transcendental function. It is understandable that the more sub-brightness intervals there are, the narrower the target size of each sub-brightness interval, resulting in higher approximation accuracy, and the approximation error can be reduced to O(h). 4 The rate of decrease (where h can be the target size of the sub-brightness interval) means that as the number of segments increases moderately, the approximation accuracy will be significantly improved, while the computational cost only increases linearly, resulting in good cost-effectiveness. By reasonably selecting the number of segments, computational efficiency can be maximized while ensuring lossless perception (ΔE_ITP<1.0 JND).

[0063] (2) Non-uniform division: Based on the degree of local change of the current line, fewer boundary points (i.e., larger sub-interval width) can be set in areas with low degree of local change, and more boundary points (i.e., smaller sub-interval width) can be set in areas with high degree of local change. Then, based on the boundary points, the brightness interval can be divided into sub-brightness intervals of the interval division number.

[0064] The degree of local change can be understood as the magnitude of the second derivative (curvature) of the transcendental function corresponding to the current line. The larger the absolute value of the second derivative, the more drastic the curve change, requiring denser sampling points to ensure approximation accuracy; the smaller the absolute value of the second derivative, the more gradual the curve change, requiring only sparse sampling points to ensure approximation accuracy.

[0065] Optional, non-uniform division methods can be varied, such as: For example, the distribution of the second derivative of the transcendental function corresponding to the current line within the brightness range can be calculated, and the boundary point density can be increased in areas with larger absolute values ​​of the second derivative and decreased in areas with smaller second derivatives. Alternatively, several coarse segments can be uniformly divided first, and the approximation error at the midpoint of each segment can be calculated. If the error exceeds a preset threshold, the segment can be further subdivided until the error of the width of all sub-intervals meets the requirements. Alternatively, the boundary point position can be determined based on the rate of change of the slope of the curve, and more sub-interval widths can be allocated to areas with rapid slope changes. Alternatively, based on the characteristics of human visual perception, more sub-interval widths can be allocated to areas with sensitive brightness perception (such as the medium brightness range), and fewer sub-interval widths can be allocated to areas with low sensitivity (such as extremely dark or extremely bright areas), and so on.

[0066] It should be noted that after determining the number of intervals, the brightness interval can be divided into multiple sub-brightness intervals using either uniform or non-uniform partitioning. Uniform partitioning is simple to implement and has low computational overhead, making it suitable for basic curve segments; non-uniform partitioning, on the other hand, adaptively adjusts the node distribution based on the degree of local variation in the curve, achieving higher approximation accuracy with the same segmented budget, and is suitable for scenarios where the curve variation is uneven.

[0067] Furthermore, non-uniform partitioning can achieve higher approximation accuracy than uniform partitioning under the same piecewise budget because computational resources are concentrated on the most complex and difficult-to-approximate regions of the curve. However, non-uniform partitioning is more complex to implement, requiring additional analysis and computation in the CPU (Central Processing Unit). The CPU can be understood as the computational and control core of a computer system, responsible for executing instructions, processing data, scheduling tasks, and managing system resources. In this application, the CPU can be used to parse HDR Vivid metadata to calculate boundary points and their corresponding Hermite interpolations (i.e., offset coefficients) to transform the transcendental function into multiple cubic polynomial functions. In this application, since the overall shape of the basic curve segment is relatively smooth and monotonic, uniform partitioning can already meet the requirement of perceptual losslessness (ΔE_ITP<1.0JND), therefore, uniform partitioning is a more preferred implementation method.

[0068] 104. Based on the boundary points of the sub-brightness intervals, construct the objective function corresponding to the sub-brightness intervals.

[0069] The objective function can be understood as a function that replaces the preset function in the sub-brightness range, and the computational complexity of the objective function is less than that of the preset function.

[0070] For example, taking a preset function as a transcendental function, the objective function can be a cubic polynomial function, for the sub-brightness interval [x i x i+1 The corresponding objective function can be expressed as: y = a0 + a1·t + a2·t² + a3·t³, or y = ((a3·t + a2)·t + a1)·t + a0, where t = x i The computational complexity of this cubic polynomial function is significantly less than that of a transcendental function.

[0071] In other words, in a GPU, because the preset function involves multiple exponentiation operations, executing the preset function requires calling a Special Function Unit (SFU). However, the SFU has limited throughput and high execution latency. In contrast, executing a cubic polynomial function only requires three multiply-add operations (FMA instructions), executed by the Arithmetic Logic Unit (ALU), without calling the SFU, significantly reducing computational complexity. Therefore, the preset function (transcendental function) within the brightness range can be transformed into the objective function (cubic polynomial function) corresponding to multiple sub-brightness ranges, thereby greatly improving GPU execution efficiency while maintaining approximation accuracy.

[0072] Specifically, the method for constructing the objective function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness interval can be described in the following steps S41-S43: Step S41: Identify the boundary points of the sub-brightness intervals within the brightness interval, and call a preset function to calculate the node data of the boundary points.

[0073] Among them, node data can be understood as the key information determined from a preset function at each boundary point of the sub-brightness interval, which is used to uniquely identify a cubic polynomial. It can include the function value and the first derivative value of the boundary point.

[0074] There are several ways to identify the boundary points of sub-brightness intervals within a brightness interval. For example, the brightness value of each boundary point can be calculated sequentially based on the left endpoint, right endpoint, and width of the sub-interval. Alternatively, a pre-generated list of boundary points can be read directly based on a preset node distribution strategy (such as curvature adaptation). Or, the left and right endpoints of each sub-interval can be directly obtained from the divided sub-brightness intervals as boundary points, and so on.

[0075] Optionally, in some embodiments, after identifying the boundary points of the sub-brightness interval, a preset function can be called to calculate the node data of the boundary points. The method of calculating the node data may specifically include: obtaining the current brightness value of the boundary point in the brightness interval, and based on the current brightness value, calling the preset function to calculate the function value of the boundary point, then performing a first-order derivative on the preset function to obtain an analytical derivative function, and based on the current brightness value, calling the analytical derivative function to calculate the derivative value of the boundary point, and then using the function value and the derivative value as the node data of the boundary point.

[0076] Here, the current brightness value can be understood as the brightness coordinates of the boundary point in the PQ domain, and can be represented by x. The function value can be understood as the output brightness value obtained by substituting the boundary point brightness value into a preset function, and can be represented by y. The analytical derivative function can be understood as the derivative expression obtained by taking the first derivative of the preset function. The derivative value can be understood as the slope value obtained by substituting the boundary point brightness value into the analytical derivative function, and can be represented by m.

[0077] Optionally, in some embodiments, after obtaining the current brightness value, it can be substituted into a preset function to calculate the function value of the boundary point. Taking the preset function as the function corresponding to the base curve segment as an example, the current brightness value can be substituted into the function corresponding to the base curve segment to obtain the function value corresponding to the current brightness value.

[0078] The method for obtaining the analytical derivative function by taking the first derivative of the preset function can be as follows: (1) When the preset function is the function corresponding to the basic curve segment, when taking the first derivative of the preset function, the chain rule can be used to expand it precisely. For example, we can let , If r = u / v, then the analytic derivative function corresponding to this preset function can be: .

[0079] (2) When the preset function is the function corresponding to a cubic polynomial spline segment, the first derivative of the preset function can be taken, and the analytic derivative function corresponding to the preset function can be: .

[0080] (3) When the preset function is the function corresponding to a linear segment or a linear extension segment, the first derivative of the preset function can be taken, and the analytical derivative function corresponding to the preset function can be: y` = k.

[0081] After obtaining the analytical derivative function, the current brightness value can be substituted into the analytical derivative function to calculate the derivative value at the boundary point. Taking the function corresponding to the base curve segment as an example, the current brightness value can be substituted into the analytical derivative function corresponding to the base curve segment to obtain the derivative value corresponding to the current brightness value.

[0082] What can be understood is that, for the boundary points [x] of the sub-brightness range i x i+1 ], boundary point x i The function value is y i Boundary point x i The derivative value is m i Boundary point x i+1 The function value is y i+1 Boundary point x i+1 The derivative value is m i+1 The node data of the boundary points of the sub-brightness interval can include [y i m i y i+1 m i+1 ].

[0083] Step S42: Perform polynomial interpolation on the node data to obtain the offset coefficients corresponding to the sub-brightness intervals.

[0084] Polynomial interpolation can be understood as constructing a cubic polynomial function using known nodal data (function values ​​and derivative values) so that it satisfies the given function values ​​and derivative values ​​within the corresponding brightness range. For example, Hermite cubic interpolation is a typical polynomial interpolation method, which can uniquely determine a cubic polynomial function through four constraints (function values ​​and derivative values ​​at both endpoints).

[0085] The offset coefficient can be understood as the local offset coordinate t=xx used in the Hermite cubic polynomial. i The polynomial coefficients at time t can include a0, a1, a2, and a3, and can be directly used to construct the corresponding cubic polynomial function.

[0086] The method of obtaining the offset coefficients corresponding to the sub-brightness interval by performing polynomial interpolation on the node data can specifically include: determining the interval size of the sub-brightness interval based on the boundary points of the sub-brightness interval, selecting the starting boundary point under the target offset direction from the boundary points to obtain the target boundary point, then calculating the difference between the function values ​​of the boundary points in the sub-brightness interval to obtain the function value offset under at least one offset direction, and then determining the offset coefficients corresponding to the sub-brightness interval based on the target boundary point, the interval size and the function value offset.

[0087] The offset direction can be understood as the local coordinate direction within the sub-brightness interval, such as offsetting to the right from the left endpoint or offsetting to the left from the right endpoint. At least one offset direction includes a target offset direction, which can be understood as the direction from the left endpoint to the right endpoint of the sub-brightness interval. The function value offset can be understood as the difference between the function values ​​at the two endpoints of the sub-brightness interval, and may include the function value offset y in the target offset direction. i+1 -y i And the function value offset y in another offset direction. i -y i+1 .

[0088] Optionally, in some implementations, after obtaining the target boundary point, interval size, and function value offset, the offset coefficient corresponding to the sub-brightness interval can be determined based on the target boundary point, interval size, and function value offset. Specifically, the determination method can be as follows: determine the starting height of the target line corresponding to the sub-brightness interval based on the function value of the target boundary point; calculate the starting offset direction of the target line corresponding to the sub-brightness interval based on the derivative value of the target boundary point; then determine the bending parameter of the target line corresponding to the sub-brightness interval based on the interval size and function value offset; and finally, use the starting height, starting offset direction, and bending parameter as the offset coefficient corresponding to the sub-brightness interval.

[0089] Here, the target line can be understood as the curve corresponding to the constructed objective function. The initial height can be understood as the output brightness value at the left endpoint of the sub-brightness interval, i.e., the function value a0 of the objective function at t=0. The initial offset direction can be understood as the slope of the curve at the left endpoint of the sub-brightness interval, i.e., the derivative value a1 of the objective function at t=0. The curvature parameter indicates the degree of curvature and the rate of change of curvature of the target line. The degree of curvature can be understood as the extent to which the target line deviates from a straight line within the sub-brightness interval, mainly controlled by a2. The larger |a2| is, the more pronounced the curvature of the target line. The rate of change of curvature can be understood as the degree of uneven distribution of curvature along the sub-brightness interval, mainly controlled by a3. a3 determines whether the curvature of the target line is concentrated on the left or right side of the sub-brightness interval.

[0090] For example, regarding the boundary point [x] of the sub-brightness range i x i+1 The node data of the boundary points of the sub-brightness interval can include [y]. i m i y i+1 m i+1 Then, the offset coefficients [a0, a1, a2, a3] corresponding to the sub-brightness intervals can be determined by the following formula, where h can represent the size of the sub-brightness interval, h = x i+1 - x i : (1) a0 = yi; (2) a1 = mi; (3) a2={3(y i+1 - y i ) - h(2m i + m i+1 )} / h²; (4) a2 = {2(y i - y i+1 ) + h(m i + m i+1 )} / h³.

[0091] It should be noted that for segmentations in the original curve that are themselves cubic polynomial spline segments and linear segments, the interpolation result is mathematically identical to the original curve (the approximation error is the machine precision). Therefore, it is optional to update this type of line or not. For transcendental function segmentations, as the size h of the sub-brightness interval decreases, the approximation error decreases by approximately O(h). 4 The rate of decrease of ) enables lossless perceptual approximation with fewer segments (ΔE_ITP<1.0JND).

[0092] Step S43: Construct the objective function corresponding to the sub-brightness interval based on the offset coefficient.

[0093] After obtaining the offset coefficients for the sub-brightness interval, the calculated offset coefficients can be substituted into the cubic polynomial function to construct the objective function corresponding to the sub-brightness interval.

[0094] It should be noted that, through steps S41-S43, the preset function (such as the transcendental function in the basic curve segment) is replaced with multiple piecewise cubic polynomial functions (target functions). Each sub-brightness interval corresponds to a set of cubic polynomial coefficients. The above target functions can have the following characteristics: uniform form, all of which are cubic polynomial functions; simple calculation, requiring only addition and multiplication; controllable approximation accuracy, meeting the requirements of lossless perception.

[0095] 105. Based on the objective function, update the current line in the original tone mapping curve, and adjust the brightness of the video frame to be processed according to the updated tone mapping curve.

[0096] The updated tone mapping curve can be understood as a new tone mapping curve obtained by replacing the lines corresponding to the preset functions (such as the basic curve segments) in the original tone mapping curve with the lines corresponding to the target functions. The function type of the current function corresponding to each segment of the updated tone mapping curve is the same as the function type of the target function (i.e., both are cubic polynomial functions). The current function can include the target function. For example, taking the basic curve segment as an example, in the original tone mapping curve, this segment is a transcendental function. After the update, this segment is replaced with multiple piecewise cubic polynomial functions (target functions). Each cubic polynomial spline segment has the same function type as the linear segments and cubic polynomial spline segments in the original tone mapping curve. Here, the same function type can refer to the function type of a low-degree polynomial, which can include linear functions and cubic polynomial functions.

[0097] The method of updating the current line in the original tone mapping curve based on the objective function can be as follows: sample at least one candidate brightness value in the sub-brightness interval corresponding to the objective function, and call the objective function to calculate the current function value corresponding to the candidate brightness value. Then, based on the candidate brightness value and the current function value, generate the target line corresponding to the objective function. Finally, replace the current line with the target line in the original tone mapping curve to obtain the updated tone mapping curve.

[0098] Here, candidate brightness values ​​can be understood as brightness sampling points selected within a sub-brightness interval for verifying or generating target lines. The current function value can be understood as the output brightness value calculated by substituting the candidate brightness values ​​into the objective function.

[0099] It should be noted that, since the objective function is an exact approximation of the preset function within the sub-luminance range (Hermite interpolation ensures a perfect match between the function value and its derivative at the boundary points), the updated tone mapping curve at the boundary points seamlessly connects with the original tone mapping curve without any jumps or sharp angles. By replacing the lines of the preset function with the lines of multiple objective functions, the entire tone mapping curve is unified into a curve composed of piecewise cubic polynomials, each segment having the same mathematical expression.

[0100] Optionally, in some implementations, after obtaining the updated tone mapping curve, the brightness of the video frame to be processed can be adjusted according to the updated tone mapping curve. Specific adjustment methods may include: extracting the boundary array of each segment and the offset coefficient of the current function corresponding to the line of the segment from the updated tone mapping curve; then writing the boundary array and the offset coefficient of the current function into the buffer area of ​​the target calculation component; then obtaining the current brightness value of each pixel in the video frame to be processed, and adjusting the current brightness value through the target calculation component.

[0101] In this context, the boundary array for each segment can be understood as the set of brightness values ​​at the endpoints of each brightness interval and sub-brightness interval. For example, the boundary array could be [x, x1, x2, ..., x...]. n-1 x n The offset coefficients of the current function can be understood as the offset coefficients [a0, a1, a2, a3] corresponding to each segment. The target computation component can be understood as the GPU, specifically a shader or computation core within the GPU. The buffer area of ​​the target computation component can be understood as a constant buffer or texture buffer within the GPU, used to store the boundary array and offset coefficients for fast access by the shader.

[0102] It should be noted that in this application, a CPU and a GPU can be configured. The CPU can be used to divide the brightness range of the preset function in the original tone mapping curve to obtain the boundary points and the offset coefficients corresponding to each sub-brightness range. The boundary point data and the offset coefficients corresponding to each sub-brightness range can then be uploaded to the buffer in the GPU so that the GPU can directly call and calculate them.

[0103] The method of extracting the boundary array of each segment and the offset coefficient of the current function corresponding to the line of the segment from the updated tone mapping curve can specifically include: traversing all segments of the updated tone mapping curve, collecting the left and right boundary points of each segment, and sorting them after deduplication to obtain the boundary array; at the same time, collecting the Hermite interpolation corresponding to each segment, i.e., the offset coefficients [a0, a1, a2, a3], and storing them as a coefficient array in segment order, so as to extract the offset coefficient of the current function corresponding to the line of the segment.

[0104] Specifically, writing the boundary array and the offset coefficients of the current function into the buffer area of ​​the target computing component can include: calling interfaces provided by graphics APIs (such as OpenGL, Vulkan, DirectX, or Metal) to copy the boundary array of each segment and the offset coefficients of the current function corresponding to the lines of that segment from the CPU-side memory to the GPU-side constant buffer. For mobile devices, interfaces such as OpenGL ES's glBufferData or Vulkan's vkCmdUpdateBuffer can be used to complete the data transmission. Since metadata is usually updated every frame or every few frames, this transmission overhead is minimal.

[0105] It should be noted that after writing the boundary array of each segment and the offset coefficient of the current function corresponding to the segment line into the GPU constant buffer, all shader threads in the GPU can access this data in read-only mode without repeated transmission, thereby significantly reducing the communication overhead between the CPU and GPU.

[0106] The method of obtaining the current brightness value of each pixel in the video frame to be processed and adjusting the current brightness value through the target calculation component can specifically include: based on the boundary array, the target calculation component filters out the target segment corresponding to the current brightness value in the segmentation; then, the target offset coefficient corresponding to the target segment is extracted in the buffer area; based on the target offset coefficient, a brightness adjustment function corresponding to the function type is constructed; then, the target calculation component calls the brightness adjustment function to calculate the adjusted brightness value corresponding to the current brightness value; and finally, the current brightness value of the corresponding pixel in the video frame to be processed is adjusted to the adjusted brightness value.

[0107] Here, the target segment can be understood as the brightness range to which the current pixel brightness value belongs. The target offset coefficient can be understood as the offset coefficient [a0, a1, a2, a3] corresponding to the target segment. The brightness adjustment function can be understood as a cubic polynomial function used to calculate the output brightness value. The adjusted brightness value can be understood as the output brightness value obtained by substituting the current brightness value into the brightness adjustment function.

[0108] It should be noted that since the brightness adjustment functions corresponding to all segments have the same form (except for the linear segment and the linear extension segment, they are all Horner evaluation forms of cubic polynomials), with only the offset coefficients being different, the GPU can reuse the same evaluation code and obtain the offset coefficients by looking up a table. There is no need to write different calculation logic for each segment, which is the key to eliminating conditional branches and avoiding warp divergence.

[0109] For example, the evaluation code can be as follows: float 4c = coeffs[seg]; / / Get the offset coefficient float t = x - x_bounds[seg]; / / Local offset calculation float y = ((cw ·t + cz) · t + cy) · t + cx; / / Horner evaluation.

[0110] Specifically, the method of filtering the target segment corresponding to the current brightness value in the segments based on the boundary array and using the target calculation component can include: constructing a comparison instruction for the boundary point of each segment based on the boundary array, executing the comparison instruction in parallel by at least one thread of the target calculation component to obtain the comparison result corresponding to each comparison instruction, then accumulating the comparison results to obtain the segment index corresponding to the current brightness value, and filtering the target segment corresponding to the current brightness value in the segments based on the segment index.

[0111] The comparison instruction can be used to compare the current brightness value with the boundary point. The comparison result can be understood as a Boolean value (0 or 1) output by each comparison instruction, indicating whether the current brightness value is greater than or equal to the boundary point. The segment index can be understood as the integer value obtained by accumulating the comparison results. This value directly corresponds to the position of the target segment in the coefficient array (for example, index 0 represents the first segment, index 1 represents the second segment, and so on).

[0112] It should be noted that using the cumulative comparison method can determine the target segment and has the following advantages: all threads execute the exact same sequence of comparison instructions, eliminating branch divergence and fully utilizing GPU parallel efficiency; the comparison instructions are compiled into efficient step or comparison instructions on the GPU, resulting in extremely low execution latency. Compared with the nested if-else branches of related technologies, the embodiments of this application transform the branch prediction problem into simple integer cumulative operations, fundamentally solving the warp divergence problem.

[0113] For example, a comparison instruction can be as follows: int seg = 0; seg += (x>= x_bounds[1]) ? 1 : 0; seg += (x>= x_bounds[2]) ? 1 : 0; ... / / A total of N-1 comparisons.

[0114] It is understood that this application decomposes cubic polynomial evaluation into 3 multiply-add operations, corresponding one-to-one with the GPU's FMA (Fused Multiply-Add) hardware unit. Each FMA instruction can complete one multiplication and one addition within a single clock cycle. Therefore, the GPU computational overhead of the entire tone mapping process is: (1) 3 MAD / FMA instructions (Horner evaluation) and (2) N-1 comparison instructions, totaling approximately 7 instructions per pixel, replacing several layers of conditional branches and 3 pow() function calls in the original shader. Each pow() function call expands to exp2(y·log2(x)) at the bottom layer, requiring 2 SFU instructions. The 3 pow() calls total approximately 6 SFU instructions. Including branch judgment and arithmetic operations, the number of instructions per pixel in related technologies far exceeds that of the embodiments in this application.

[0115] Furthermore, it can be referred to Figure 3 , Figure 4 , Figure 5 ,in, Figure 3This allows you to fit the updated tone mapping curve to the original tone mapping curve in multiple scenes (Standard Scene_600, Bright Scene_600, Dark Scene_600, Basic Curve Segment_600, Multi-Spline Curve Segment_600, Spline Segment_600), as shown below. Figure 3 As shown, the fitting results of the two are basically consistent. The updated tone mapping curve and the original tone mapping curve (defined by the HDR Vivid standard) are highly consistent in their overall trend, and the difference is indistinguishable visually. Therefore, it can be shown that this application has excellent accuracy in curve approximation. Figure 4 The time spent calculating parameters on the CPU side is controlled within a low range under various test configurations (such as standard scene_600, bright scene_600, dark scene_600, etc.), which is far lower than the processing time budget per frame. Therefore, it can be shown that the pre-computation overhead of this application is negligible and will not become a performance bottleneck. Figure 5 Even in the worst-case scenario, the JND is still within the range that is imperceptible to the human eye. As shown in the figure, the maximum perceptual color difference ΔE_ITP between the updated tone mapping curve and the original tone mapping curve is less than 1.0 JND (just perceptible difference), and even in the worst case, it does not exceed JND. Therefore, it can be shown that the approximation error of this application is completely lossless in perception, and the difference in image quality cannot be perceived by the human eye.

[0116] In summary, this application not only eliminates Warp divergence through cumulative comparison, but also frees up special function units (SFUs) with limited throughput on the GPU by replacing transcendental functions with polynomial evaluation, thus transferring computational tasks to a larger number of arithmetic logic units (ALUs) with higher throughput, thereby significantly improving the computational efficiency and energy efficiency of the GPU.

[0117] Therefore, it can be widely applied to various terminal products requiring real-time HDR tone mapping, including but not limited to: HDR video players: enabling smooth playback, low power consumption, and low heat generation when playing HDR Vivid format videos, improving the user viewing experience; video editing software: enabling real-time and efficient tone mapping when previewing or exporting HDR video clips, improving editing efficiency; streaming media clients: providing high-quality, high-frame-rate HDR video playback capabilities on mobile and TV devices; set-top boxes: achieving stable HDR Vivid decoding and display output on resource-constrained embedded platforms; smart TVs: meeting the real-time processing requirements of the HDR Vivid standard on low-power, low-cost TV chips while reducing system power consumption. Furthermore, it is particularly suitable for mobile devices, low-power scenarios, and GPU scenarios with limited computing resources, effectively solving the problems of low computational efficiency and poor energy efficiency in related technologies.

[0118] As can be seen from the above, after obtaining the metadata of the video frame to be processed, this embodiment of the application can construct the original tone mapping curve of the video frame to be processed based on the metadata. The original tone mapping curve includes lines with at least two segments. Then, the current line corresponding to the preset function is selected from the lines with at least two segments, and the brightness interval corresponding to the current line is determined. Based on the number of lines, the number of interval divisions for the brightness interval is determined, and based on the number of interval divisions, the brightness interval is divided into sub-brightness intervals. Then, based on the boundary points of the sub-brightness intervals, the objective function corresponding to the sub-brightness intervals is constructed. Furthermore, the computational complexity of the objective function is less than the computational complexity of the preset function. Finally, based on the objective function, the original tone mapping curve is... The curve updates the current line and adjusts the brightness of the video frame to be processed based on the updated tone mapping curve. Since this scheme can reconstruct the computationally complex preset function in the original tone mapping curve into multiple objective functions with lower computational complexity than the preset function, it can reduce the GPU's calls to throughput-limited threads, thereby significantly improving the GPU's resource utilization and significantly reducing the processing overhead of a single pixel. In addition, the brightness of the video frame to be processed can be adjusted by the updated tone mapping curve, so as to significantly improve the overall computational efficiency and energy efficiency of video tone mapping while ensuring that the output image quality is lossless to the human eye. This makes it suitable for high frame rate real-time processing scenarios with strict limitations on computing and power consumption, such as mobile devices.

[0119] Based on the method described in the above embodiments, the following examples will provide further detailed explanations.

[0120] In this embodiment, the data processing device will be specifically integrated into an electronic device, which is a server. The server may integrate a CPU and a GPU. The video frame to be processed may be a video frame in an HDR Vivid format video. The preset function is a transcendental function, and the target function is a cubic polynomial function. like Figure 6 As shown, a data processing method has the following specific steps: 201. The CPU acquires the metadata of the video frame to be processed.

[0121] For example, the CPU can parse or derive the metadata corresponding to each frame of the video to be processed from the video stream corresponding to the video to be processed, based on the HDR Vivid standard. The metadata may include the division points of the above segments (e.g., TH30, TH21, TH31, TH12, TH22, TH32, etc.) and the relevant parameters corresponding to each segment (e.g., k, b, MD, MC, MB, MA, m_p, m_n, K1, K2, K3, m_m, m_b, etc.).

[0122] 202. The CPU constructs the original tone mapping curve of the video frame to be processed based on the metadata.

[0123] For example, the CPU parses the segmentation points and the relevant parameters corresponding to each segment from the metadata. Based on these parameters, according to the segmentation function defined by the HDR Vivid standard, it constructs the function corresponding to each segment line in the PQ domain [0, 1] segment by segment. Finally, the constructed segment functions are spliced ​​together in order to form a complete original tone mapping curve, which describes the complete mapping relationship from the input brightness value x to the output brightness value y.

[0124] Because the tone mapping relationship defined by the HDR Vivid standard is quite complex, it is impossible to use a unified mathematical formula to accurately express the entire brightness range (the brightness range can be represented by the PQ domain [0, 1], compressing the brightness range from the darkest to the brightest into a numerical representation between 0 and 1). Therefore, the original tone mapping curve can be a non-linear mapping curve composed of multiple segments of different types of mathematical formulas. The original tone mapping curve can include lines corresponding to linear segments, lines corresponding to cubic polynomial spline segments, lines corresponding to basic curve segments, and lines corresponding to linear extension segments.

[0125] 203. The CPU selects the current line corresponding to the transcendental function from at least two segmented lines and determines the brightness range corresponding to the current line.

[0126] For example, the CPU can iterate through the mathematical expressions of all lines for function expression analysis (it can analyze whether the expression contains transcendental function calls such as pow(), exp(), log(), and identify lines containing such functions as transcendental functions). Alternatively, when constructing the original tone mapping curve, each line segment can be labeled with its type (such as TYPE_LINEAR, TYPE_SPLINE, TYPE_BASE_CURVE). Then, when filtering the current line corresponding to the transcendental function, it can directly match based on this identifier, such as filtering the line labeled TYPE_BASE_CURVE as the current line. Or, it can estimate the instruction overhead or execution cycle on the GPU for each line segment and set a complexity threshold, identifying lines whose estimated overhead exceeds the complexity threshold as transcendental functions that need optimization, and so on.

[0127] Optionally, in some embodiments, the CPU parses the interval endpoint parameters corresponding to the segment (such as the left endpoint TH31 and right endpoint TH12 of the basic curve segment) from the metadata, and uses these two endpoint values ​​as the brightness interval corresponding to the line. The interval can be represented as a left-closed and right-open or closed interval, for example, [TH31, TH12).

[0128] 204. The CPU determines the number of brightness intervals based on the number of lines.

[0129] The CPU identifies the total number of lines in the original tone mapping curve and counts the current number of lines; it calculates the difference between the total number of lines and the current number to obtain the current number difference and acquires local computing performance information; based on the local computing performance information, it determines the line number threshold under the current hardware constraints, and based on the difference between the line number threshold and the current number difference, it obtains the number of interval divisions.

[0130] 205. The CPU divides the brightness interval into sub-brightness intervals based on the number of interval divisions.

[0131] After determining the number of intervals to divide the brightness range, the CPU can then divide the brightness range into sub-brightness ranges according to that number of intervals. There are several ways to divide it, for example: (1) Uniform division: The CPU can calculate the ratio between the size of the brightness interval and the number of interval divisions to obtain the target size of each sub-brightness interval; then, based on the target size, the boundary point of each sub-brightness interval is identified in the brightness interval; according to the boundary point, the brightness interval is divided into the number of sub-brightness intervals of the interval division.

[0132] (2) Non-uniform division: The CPU can set fewer boundary points (i.e., larger sub-interval width) in areas with low local variation and more boundary points (i.e. smaller sub-interval width) in areas with high local variation based on the degree of local variation of the current line. Then, based on the boundary points, the brightness interval can be divided into sub-brightness intervals of the interval division number.

[0133] 206. The CPU constructs a cubic polynomial function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness interval.

[0134] For example, taking the preset function as a transcendental function, the objective function can be a cubic polynomial function, for the sub-brightness interval [x i x i+1 The corresponding objective function can be expressed as: y = a0 + a1·t + a2·t² + a3·t³, or y = ((a3·t + a2)·t + a1)·t + a0, where t = x i The computational complexity of this cubic polynomial function is significantly less than that of a transcendental function.

[0135] Optionally, in some embodiments, the CPU identifies the boundary points of the sub-brightness intervals within the brightness interval, obtains the current brightness value of the boundary points within the brightness interval, and, based on the current brightness value, calls a transcendental function to calculate the function value of the boundary points. Then, it performs a first-order derivative on the transcendental function to obtain the analytic derivative function, and, based on the current brightness value, calls the analytic derivative function to calculate the derivative value of the boundary points. Finally, the function value and the derivative value are used as the node data of the boundary points.

[0136] Optionally, in some embodiments, after obtaining the node data of the boundary points, the CPU can determine the size of the sub-brightness interval based on the boundary points of the sub-brightness interval, and filter out the starting boundary points under the target offset direction from the boundary points to obtain the target boundary points. Then, it calculates the difference between the function values ​​of the boundary points in the sub-brightness interval to obtain the function value offset under at least one offset direction. Then, based on the function value of the target boundary points, it determines the starting height of the target line corresponding to the sub-brightness interval. Based on the derivative value of the target boundary points, it calculates the starting offset direction of the target line corresponding to the sub-brightness interval. Then, based on the interval size and the function value offset, it determines the bending parameter of the target line corresponding to the sub-brightness interval. Finally, it uses the starting height, the starting offset direction, and the bending parameter as the offset coefficient corresponding to the sub-brightness interval.

[0137] Optionally, in some embodiments, after obtaining the offset coefficients for the sub-brightness interval, the CPU can substitute the calculated offset coefficients into the cubic polynomial function to construct the cubic polynomial function corresponding to the sub-brightness interval.

[0138] 207. The CPU updates the current line in the original tone mapping curve based on a cubic polynomial function to obtain the updated tone mapping curve.

[0139] The CPU samples at least one candidate brightness value in the sub-brightness interval corresponding to the cubic polynomial function, and calls the cubic polynomial function to calculate the current function value corresponding to the candidate brightness value. Then, based on the candidate brightness value and the current function value, the target line corresponding to the cubic polynomial function is generated. Finally, the current line is replaced with the target line in the original tone mapping curve to obtain the updated tone mapping curve.

[0140] 208. The CPU writes the boundary array in the updated tone mapping curve and the offset coefficient of the current function to the GPU's buffer area.

[0141] After obtaining the updated tone mapping curve, the CPU can traverse all segments of the updated tone mapping curve, collect the left and right boundary points of each segment, and obtain a boundary array after deduplication and sorting. At the same time, it collects the Hermite interpolation corresponding to each segment, i.e., the offset coefficients [a0, a1, a2, a3], and stores them as a coefficient array in segment order, so as to extract the offset coefficients of the corresponding current function based on the line of the segment. Then, the boundary array and the offset coefficients of the current function are written into the buffer area of ​​the GPU.

[0142] 209. The GPU adjusts the brightness of the video frame to be processed using the boundary array in the buffer area and the offset coefficient of the current function.

[0143] The GPU obtains the current brightness value of each pixel in the video frame to be processed, and constructs a comparison instruction for the boundary point of each segment based on the boundary array. The comparison instruction is executed in parallel by at least one thread of the target computing component to obtain the comparison result corresponding to each comparison instruction. Then, the comparison results are accumulated to obtain the segment index corresponding to the current brightness value, and the target segment corresponding to the current brightness value is selected from the segments based on the segment index.

[0144] Optionally, in some embodiments, after the GPU selects the target segment corresponding to the current brightness value, it can extract the target offset coefficient corresponding to the target segment in the buffer area, and construct a brightness adjustment function corresponding to the function type based on the target offset coefficient. Then, the brightness adjustment function is called through the target calculation component to calculate the adjusted brightness value corresponding to the current brightness value. After that, the current brightness value of the corresponding pixel in the video frame to be processed is adjusted to the adjusted brightness value.

[0145] As can be seen from the above, in this embodiment of the application, after the CPU obtains the metadata of the video frame to be processed, it can construct the original tone mapping curve of the video frame to be processed based on the metadata. The original tone mapping curve includes lines with at least two segments. Then, the current line corresponding to the transcendental function is selected from the lines with at least two segments, and the luminance interval corresponding to the current line is determined. Based on the number of lines, the number of interval divisions for the luminance interval is determined, and based on the number of interval divisions, the luminance interval is divided into sub-luminance intervals. Then, based on the boundary points of the sub-luminance intervals, a cubic polynomial function corresponding to the sub-luminance interval is constructed. Furthermore, the computational complexity of the cubic polynomial function is less than that of the transcendental function. Finally, based on the cubic polynomial function, in the original tone mapping curve... The current lines are updated, and the GPU can adjust the brightness of the video frame to be processed based on the updated tone mapping curve. Since the CPU in this scheme can reconstruct the computationally complex transcendental function in the original tone mapping curve into multiple cubic polynomial functions with lower computational complexity than the transcendental function, the GPU's calls to throughput-limited threads can be reduced, thereby significantly improving the GPU's resource utilization and significantly reducing the processing overhead per pixel. In addition, the GPU can also adjust the brightness of the video frame to be processed based on the updated tone mapping curve, so as to significantly improve the overall computational efficiency and energy efficiency of video tone mapping while ensuring that the output image quality is lossless to the human eye. This makes it suitable for high frame rate real-time processing scenarios with strict limitations on computing and power consumption, such as mobile devices.

[0146] To better implement the above methods, this application also provides a data processing apparatus.

[0147] For example, such as Figure 7 As shown, the data processing device may include an acquisition unit 301, a filtering unit 302, a division unit 303, a construction unit 304, and an adjustment unit 305, as follows: (1) Obtain unit 301; The acquisition unit 301 is used to acquire the metadata of the video frame to be processed, and construct the original tone mapping curve of the video frame to be processed based on the metadata. The original tone mapping curve includes at least two segmented lines.

[0148] (2) Screening unit 302; The filtering unit 302 is used to filter out the current line corresponding to the preset function from at least two segmented lines and determine the brightness range corresponding to the current line.

[0149] (3) Divide into units 303; The division unit 303 is used to determine the number of interval divisions of the brightness interval based on the number of lines, and to divide the brightness interval into sub-brightness intervals based on the number of interval divisions.

[0150] For example, the partitioning unit 303 can be used to identify the total number of lines in the original tone mapping curve and count the current number of lines. Then, it calculates the difference between the total number of lines and the current number to obtain the current number difference and acquires local computing performance information. Then, based on the local computing performance information, it determines the line number threshold under the current hardware constraints and obtains the interval partitioning number based on the difference between the line number threshold and the current number difference. Then, it calculates the ratio between the interval size of the brightness interval and the interval partitioning number to obtain the target size of each sub-brightness interval. Then, based on the target size, it identifies the boundary points of each sub-brightness interval in the brightness interval. Finally, based on the boundary points, it divides the brightness interval into sub-brightness intervals of the interval partitioning number.

[0151] (4) Constructing unit 304; The construction unit 304 is used to construct the objective function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness interval. The computational complexity of the objective function is less than that of the preset function.

[0152] For example, the construction unit 304 can be specifically used to identify the boundary points of sub-brightness intervals within a brightness interval, obtain the current brightness value of the boundary points within the brightness interval, and, based on the current brightness value, call a preset function to calculate the function value of the boundary points. Then, perform a first-order derivative on the preset function to obtain an analytical derivative function, and, based on the current brightness value, call the analytical derivative function to calculate the derivative value of the boundary points. Finally, use the function value and the derivative value as the node data of the boundary points. Then, based on the boundary points of the sub-brightness intervals, determine the interval size of the sub-brightness intervals, and select the starting boundary points in the target offset direction from the boundary points to obtain the target boundary points. Finally, calculate the difference between the function values ​​of the boundary points in the sub-brightness intervals. Obtain the function value offset in at least one offset direction, including the target offset direction. Then, determine the offset coefficient corresponding to the sub-brightness interval based on the target boundary point, interval size, and function value offset. Next, determine the starting height of the target line corresponding to the sub-brightness interval based on the function value of the target boundary point. Then, calculate the starting offset direction of the target line corresponding to the sub-brightness interval based on the derivative value of the target boundary point. Then, determine the curvature parameter of the target line corresponding to the sub-brightness interval based on the interval size and function value offset. The curvature parameter indicates the degree of curvature and the rate of change of curvature of the target line. Finally, use the starting height, starting offset direction, and curvature parameter as the offset coefficient corresponding to the sub-brightness interval.

[0153] (5) Adjustment unit 305 is used to update the current line in the original tone mapping curve based on the objective function, and adjust the brightness of the video frame to be processed according to the updated tone mapping curve.

[0154] For example, adjustment unit 305 can be specifically used to sample at least one candidate brightness value in the sub-brightness interval corresponding to the target function, and call the target function to calculate the current function value corresponding to the candidate brightness value. Then, based on the candidate brightness value and the current function value, a target line corresponding to the target function is generated. Then, the current line is replaced with the target line in the original tone mapping curve to obtain an updated tone mapping curve. Then, the boundary array of each segment and the offset coefficient of the current function corresponding to the line of the segment are extracted from the updated tone mapping curve. Then, the boundary array and the offset coefficient of the current function are written into the buffer area of ​​the target calculation component. Then, the current brightness value of each pixel in the video frame to be processed is obtained, and based on the boundary array, a target function for each segment is constructed. The comparison instructions for segment boundary points are used to compare the current brightness value with the boundary point. Then, the comparison instructions are executed in parallel by at least one thread of the target calculation component to obtain the comparison result corresponding to each comparison instruction. The comparison results are then accumulated to obtain the segment index corresponding to the current brightness value, and the target segment corresponding to the current brightness value is selected from the segments based on the segment index. Then, the target offset coefficient corresponding to the target segment is extracted from the buffer area, and a brightness adjustment function corresponding to the function type is constructed based on the target offset coefficient. Then, the brightness adjustment function is called by the target calculation component to calculate the adjusted brightness value corresponding to the current brightness value. Finally, the current brightness value of the corresponding pixel in the video frame to be processed is adjusted to the adjusted brightness value.

[0155] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0156] As can be seen from the above, the acquisition unit 301 in this embodiment can acquire the metadata of the video frame to be processed and construct the original tone mapping curve of the video frame to be processed based on the metadata. Then, the filtering unit 302 can filter out the current line corresponding to the preset function from at least two segmented lines and determine the brightness interval corresponding to the current line. Then, the division unit 303 can determine the number of interval divisions of the brightness interval based on the number of lines and divide the brightness interval into sub-brightness intervals based on the number of interval divisions. Then, the construction unit 304 can construct the target function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness interval. The computational complexity of the target function is less than that of the preset function. Then, the adjustment unit 305 can update the current line in the original tone mapping curve based on the target function and adjust the brightness of the video frame to be processed based on the updated tone mapping curve. Therefore, the computational efficiency and energy efficiency ratio can be improved.

[0157] This application also provides an electronic device, such as... Figure 8 As shown, it illustrates a schematic diagram of the structure of an electronic device involved in an embodiment of this application. Specifically: The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and computer programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0158] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0159] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0160] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0161] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 runs the computer programs stored in the memory 402 to realize various functions.

[0162] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0163] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0164] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of computer programs that can be loaded by a processor to execute steps in any of the data processing methods provided in embodiments of this application.

[0165] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0166] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0167] Since the computer program stored in the computer-readable storage medium can execute the steps of any of the data processing methods provided in the embodiments of this application, the beneficial effects that any of the data processing methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0168] According to one aspect of this application, a computer program product or computer program is provided, comprising a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various alternative implementations of the data processing described above.

[0169] The foregoing has provided a detailed description of a data processing method, data processing apparatus, electronic device, computer program product, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data processing method, characterized in that, include: Obtain the metadata of the video frame to be processed, and construct the original tone mapping curve of the video frame to be processed based on the metadata. The original tone mapping curve includes at least two segmented lines. The video frame to be processed is encoded in the Jingcai image format. Filter out the current line corresponding to the preset function from the at least two segmented lines, and determine the brightness range corresponding to the current line; Determining the number of interval divisions for the brightness range based on the number of lines includes: acquiring local computing performance information; determining a line number threshold under the current hardware constraints based on the local computing performance information; and obtaining the number of interval divisions based on the difference between the line number threshold and the current number difference, wherein the current number difference is the difference between the total number of lines and the current number of lines. Based on the number of interval divisions, the brightness interval is divided into sub-brightness intervals of the number of interval divisions; Based on the boundary points of the sub-brightness interval, construct the objective function corresponding to the sub-brightness interval, wherein the computational complexity of the objective function is less than that of the preset function; Based on the objective function, the current line is updated in the original tone mapping curve, and the brightness of the video frame to be processed is adjusted according to the updated tone mapping curve.

2. The data processing method according to claim 1, characterized in that, The step of constructing the objective function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness interval includes: Identify the boundary points of the sub-brightness intervals within the brightness intervals, and call the preset function to calculate the node data of the boundary points; Polynomial interpolation is performed on the node data to obtain the offset coefficients corresponding to the sub-brightness intervals; Based on the offset coefficient, construct the objective function corresponding to the sub-brightness interval.

3. The data processing method according to claim 2, characterized in that, The step of calling the preset function to calculate the node data of the boundary points includes: Obtain the current brightness value of the boundary point within the brightness range, and based on the current brightness value, call the preset function to calculate the function value of the boundary point; The first derivative of the preset function is obtained to obtain the analytical derivative function, and the derivative value of the boundary point is calculated by calling the analytical derivative function based on the current brightness value; The function value and the derivative value are used as the node data of the boundary point.

4. The data processing method according to claim 2, characterized in that, The step of performing polynomial interpolation on the node data to obtain the offset coefficients corresponding to the sub-brightness intervals includes: Based on the boundary points of the sub-brightness interval, the interval size of the sub-brightness interval is determined, and the starting boundary point under the target offset direction is selected from the boundary points to obtain the target boundary point; Calculate the difference between function values ​​at boundary points in the sub-brightness interval to obtain the function value offset in at least one offset direction, wherein the at least one offset direction includes the target offset direction; The offset coefficient corresponding to the sub-brightness interval is determined based on the target boundary point, the interval size, and the function value offset.

5. The data processing method according to claim 4, characterized in that, The step of determining the offset coefficient corresponding to the sub-brightness interval based on the target boundary point, the interval size, and the function value offset includes: Based on the function value of the target boundary point, determine the starting height of the target line corresponding to the sub-brightness interval; Based on the derivative value of the target boundary point, calculate the starting offset direction of the target line corresponding to the sub-brightness interval; Based on the interval size and the function value offset, the bending parameters of the target line corresponding to the sub-brightness interval are determined, and the bending parameters indicate the degree of bending and the rate of change of bending of the target line; The starting height, the starting offset direction, and the bending parameter are used as the offset coefficients corresponding to the sub-brightness range.

6. The data processing method according to claim 1, characterized in that, Before obtaining the interval division quantity based on the difference between the line quantity threshold and the current quantity difference, the method further includes: The total number of lines in the original tone mapping curve is identified, and the current number of the current lines is counted. Calculate the difference between the total number of lines and the current number to obtain the current number difference.

7. The data processing method according to claim 1, characterized in that, The step of dividing the brightness interval into sub-brightness intervals according to the specified number of interval divisions includes: Calculate the ratio between the size of the brightness interval and the number of interval divisions to obtain the target size of each sub-brightness interval; Based on the target size, the boundary points of each of the sub-brightness intervals are identified in the brightness interval; Based on the boundary points, the brightness range is divided into the number of sub-brightness ranges specified in the interval division.

8. The data processing method according to claim 1, characterized in that, The function type of the current function corresponding to each segment of the updated tone mapping curve is the same as the function type of the target function, and the current function includes the target function. Adjusting the brightness of the video frame to be processed according to the updated tone mapping curve includes: Extract the boundary array of each segment and the offset coefficient of the current function corresponding to the line of the segment from the updated tone mapping curve; Write the boundary array and the offset coefficient of the current function into the buffer area of ​​the target computing component; The current brightness value of each pixel in the video frame to be processed is obtained, and the current brightness value is adjusted by the target calculation component.

9. The data processing method according to claim 8, characterized in that, The step of adjusting the current brightness value through the target calculation component includes: Based on the boundary array, the target segment corresponding to the current brightness value is selected from the segment by the target calculation component; Extract the target offset coefficient corresponding to the target segment from the buffer area, and construct the brightness adjustment function corresponding to the function type based on the target offset coefficient; The target calculation component calls the brightness adjustment function to calculate the adjusted brightness value corresponding to the current brightness value; In the video frame to be processed, the current brightness value of the corresponding pixel is adjusted to the adjusted brightness value.

10. The data processing method according to claim 9, characterized in that, The step of filtering out the target segment corresponding to the current brightness value from the segments based on the boundary array using the target calculation component includes: Based on the boundary array, a comparison instruction is constructed for the boundary point of each segment, the comparison instruction being used to compare the current brightness value with the size between the boundary point; The comparison instructions are executed in parallel by at least one thread of the target computing component to obtain a comparison result corresponding to each comparison instruction; The comparison results are summed to obtain the segment index corresponding to the current brightness value, and the target segment corresponding to the current brightness value is selected from the segments based on the segment index.

11. The data processing method according to claim 1, characterized in that, The step of updating the current line in the original tone mapping curve based on the objective function includes: At least one candidate brightness value is sampled in the sub-brightness interval corresponding to the objective function, and the objective function is called to calculate the current function value corresponding to the candidate brightness value; Based on the candidate brightness values ​​and the current function value, generate the target line corresponding to the target function; The current line is replaced with the target line in the original tone mapping curve to obtain the updated tone mapping curve.

12. A data processing apparatus, characterized in that, include: The acquisition unit is used to acquire the metadata of the video frame to be processed, and construct the original tone mapping curve of the video frame to be processed based on the metadata. The original tone mapping curve includes at least two segmented lines, and the video frame to be processed is encoded in the Jingcai image format. A filtering unit is used to filter out the current line corresponding to a preset function from the lines of the at least two segments, and to determine the brightness range corresponding to the current line; A segmentation unit is used to determine the number of intervals to divide the brightness interval based on the number of lines, including: acquiring local computing performance information; determining a line number threshold under the current hardware constraints based on the local computing performance information, and obtaining the number of intervals based on the difference between the line number threshold and the current number difference, wherein the current number difference is the difference between the total number of lines and the current number of lines; and dividing the brightness interval into sub-brightness intervals according to the number of intervals. A construction unit is used to construct an objective function corresponding to the sub-brightness interval based on the boundary points of the sub-brightness interval, wherein the computational complexity of the objective function is less than the computational complexity of the preset function; An adjustment unit is used to update the current line in the original tone mapping curve based on the objective function, and to adjust the brightness of the video frame to be processed according to the updated tone mapping curve.

13. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor running the computer program in the memory to perform the steps of the data processing method according to any one of claims 1 to 11.

14. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the data processing method according to any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer programs adapted for loading by a processor to perform the steps of the data processing method according to any one of claims 1 to 11.

Images