A method, apparatus, equipment, and medium for optimizing control points of tone mapping curves.

By constructing and executing tone mapping curves in the linear domain and optimizing the spatial distribution of control points, the intra-segment shape deviation caused by inconsistency in coordinate systems in HDR video frame processing is resolved, achieving more accurate image quality optimization.

CN123123732APending Publication Date: 2026-11-17MALANSHAN AUDIO & VIDEO LABORATORY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611277875.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In existing HDR video frame processing schemes, the selection and execution of control points are carried out in the same coordinate system, which causes the piecewise curve function to be non-closed under the transformation between the perceptual domain and the linear domain, resulting in shape deviation within the segment.

Method used

The construction and mapping of the tone mapping curve are both set in the linear domain. By obtaining the initial control point set in the perceptual domain and converting it into a linear domain control point set, the tone mapping curve is constructed. The spatial distribution characteristics of the control points are optimized using the target value, and iterative adjustments are made until the preset iteration stop condition is met.

Benefits of technology

This method solves the problem of intra-segment shape deviation caused by the non-closed form of piecewise curve functions. The optimization target directly reflects the final output image quality, unifies the error distribution of brightness segments, and avoids the problem of the optimization target being out of touch with the actual display image quality in traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123732A_ABST
    Figure CN123123732A_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, device, and medium for optimizing control points of tone mapping curves, relating to the field of HDR video frame processing. The method includes: acquiring a current initial control point set in the perceptual domain and converting it into a linear domain control point set; constructing a corresponding tone mapping curve in the linear domain based on the linear domain control point set, and performing mapping processing on the input content using the tone mapping curve to obtain the corresponding mapped content; determining the target value corresponding to the mapped content, and adjusting the spatial distribution characteristics of the control points in the current initial control point set with the target value as the guide to obtain an updated control point set; if the updated control point set meets a preset iteration stopping condition, then the updated control point set is determined as the final optimized control point set. By placing both curve construction and mapping execution in the linear domain, the method solves the problem of intra-segment shape deviation caused by the non-closed function form of piecewise curves.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of HDR video frame processing, and in particular to a method, apparatus, device, and medium for optimizing tone mapping curve control points. Background Technology

[0002] HDR (High-Dynamic Range) conversion (tone mapping) generally uses piecewise curves, the shape of which is determined by a set of control points and their constraints. The selection of control points is usually based on content statistics.

[0003] In existing HDR video frame processing schemes, the statistical domain and the execution domain are consistent, and the selection of control points and the execution of curves are completed in the same coordinate system, without any coordinate transformation. If the selected points are in the perceptual quantization (PQ) domain and the execution is in the linear optical domain, the existing technology has the problem that the piecewise curve function class is not closed under the transformation between the perceptual domain and the linear domain, resulting in the problem of shape deviation within the segment. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for optimizing control points of tone mapping curves, which solves the problem of intra-segment shape deviation caused by the non-closed function form of piecewise curves by setting both curve construction and mapping execution in the linear domain. The specific solution is as follows: Firstly, this application provides a method for optimizing control points of tone mapping curves, including: The current initial control point set under the perceptual domain is obtained, and the current initial control point set is converted into the linear domain control point set corresponding to the linear domain; wherein, the linear domain is the signal space in which the brightness value of the video frame pixel is linearly proportional to the physical light intensity, and the perceptual domain is the nonlinear signal space obtained by applying a perceptual quantization transfer function to the brightness value of the linear domain. Based on the linear domain control point set, a corresponding tone mapping curve is constructed in the linear domain, and the tone mapping curve is used to perform mapping processing on the input content to obtain the corresponding mapped content; the input content is the pixel brightness signal of the video frame in the linear domain. A target value corresponding to the mapped content is determined, and the spatial distribution characteristics of the control points in the current initial control point set are adjusted based on the optimization of the target value to obtain an updated control point set; the target value represents the quality deviation of the mapped content relative to a preset reference standard. Determine whether the updated control point set meets the preset iteration stop condition. If the updated control point set does not meet the preset iteration stop condition, then determine the updated control point set as the new current initial control point set, and jump to the step of converting the current initial control point set into the linear domain control point set corresponding to the linear domain. If the updated control point set satisfies the preset iteration stop condition, then the updated control point set is determined as the final optimized control point set.

[0005] Optionally, determining the target value corresponding to the mapped content includes: The target value corresponding to the mapped content is determined using a supervised evaluation method or an unsupervised evaluation method; wherein, the supervised evaluation method is an evaluation method that compares the mapped content with the reference image frame pixel by pixel when a reference image frame exists, and the unsupervised evaluation method is an evaluation method that inputs the mapped content into a preset perceptual quality model to obtain the target value when no reference image frame exists.

[0006] Optionally, when the supervised evaluation method is used, the target value is the image domain error between the mapped content and the reference image frame; the image domain error characterizes the pixel-by-pixel difference between image frames. Accordingly, adjusting the spatial distribution characteristics of the control points in the current initial control point set to optimize the target value, thereby obtaining the updated control point set, includes: With the goal of reducing the image domain error, the positions of the control points in the current initial control point set are adjusted to obtain the updated control point set.

[0007] Optionally, when the unsupervised evaluation method is used, the target value is the perceived quality score output by the preset perceived quality model; Accordingly, adjusting the spatial distribution characteristics of the control points in the current initial control point set to optimize the target value, thereby obtaining the updated control point set, includes: With the goal of increasing the perceived quality score, the positions of the control points in the current initial control point set are adjusted to obtain the updated control point set.

[0008] Optionally, adjusting the spatial distribution characteristics of the control points in the current initial control point set with the goal of optimizing the target value includes: Keeping the total number of control points in the current initial control point set unchanged, the position coordinates of at least one target control point in the perception domain are adjusted, and the endpoint slope or shape parameter of the base curve associated with the target control point is adjusted; wherein, the endpoint slope is used to define the tangent direction of the tone mapping curve at the control point, and the shape parameter of the base curve is used to define the curvature of the curve segment between adjacent control points.

[0009] Optionally, adjusting the spatial distribution characteristics of the control points in the current initial control point set with the goal of optimizing the target value includes: If the target value still does not reach the preset expected value after adjusting the coordinate position of the target control point in the perception domain, then at least one new control point is added to the current initial control point set in the perception domain.

[0010] Optionally, adjusting the endpoint slope or shape parameters of the base curve associated with the target control point includes: The endpoint slope or the shape parameter of the base curve is adjusted under preset constraints; wherein, the constraints include monotonicity constraints and smoothness constraints; the monotonicity constraints are used to maintain the monotonically increasing or monotonically decreasing characteristics of the tone mapping curve on each curve segment, and the smoothness constraints are used to limit the slope change of adjacent curve segments at the common control point to not exceed a preset threshold.

[0011] Secondly, this application provides a tone mapping curve control point optimization device, comprising: The control point set conversion module is used to obtain the current initial control point set in the perceptual domain and convert the current initial control point set into a linear domain control point set corresponding to the linear domain; wherein, the linear domain is a signal space in which the brightness value of the video frame pixel is linearly proportional to the physical light intensity, and the perceptual domain is a nonlinear signal space obtained by applying a perceptual quantization transfer function to the brightness value of the linear domain. The mapping curve construction module is used to construct a corresponding tone mapping curve in the linear domain based on the linear domain control point set, and to perform mapping processing on the input content using the tone mapping curve to obtain the corresponding mapped content; the input content is the pixel brightness signal of the video frame in the linear domain. The target value determination module is used to determine the target value corresponding to the mapped content, so as to optimize the spatial distribution characteristics of the control points in the current initial control point set as a guide to obtain the updated control point set; the target value represents the quality deviation of the mapped content relative to the preset reference standard; The step jump module is used to determine whether the updated control point set meets the preset iteration stop condition. If the updated control point set does not meet the preset iteration stop condition, the updated control point set is determined as the new current initial control point set, and the process jumps to the step of converting the current initial control point set into the linear domain control point set corresponding to the linear domain. The control point set determination module is used to determine the updated control point set as the final optimized control point set if the updated control point set meets the preset iteration stop condition.

[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the aforementioned tone mapping curve control point optimization method.

[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the aforementioned tone mapping curve control point optimization method.

[0014] This application first obtains the current initial control point set in the perceptual domain and converts it into a linear domain control point set corresponding to the linear domain. The linear domain is a signal space where the brightness value of a video frame pixel is linearly proportional to the physical light intensity, and the perceptual domain is a nonlinear signal space obtained by applying a perceptual quantization transfer function to the brightness value of the linear domain. Then, based on the linear domain control point set, a corresponding tone mapping curve is constructed in the linear domain, and the tone mapping curve is used to perform mapping processing on the input content to obtain the corresponding mapped content. The input content is the video frame pixel brightness signal in the linear domain. Finally, a target value corresponding to the mapped content is determined. The spatial distribution characteristics of control points in the current initial control point set are adjusted with the target value as the guide to obtain an updated control point set. The target value represents the quality deviation of the mapped content relative to a preset reference standard. Then, it is determined whether the updated control point set meets the preset iteration stopping condition. If the updated control point set does not meet the preset iteration stopping condition, the updated control point set is determined as the new current initial control point set, and the process jumps to the step of converting the current initial control point set into the linear domain control point set corresponding to the linear domain. If the updated control point set meets the preset iteration stopping condition, the updated control point set is determined as the final optimized control point set. Therefore, this application solves the problem of segment shape deviation caused by the non-closed form of piecewise curve functions due to the non-linearity of the transfer function between the perceptual domain and the linear domain by setting both curve construction and mapping execution in the linear domain; by using the target value of the mapped content relative to the preset reference standard as the optimization guide, the optimization target directly reflects the final output image quality rather than the fitting error of the intermediate domain, thus solving the problem of the traditional method ignoring the execution domain error and causing the optimization target to be out of touch with the actual display image quality; by iteratively adjusting the spatial distribution of the control points in the perceptual domain until the target value is met, the error of each brightness segment is constrained by the unified output target value, thus solving the problem of error distribution imbalance caused by the extreme unevenness of the linear domain segment width. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a method for optimizing control points of a tone mapping curve disclosed in this application. Figure 2 This is a schematic diagram of a tone mapping curve control point optimization device disclosed in this application. Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 As shown, this embodiment of the invention discloses a method for optimizing control points of a tone mapping curve, including: Step S11: Obtain the current initial control point set under the perceptual domain, and convert the current initial control point set into the linear domain control point set corresponding to the linear domain; wherein, the linear domain is the signal space in which the brightness value of the video frame pixel is linearly proportional to the physical light intensity, and the perceptual domain is the nonlinear signal space obtained by applying a perceptual quantization transfer function to the brightness value of the linear domain.

[0019] The core of this embodiment lies in incorporating the domain transformation and linear domain reconstruction steps into the optimization loop of control point selection. This allows the optimization objective to directly reflect the end-to-end perceptual domain error, rather than simply optimizing the fitting error of the selected domain. Specifically, this embodiment no longer treats point selection and transformation as two separate sequential steps, but rather links them into a closed loop. For any set of candidate control points, the entire process is completed, including perceptual domain candidate point transformation, transformation to the linear domain (coordinates and slopes are transformed together), linear actuator curve reconstruction, obtaining the mapping execution result using the curve, and using content-oriented target evaluation values ​​to obtain the measurement error. This end-to-end error is then used to guide the adjustment of the control points.

[0020] Content-oriented goals come from two sources: Supervised: When a reference image or target image exists, it is judged as "the image domain error between the result image and the reference image" (which can be pixel-by-pixel, partition-by-partition, or weighted).

[0021] Unsupervised: When there is no reference, the execution result is directly fed into the unsupervised perceptual quality criterion (such as brightness zone preservation, local contrast / detail preservation, text or salient area protection, peak and dark area constraints, etc.), and the value of the criterion is used as the evaluation. No reference is required throughout the process.

[0022] This embodiment provides two implementation methods: 1. Complete end-to-end optimization scheme: Taking the end-to-end error as the target, the position of the control point (and the optional endpoint slope) is continuously adjusted iteratively to minimize the end-to-end error.

[0023] 2. Lightweight error constraint scheme: First, select points using conventional methods, then verify the end-to-end error. If the error exceeds the standard, automatically add points. Repeat the verification until the standard is met or the number of control points reaches the limit.

[0024] The two options can be flexibly selected based on computing resources and accuracy requirements.

[0025] In this embodiment, the content-oriented end-to-end feedback link is common in both optimization schemes. For a set of candidate control points, the steps to calculate their content-oriented end-to-end target values ​​include (1) determining the source of the evaluation target, (2) domain transformation of the candidate control points, (3) online domain construction and execution curves, and (4) content-oriented evaluation.

[0026] Among them, (1) determine the source of the evaluation target (choose one, depending on the system form): Supervised: Prepare a reference image / target image Iref; the reference image / target image Iref refers to an image known to have good mapping effect, which is used as a benchmark for evaluation and is compared pixel by pixel with the execution result of the current tone mapping curve to quantify the mapping quality of the current curve.

[0027] Unsupervised: Prepare unsupervised perceived quality criteria J (applied to the execution result graph, no reference required).

[0028] (2) Domain transformation of candidate control points:

[0029] Let the candidate control points (i.e., the control points in the initial control point set) have x_pq as their x-coordinate and y_pq as their y-coordinate in the perceptual domain, and k_pq as their endpoint slope. Transforming to the linear domain, we obtain the linear domain control point set: x-axis: x_lin = EOTF(x_pq); The ordinate is: y_lin = EOTF(y_pq); Slope: The slope k_lin of the linear domain is obtained by following the chain rule.

[0030] The aforementioned EOTF (Electro-Optical Transfer Function) is a photoelectric conversion function. Through the above two steps, the initial control point set in the sensing domain can be converted into a linear domain control point set in the linear domain.

[0031] Step S12: Based on the linear domain control point set, construct a corresponding tone mapping curve in the linear domain, and use the tone mapping curve to perform mapping processing on the input content to obtain the corresponding mapped content; the input content is the video frame pixel brightness signal in the linear domain.

[0032] (3) Construct and execute curves in the linear domain: Using the transformed linear domain control points (x_lin, y_lin) and shape parameters (such as slope k_lin, or the shape parameters of the base curve), each segment is constructed according to the parameterized basis function form (polynomial / spline, base curve / rational form, etc.) used for each segment, resulting in the linear domain curve G_lin (i.e., the tone mapping curve). G_lin is then used as a linear actuator on the input content Iin_lin to obtain the execution result image (i.e., the mapped content) Iout_lin = G_lin(Iin_lin). The input content is the pixel brightness signal of the video frame in the linear domain; the base curve refers to a predefined curve with a specific mathematical form, serving as a template for generating the final tone mapping curve.

[0033] Step S13: Determine the target value corresponding to the mapped content, and adjust the spatial distribution characteristics of the control points in the current initial control point set to optimize the target value, so as to obtain the updated control point set; the target value represents the quality deviation of the mapped content relative to the preset reference standard.

[0034] In this embodiment, determining the target value corresponding to the mapped content includes: using a supervised evaluation method or an unsupervised evaluation method to determine the target value corresponding to the mapped content; wherein, the supervised evaluation method is an evaluation method that compares the mapped content with the reference image frame pixel by pixel when a reference image frame exists, and the unsupervised evaluation method is an evaluation method that inputs the mapped content into a preset perceptual quality model to obtain the target value when no reference image frame exists.

[0035] When a supervised evaluation method is used, the objective value is the image domain error between the mapped content and the reference image frame; the image domain error characterizes the pixel-by-pixel difference between image frames. Accordingly, the spatial distribution characteristics of the control points in the current initial control point set are adjusted with the goal of optimizing the target value to obtain the updated control point set, including: adjusting the position of the control points in the current initial control point set with the goal of reducing image domain error, so as to obtain the updated control point set.

[0036] When using an unsupervised evaluation method, the target value is the perceived quality score output by the preset perceived quality model; Accordingly, the spatial distribution characteristics of the control points in the current initial control point set are adjusted with the optimization target value as the guide to obtain the updated control point set, including: adjusting the position of the control points in the current initial control point set with the goal of increasing the perception quality score, in order to obtain the updated control point set.

[0037] Specifically, (4) Content-oriented evaluation: The comparison is performed within a domain consistent with the evaluation objective, and a domain transformation is only performed on the execution result to align with the domain where the evaluation objective resides, when necessary. For example, when the reference graph / criteria is defined in the perception domain, it is used... Transform Iout to the receptive domain before comparison. If the evaluation target is already defined in the linear domain, this transformation is unnecessary.

[0038] Supervised: The target value D is the image domain error between the execution result and the reference image Iref. A feasible algorithm is as follows: ; Where pixel_w represents the pixel weight, which can be designed adaptively.

[0039] Unsupervised: The objective value D = J(Iout, Iin). Here, the objective value is the perceived quality score output by the pre-defined perceived quality model, meaning the unsupervised perception criterion J (i.e., the pre-defined perceived quality model) directly affects the execution result. A feasible algorithm for J is: J = λ1 Lum +λ2 Contrast +λ3 Text +λ4 PeakDark, or J, is composed of brightness zone preservation, local contrast / detail preservation, text / salient area protection, and peak and dark area constraint terms, which are added together with weights λ. The weights λ can be adaptively designed according to the content.

[0040] In addition, the spatial distribution characteristics of the control points in the current initial control point set are adjusted with the optimization target value as the guide. This includes: keeping the total number of control points in the current initial control point set unchanged, adjusting the position coordinates of at least one target control point in the perception domain, and adjusting the endpoint slope or shape parameter of the base curve associated with the target control point. The endpoint slope is used to define the tangent direction of the tone mapping curve at the control point, and the shape parameter of the base curve is used to define the curvature of the curve segment between adjacent control points.

[0041] The process of adjusting the endpoint slope or shape parameters of the base curve associated with the target control point can specifically include: adjusting the endpoint slope or shape parameters of the base curve under preset constraints; wherein, the constraints include monotonicity constraints and smoothness constraints; the monotonicity constraints are used to maintain the monotonically increasing or monotonically decreasing characteristics of the tone mapping curve on each curve segment, and the smoothness constraints are used to limit the slope change of adjacent curve segments at the common control point to not exceed a preset threshold.

[0042] In addition, the spatial distribution characteristics of the control points in the current initial control point set are adjusted with the goal of optimizing the target value as the guide, including: if the target value still does not reach the preset expected value after adjusting the coordinate position of the target control point in the perception domain, then at least one new control point is added to the current initial control point set in the perception domain.

[0043] In this embodiment, the specific end-to-end optimization method flow is as follows: 1. Initialization: The number of control points follows the number determined by the upstream algorithm (such as the number of segments for spline fitting); the initial position is taken from the node position given by the upstream algorithm, or set according to conventional rules (equidistant, histogram quantile, sampling from unsupervised optimization curves), and the first and last points are usually fixed at the two ends of the brightness range.

[0044] 2. Calculate the current target value D: Run the content-oriented end-to-end feedback loop through the current control point to obtain the target value D.

[0045] 3. Adjusting Optimization Variables: The optimization variables are the control point locations, and can optionally be extended to endpoint slopes or shape parameters of the base curve; location and slope / shape parameters can be optimized alternately or combined into a unified set of variables for end-to-end optimization. Taking control point optimization as an example, the available optimization strategies are: Greedy point-by-point search: Fix other points in turn, only move the i-th point, and use a one-dimensional search (golden section, etc.) between its left and right adjacent points to find the position that makes D optimal, and process all intermediate points in turn; Gradient descent (taking minimizing D as an example, where D is the error / loss that is as small as possible): Take the partial derivative of D with respect to the control point positions p_i. D / p_i is obtained using numerical difference or automatic differentiation, and updated along the negative gradient: p_i ← p_i - η D / p_i (η is the step size): When the entire link is differentiable (e.g., J is differentiable without supervision), end-to-end reverse transmission is possible; Global optimization: When the number of control points is small, use particle swarm optimization / genetic algorithm to search within the feasible region to avoid local optima.

[0046] Regarding the optimization of slope / shape parameters, the conventional approach requires the slope to precisely match the reference curve in the PQ domain. However, end-to-end optimization allows the slope to deviate slightly from the precise value in the PQ domain to compensate for shape differences within segments, further optimizing the content-oriented target D. To maintain the reasonableness of the curve, constraints can be imposed on the slope, including monotonicity constraints and smoothing constraints that limit the slope changes between adjacent segments.

[0047] 5. Convergence judgment: Terminate the iteration when the target value is better than the threshold, or the improvement of the adjacent iteration is less than the set value, or the maximum number of iterations is reached; otherwise, return to step 2 and continue iterating.

[0048] 6. Output: Write out the optimized control points (and optional slopes) as the final metadata.

[0049] In addition, the process of the lightweight error constraint method is as follows: 1. Initial Point Selection: Initial control points can be taken from nodes provided by the upstream algorithm, or set according to conventional rules. Since subsequent steps will automatically add points based on errors, the initial number does not need to be precise. Consistent with the number of points used in end-to-end optimization and prevention, only the optimization locations differ; this scheme adds the number of points as needed based on the upstream initial points.

[0050] 2. Calculate segmented targets: Run the content-oriented end-to-end feedback loop through the current control point. In addition to the overall target, calculate the content-oriented target value for each segment [p_i, p_(i+1)] of the tone mapping curve. The maximum intra-segment error can be taken as e_i = max_{x∈[p_i,p_(i+1)]} |Iout(x) - Iref(x)| or the intra-segment root mean square error. In unsupervised cases, the value within the segment of criterion J is used instead.

[0051] 3. Determine segments exceeding the limit: Set a threshold th. Any segment where e_i > th is marked as "needs processing".

[0052] 4. Slope / shape parameter fine-tuning (optional): For the segment to be processed, first fine-tune the slope of its endpoints or the shape parameters of the basic curve under constraints (slope / shape parameter optimization of the same end-to-end optimization method, satisfying monotonicity and smoothness constraints), and try to reduce the error within the segment without increasing the number of control points; if the segment meets the standard after fine-tuning, no additional points are needed.

[0053] 5. Automatic Point Filling: Perform point filling for segments that still exceed the limit after fine-tuning. Possible strategies include: Strategy A (insertion point at the worst point of the target): Insert a new control point at the position where the target takes the worst value within this segment, which directly reduces the error of this segment; Strategy B (interpolation at the midpoint of the segment): Insert the point directly at the midpoint of the segment; this is the simplest strategy. Strategy C (Adaptive Segmentation): Determine the segmentation granularity (the number of points) based on the severity of the exceedance (e.g., e_i > 2). If th is trisected, insert two points; if th < e_i ≤ 2 (Then the two parts are divided into two equal parts and a single point is inserted).

[0054] 6. Iterative verification: After adding points, return to step 2 and recalculate, repeating the process.

[0055] 7. Termination conditions: Stop when all segment targets are met, or the number of points reaches the upper limit x, or when adding more points cannot significantly improve the situation.

[0056] 8. Output: Write down the final set of control points.

[0057] Step S14: Determine whether the updated control point set meets the preset iteration stop condition. If the updated control point set does not meet the preset iteration stop condition, then determine the updated control point set as the new current initial control point set, and jump to the step of converting the current initial control point set into the linear domain control point set corresponding to the linear domain.

[0058] As can be seen from the aforementioned steps in this embodiment, after completing the process of updating the control points once, it is necessary to determine whether the updated control point set meets the preset iteration stopping condition. When this embodiment adopts an end-to-end optimization method, the preset iteration stopping condition can be any one of the following: the target value is better than the threshold, the improvement amount of adjacent iterations is less than the set value, or the maximum number of iterations is reached. When this embodiment adopts a lightweight error constraint optimization method, the preset iteration stopping condition can be one of the following: all segment targets are met, the number of points reaches the upper limit x, or adding more points cannot significantly improve the effect.

[0059] Understandably, if the updated control point set does not meet the preset iteration stop condition, a new round of iteration optimization process is required until the updated control point set meets the preset iteration stop condition.

[0060] Step S15: If the updated control point set satisfies the preset iteration stop condition, then the updated control point set is determined as the final optimized control point set.

[0061] In this embodiment, if the updated control point set meets the preset iteration stop condition, it proves that after several rounds of iteration optimization, the updated control points have reached the optimization target. Then, the updated control point set can be determined as the final optimized control point set.

[0062] Therefore, this application solves the problem of segment shape deviation caused by the non-closed form of piecewise curve functions due to the non-linearity of the transfer function between the perceptual domain and the linear domain by setting both curve construction and mapping execution in the linear domain; by using the target value of the mapped content relative to the preset reference standard as the optimization guide, the optimization target directly reflects the final output image quality rather than the fitting error of the intermediate domain, thus solving the problem of the traditional method ignoring the execution domain error and causing the optimization target to be out of touch with the actual display image quality; by iteratively adjusting the spatial distribution of the control points in the perceptual domain until the target value is met, the error of each brightness segment is constrained by the unified output target value, thus solving the problem of error distribution imbalance caused by the extreme unevenness of the linear domain segment width.

[0063] See Figure 2 As shown, an embodiment of the present invention discloses a tone mapping curve control point optimization device, comprising: The control point set conversion module 11 is used to obtain the current initial control point set in the perception domain and convert the current initial control point set into a linear domain control point set corresponding to the linear domain; wherein, the linear domain is a signal space in which the brightness value of the video frame pixel is linearly proportional to the physical light intensity, and the perception domain is a nonlinear signal space obtained by applying a perception quantization transfer function to the brightness value of the linear domain. The mapping curve construction module 12 is used to construct a corresponding tone mapping curve in the linear domain based on the linear domain control point set, and to perform mapping processing on the input content using the tone mapping curve to obtain the corresponding mapped content; the input content is the pixel brightness signal of the video frame in the linear domain. The target value determination module 13 is used to determine the target value corresponding to the mapped content, so as to optimize the spatial distribution characteristics of the control points in the current initial control point set as a guide to obtain the updated control point set; the target value represents the quality deviation of the mapped content relative to the preset reference standard. The step jump module 14 is used to determine whether the updated control point set meets the preset iteration stop condition. If the updated control point set does not meet the preset iteration stop condition, the updated control point set is determined as the new current initial control point set, and the process jumps to the step of converting the current initial control point set into the linear domain control point set corresponding to the linear domain. The control point set determination module 15 is used to determine the updated control point set as the final optimized control point set if the updated control point set meets the preset iteration stop condition.

[0064] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0065] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the tone mapping curve control point optimization method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0066] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0067] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0068] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the tone mapping curve control point optimization method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0069] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned tone mapping curve control point optimization method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0070] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0071] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0072] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0073] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0074] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for optimizing control points of a tone mapping curve, characterized in that, include: The current initial control point set under the perceptual domain is obtained, and the current initial control point set is converted into the linear domain control point set corresponding to the linear domain; wherein, the linear domain is the signal space in which the brightness value of the video frame pixel is linearly proportional to the physical light intensity, and the perceptual domain is the nonlinear signal space obtained by applying a perceptual quantization transfer function to the brightness value of the linear domain. Based on the linear domain control point set, a corresponding tone mapping curve is constructed in the linear domain, and the tone mapping curve is used to perform mapping processing on the input content to obtain the corresponding mapped content; the input content is the pixel brightness signal of the video frame in the linear domain. A target value corresponding to the mapped content is determined, and the spatial distribution characteristics of the control points in the current initial control point set are adjusted based on the optimization of the target value to obtain an updated control point set; the target value represents the quality deviation of the mapped content relative to a preset reference standard. Determine whether the updated control point set meets the preset iteration stop condition. If the updated control point set does not meet the preset iteration stop condition, then determine the updated control point set as the new current initial control point set, and jump to the step of converting the current initial control point set into the linear domain control point set corresponding to the linear domain. If the updated control point set satisfies the preset iteration stop condition, then the updated control point set is determined as the final optimized control point set.

2. The method for optimizing control points of tone mapping curves according to claim 1, characterized in that, Determining the target value corresponding to the mapped content includes: The target value corresponding to the mapped content is determined using a supervised evaluation method or an unsupervised evaluation method; wherein, the supervised evaluation method is an evaluation method that compares the mapped content with the reference image frame pixel by pixel when a reference image frame exists, and the unsupervised evaluation method is an evaluation method that inputs the mapped content into a preset perceptual quality model to obtain the target value when no reference image frame exists.

3. The method for optimizing control points of tone mapping curves according to claim 2, characterized in that, When the supervised evaluation method is used, the target value is the image domain error between the mapped content and the reference image frame; the image domain error characterizes the pixel-by-pixel difference between image frames. Accordingly, adjusting the spatial distribution characteristics of the control points in the current initial control point set to optimize the target value, thereby obtaining the updated control point set, includes: With the goal of reducing the image domain error, the positions of the control points in the current initial control point set are adjusted to obtain the updated control point set.

4. The method for optimizing control points of tone mapping curves according to claim 2, characterized in that, When the unsupervised evaluation method is used, the target value is the perceived quality score output by the preset perceived quality model; Accordingly, adjusting the spatial distribution characteristics of the control points in the current initial control point set to optimize the target value, thereby obtaining the updated control point set, includes: With the goal of increasing the perceived quality score, the positions of the control points in the current initial control point set are adjusted to obtain the updated control point set.

5. The method for optimizing control points of tone mapping curves according to claim 1, characterized in that, The step of adjusting the spatial distribution characteristics of the control points in the current initial control point set with the goal of optimizing the target value includes: Keeping the total number of control points in the current initial control point set unchanged, the position coordinates of at least one target control point in the perception domain are adjusted, and the endpoint slope or shape parameter of the base curve associated with the target control point is adjusted; wherein, the endpoint slope is used to define the tangent direction of the tone mapping curve at the control point, and the shape parameter of the base curve is used to define the curvature of the curve segment between adjacent control points.

6. The method for optimizing control points of tone mapping curves according to claim 5, characterized in that, The step of adjusting the spatial distribution characteristics of the control points in the current initial control point set with the goal of optimizing the target value includes: If the target value still does not reach the preset expected value after adjusting the coordinate position of the target control point in the perception domain, then at least one new control point is added to the current initial control point set in the perception domain.

7. The method for optimizing control points of tone mapping curves according to claim 5, characterized in that, The adjustment of the endpoint slope or shape parameters of the base curve associated with the target control point includes: The endpoint slope or the shape parameter of the base curve is adjusted under preset constraints; wherein, the constraints include monotonicity constraints and smoothness constraints; the monotonicity constraints are used to maintain the monotonically increasing or monotonically decreasing characteristics of the tone mapping curve on each curve segment, and the smoothness constraints are used to limit the slope change of adjacent curve segments at the common control point to not exceed a preset threshold.

8. A tone mapping curve control point optimization device, characterized in that, include: The control point set conversion module is used to obtain the current initial control point set in the perceptual domain and convert the current initial control point set into a linear domain control point set corresponding to the linear domain; wherein, the linear domain is a signal space in which the brightness value of the video frame pixel is linearly proportional to the physical light intensity, and the perceptual domain is a nonlinear signal space obtained by applying a perceptual quantization transfer function to the brightness value of the linear domain. The mapping curve construction module is used to construct a corresponding tone mapping curve in the linear domain based on the linear domain control point set, and to perform mapping processing on the input content using the tone mapping curve to obtain the corresponding mapped content; the input content is the pixel brightness signal of the video frame in the linear domain. The target value determination module is used to determine the target value corresponding to the mapped content, so as to optimize the spatial distribution characteristics of the control points in the current initial control point set as a guide to obtain the updated control point set; the target value represents the quality deviation of the mapped content relative to the preset reference standard; The step jump module is used to determine whether the updated control point set meets the preset iteration stop condition. If the updated control point set does not meet the preset iteration stop condition, the updated control point set is determined as the new current initial control point set, and the process jumps to the step of converting the current initial control point set into the linear domain control point set corresponding to the linear domain. The control point set determination module is used to determine the updated control point set as the final optimized control point set if the updated control point set meets the preset iteration stop condition.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the tone mapping curve control point optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the tone mapping curve control point optimization method as described in any one of claims 1 to 7.