Display device color self-adaptive calibration method and system based on image real-time feedback

Abstract

The application discloses a display device color self-adaptive calibration method and system based on image real-time feedback, belongs to the field of display device color calibration and computer vision technology, and relates to the technical field of display device color calibration and computer vision.The method comprises the following steps: controlling a display device to output a preset color sampling sequence; collecting images, establishing a time sequence and space correspondence relationship, and extracting feedback measurement values; binding the display input values and the feedback measurement values into sample pairs, constructing a hierarchical deviation model comprising a global model and a local refinement model, and fusing the hierarchical deviation model; generating compensation parameters; acquiring running parameters and / or environmental parameters in real time, updating the compensation parameters when the changes of the running parameters and / or the environmental parameters meet a triggering condition; and loading the updated compensation parameters to a hardware processing unit to perform mapping processing on pixel data.The application realizes closed-loop calibration, supports dynamic updating of parameters, and improves end-to-end color consistency.

Description

Technical Field

[0001] This invention belongs to the field of display device color calibration and computer vision technology, specifically relating to a display device color adaptive calibration method and system based on real-time image feedback. Background Technology

[0002] In virtual production, real-time shooting, and high-precision display applications, display devices (such as LED displays) not only display images but also directly participate in the imaging process. In practical applications, the luminous characteristics of the display device, the driving characteristics of the display controller, the spectral response characteristics of the image acquisition terminal, and changes in shooting parameters can all contribute to color discrepancies between the displayed image and the captured image.

[0003] Existing solutions typically perform static calibration on the display device itself. For example, they use colorimeters to establish a calibration matrix or lookup table for the display device (see: ICC Color Management Standard, ICC.1:2010; Reinhard et al., "Color Transfer between Images", IEEE CGA, 2001), to ensure that the display device output meets a preset color gamut standard. This type of solution can improve the consistency of the output of individual display devices, but it is difficult to directly guarantee that the image results captured by cameras or other image acquisition terminals will still be consistent with the target color.

[0004] Existing technologies have at least the following shortcomings: 1. Most existing calibrations are open-loop calibrations, which do not include the image acquisition terminal in a unified calibration link and cannot effectively compensate for the photoelectric characteristic mismatch between the display device and the image acquisition terminal; 2. Existing calibrations usually rely on static parameters or preset lookup tables. When the camera's white balance, exposure, ISO, gain, or ambient light changes, the original compensation relationship is prone to failure, requiring frequent manual recalibration, resulting in low on-site efficiency; 3. In low grayscale areas (8-bit driving value 0-32), skin-sensitive areas, or high dynamic brightness transition areas, existing uniform sampling methods cannot balance sampling efficiency and fitting accuracy, leading to color cast in dark areas, grayscale drift, or loss of detail; 4. Manual recalibration is time-consuming and cannot meet the needs of multi-device linkage and real-time shooting scenarios for rapid updates of compensation parameters, resulting in high on-site downtime costs.

[0005] Therefore, a closed-loop calibration scheme is needed that can link display output, image feedback, and parameter changes to improve the consistency between the display results and the final imaging results without frequent manual recalibration. Summary of the Invention

[0006] This invention provides a color adaptive calibration method and system for display devices based on real-time image feedback, which solves the problem that existing display device calibration schemes are open-loop static calibrations, which cannot incorporate image acquisition terminals into a unified calibration link, and have insufficient adaptive capability to changes in shooting parameters (ISO, white balance, color temperature, exposure, etc.), making it difficult to maintain end-to-end color consistency in real-time shooting scenarios.

[0007] This invention achieves end-to-end consistency calibration of the display link and the image acquisition link by constructing a closed-loop processing mechanism of "display sampling - temporal binding - region extraction - sample construction - unified representation - hierarchical modeling - parameter triggering - hardware execution".

[0008] The core of this invention lies in: establishing a temporal synchronization relationship and a spatial mapping relationship between the sampled image block output sequence and the image acquisition image to achieve a precise correspondence between the display output and the imaging feedback; on this basis, constructing a multi-dimensional sample set including sampling input, feedback measurement value, device information and shooting parameters, and establishing a joint response model of the display link and the acquisition link in a unified computing space.

[0009] Furthermore, through a non-uniform sampling mechanism (encrypted sampling of low grayscale areas and preset sensitive color gamut areas), a layered modeling mechanism (coupling of global model and local refinement model), and a parameter association mechanism (establishing a model set according to the shooting parameter range and performing model selection / interpolation), the compensation model can be selected or dynamically updated as shooting parameters and environment change, thereby achieving adaptive color calibration for dynamic shooting scenarios.

[0010] A color adaptive calibration method for display devices based on real-time image feedback includes: controlling the display device to output sampled patches according to a preset color sampling sequence; acquiring the sampled patches output by the display device through an image acquisition terminal to obtain feedback image data, and establishing a temporal and spatial correspondence between the color sampling sequence and the feedback image data to extract feedback measurement values ​​corresponding to each sampled patch; binding the display input values ​​of the sampled patches with the feedback measurement values ​​as sample pairs, and constructing a hierarchical deviation model containing a global model and a local refinement model in a unified computational space, wherein the global model is used to characterize white point offset, Channel gain imbalance and overall brightness response offset are addressed by a local refinement model used to characterize nonlinear deviations in low grayscale regions, skin tones, neutral gray regions, or white point neighborhoods. The global model is then fused with the local refinement model. Compensation parameters for correcting the display device output are generated based on the fused layered deviation model. The image acquisition terminal's shooting parameters are acquired in real-time. When a change in the shooting parameters is detected to meet a preset trigger condition, the compensation parameters are updated. The updated compensation parameters are loaded into the hardware processing unit in the display controller or display link, and pixel data is mapped in the video output path.

[0011] In some examples, the color sampling sequence includes global base sampling points and encrypted sampling of low grayscale regions and / or preset sensitive color gamut regions; the low grayscale regions are the range of 0 to 32 under 8-bit driving values, and the preset sensitive color gamut regions include at least one of skin color regions, neutral gray regions, or regions near white points.

[0012] In some examples, establishing the temporal and spatial correspondences includes: using the output temporal identifier, frame number, synchronization trigger signal, or timestamp of the sampling patch to determine the sampling patch number corresponding to the current image frame; after determining the sampling patch number, determining the region of interest in the image frame corresponding to the spatial position of the sampling patch through a preset display layout mapping relationship, positioning mark recognition, geometric correction relationship, or perspective transformation relationship, and performing statistical processing on the pixels within the region of interest to obtain the feedback measurement value.

[0013] In some examples, the fusion of the global model and the local refinement model can be achieved by: local replacement, using the local model output as a correction term for the global model output, or using a weighting function to smoothly transition between the global output and the local output.

[0014] In some examples, the shooting parameters include at least one of white balance parameters, color temperature parameters, ISO parameters, exposure parameters, gain parameters, shutter parameters, lens filter parameters, ambient light brightness parameters, and ambient light color temperature parameters; the preset triggering conditions include single threshold strategies, combined threshold strategies, or priority triggering strategies, and introduce hysteresis intervals, minimum update time intervals, or stable frame count conditions.

[0015] In some examples, the compensation parameters are updated in the following ways: without re-sampling the entire dataset, the updated compensation parameters are obtained by selecting a model that matches the current shooting parameter value from multiple deviation models pre-established for different shooting parameter values, interpolating the two models whose shooting parameter values ​​are closest to the current shooting parameter value, or performing a partial update on the compensation parameters.

[0016] In some examples, the compensation parameters are in the form of a three-dimensional lookup table, a global color transformation matrix, a channel-specific nonlinear response curve, or a set of models indexed by parameters; when the compensation parameters are loaded into the hardware processing unit, they are written to the display controller using double buffer switching, frame boundary synchronous switching, or buffer atomic replacement methods.

[0017] In some examples, when multiple display devices participate in shooting simultaneously, a unified master control node is used to perform synchronous sampling, synchronous modeling, and synchronous compensation updates on multiple display devices. For multiple display devices of the same model, a common basic model is first established, and then personalized fine-tuning is performed based on the local deviations of each display device. For multiple display devices of different models, device-level models are established separately, and the output is uniformly constrained under the same reference color space to improve the imaging consistency between multiple devices.

[0018] A color adaptive calibration system for display devices based on real-time image feedback includes: a standard feature pattern generation module configured to control the display device to output sampled patches according to a preset color sampling sequence; an image feedback acquisition module configured to acquire the sampled patches output by the display device through an image acquisition terminal, acquire feedback image data, and establish a temporal and spatial correspondence between the color sampling sequence and the feedback image data to extract feedback measurement values ​​corresponding to each sampled patch; and an adaptive deviation analysis engine configured to bind the display input values ​​of the sampled patches and the feedback measurement values ​​as sample pairs, and construct a hierarchical deviation model containing a global model and a local refinement model in a unified computational space, wherein the global model is used to characterize white point deviation. The local refinement model is used to characterize nonlinear deviations in low grayscale regions, skin color regions, neutral gray regions, or white point neighborhoods, and the global model is fused with the local refinement model. A dynamic correction and compensation module is configured to generate compensation parameters for correcting the output of the display device based on the fused layered deviation model. A parameter monitoring and triggering module is configured to acquire the shooting parameters of the image acquisition terminal in real time, and update the compensation parameters when a change in the shooting parameters meets a preset trigger condition. A hardware loading module is configured to load the updated compensation parameters into the hardware processing unit in the display controller or display link, and perform mapping processing on the pixel data in the video output path.

[0019] In some examples, the adaptive deviation analysis engine includes: a sample binding unit for establishing temporal and spatial correspondences between the display input values ​​of sampled patches and the corresponding feedback measurements; a unified representation unit for mapping the display input values ​​and feedback measurements to the same color space; a hierarchical modeling unit for constructing a global model and a local refinement model; a model fusion unit for fusing the global model and the local refinement model; and a parameter association unit for establishing a model set based on discrete shooting parameter values, and selecting a matching model from the model set based on the current shooting parameter value, or interpolating between the two models whose shooting parameter values ​​are closest to the current shooting parameter value.

[0020] The present invention achieves the following significant advancements: (1) It realizes closed-loop calibration of the display link and the acquisition link, improving end-to-end imaging consistency; (2) It supports parameter-triggered updates, improving adaptability to changes in shooting parameters and environmental changes; (3) It improves the fitting accuracy of low grayscale and sensitive color gamut areas through non-uniform sampling and layered modeling mechanisms, reducing dark color cast, grayscale drift and detail distortion; (4) It supports hardware-level execution compensation, meeting the low latency requirements of real-time display applications; (5) It is applicable to single display devices and can be extended to synchronous calibration scenarios of multiple display devices; (6) Under parameter change conditions, it can achieve rapid response through model selection or local updates, reducing recalibration costs. Attached Figure Description

[0021] Figure 1 This is a flowchart of a display device color adaptive calibration method according to an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram of a closed-loop feedback mechanism for adaptive color calibration of a display device according to an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the sampling point distribution according to an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of the output values ​​of global model mapping and local refinement model mapping according to an embodiment of the present invention.

[0025] Figure 5 This is a block diagram of a display device color adaptive calibration system according to an embodiment of the present invention. Detailed Implementation

[0026] The term "display device" can refer to LED displays, Mini LED displays, OLED displays, projection displays, or other devices capable of receiving image signals and outputting displayed images. The term "image acquisition terminal" can refer to cameras, industrial cameras, surveillance cameras, digital cameras, or other imaging devices capable of acquiring and displaying images.

[0027] Example 1: A color adaptive calibration method for display devices based on real-time image feedback. The method includes: controlling a display device to output sampled patches according to a preset color sampling sequence; acquiring the sampled patches output by the display device through an image acquisition terminal, obtaining feedback image data, and establishing a temporal and spatial correspondence between the color sampling sequence and the feedback image data to extract feedback measurement values ​​corresponding to each sampled patch; binding the display input values ​​of the sampled patches with the feedback measurement values ​​as sample pairs, and constructing a hierarchical deviation model containing a global model and a local refinement model in a unified computational space. The global model is used to characterize white point offset, channel gain imbalance, and overall brightness response offset. A local refinement model is used to characterize nonlinear deviations in low grayscale regions, skin color regions, neutral gray regions, or white point neighborhoods, and the global model is fused with the local refinement model; compensation parameters for correcting the output of the display device are generated based on the fused layered deviation model; the operating parameters and / or environmental parameters of the image acquisition terminal are acquired in real time, and when changes in the operating parameters and / or environmental parameters meet preset trigger conditions, the compensation parameters are updated; the updated compensation parameters are loaded into the hardware processing unit in the display controller or display link, and pixel data is mapped in the video output path. The following section combines... Figure 1 , Figure 2 Please explain this method in detail.

[0028] Step 1: Establish a task to generate and output standard feature patterns.

[0029] Determine the set of sampling points based on the target color gamut and target application scenario. For example... Figure 3 As shown, for basic sampling, multiple discrete points can be selected in the RGB three-dimensional space with a preset step size; for low grayscale areas, a smaller step size than that of basic sampling can be used; for skin color areas, neutral gray areas, or areas near white points, additional sampling can be performed based on empirical ranges or historical statistical ranges.

[0030] The control display device outputs a preset color sampling sequence (sampling blocks), the color sampling sequence including at least global basic sampling points, and performing encrypted sampling on low grayscale areas and / or preset sensitive color gamut areas. The low grayscale areas are intervals with small brightness driving values; the sensitive color gamut areas include at least one of skin tone areas, neutral gray areas, and areas near white points.

[0031] In one alternative implementation, marker blocks or coded blocks for auxiliary positioning, such as border markers, corner markers, and QR code / barcode style markers, may also be inserted into the sampling sequence to quickly and stably determine the corresponding position of each sampling block in the image frame.

[0032] For example, the basic sampling uses a 9×9×9 grid distribution in RGB 3D space (729 sampling points in total), and the spacing (step size) between adjacent sampling points is 32 gray levels (8 bits, full scale 255).

[0033] The low grayscale region can be defined as the range of 8-bit driving values ​​from 0 to 32 (approximately 12.5% ​​of the full scale). Within this range, the sampling step size is shortened from the basic step size of 32 to 2 to 4 gray levels, thereby increasing the sampling density of the low grayscale region by approximately 8 to 16 times, so as to accurately capture the nonlinear response characteristics within this range.

[0034] For sensitive color gamut areas, the following areas are also sampled more encrypted: skin tones are sampled in the CIE Lab (International Commission on Illumination 1976). The region in the color space is L∈[40,70], a∈[5,25], b∈[10,30]; the neutral gray region is the area near the axis with saturation S<0.05 in the HSV color space; the region near the white point is the color difference ΔE from the target white point at Lab coordinates. 00 A spherical neighborhood of <10. Approximately 50-100 additional encrypted sampling points are added within the aforementioned region.

[0035] The control display device outputs each sampled image block in a predetermined order. Each sampled image block is displayed as a full-screen or fixed-size (e.g., 60% of the central area of ​​the screen) solid color block for a duration of not less than 1 / Fd (Fd is the display refresh rate) to ensure that the image acquisition terminal can capture at least one frame of stable image.

[0036] In the sampling sequence, a positioning marker patch (such as a checkerboard pattern with four corner points) is inserted after every N sampling patches to assist in locating the spatial position of each sampling patch and correcting perspective distortion at the acquisition end. N can be 10~20, such as inserting a positioning marker patch after every 15 sampling patches to correct the image spatial mapping in a timely manner without significantly increasing the sampling time.

[0037] Step 2: Obtain image feedback data.

[0038] The image acquisition terminal acquires sampled image blocks (standard feature mode) output by the display device and converts the acquired physical light emission image into data for calculation. This step includes: acquiring feedback image data through the image acquisition terminal, and then performing time synchronization, image region determination, feedback measurement value extraction, and stability enhancement processing.

[0039] 2.1 Timing synchronization.

[0040] The display device displays sampled image blocks one by one according to a predetermined output sequence, while the image acquisition terminal synchronously acquires image frames. The system can use the output timing identifier, frame number, synchronization trigger signal, or timestamp of the sampled image blocks to determine the target sampled image block number corresponding to the current image frame.

[0041] For example, the system establishes the timing correspondence between acquisition frames and sampled blocks in the following way. Let the display device refresh rate F... d =60Hz, image acquisition terminal frame rate F c =24fps (inconsistent frame rate scenarios). The system records the timestamp T of the start of display for each sampled tile. display (i); The acquisition end records a timestamp T for each frame of image. capture (j), where j is the acquisition frame number (j=1,2,…); will satisfy |T capture (j) T display (i)| < 1 / (2·F d The acquired frame is determined to be the corresponding frame of the i-th sampled image block; if an acquired frame cannot match any sampled image block, it is discarded. On devices with a hardware trigger interface, the display controller can send a TTL / RS422 trigger pulse to the image acquisition terminal at the start of each sampled image block display to directly achieve frame-level hardware synchronization.

[0042] 2.2 Image region determination.

[0043] After determining the sampling patch number, the region of interest (ROI) corresponding to the spatial location of the sampling patch is extracted in the corresponding image frame. The ROI can be determined by at least one of the following methods: preset display layout mapping relationship, positioning mark recognition, geometric correction relationship or perspective transformation relationship; and the ROI boundary can be further corrected by combining feature point detection (corner / edge) or template matching to reduce errors caused by shooting angle, lens distortion and screen perspective.

[0044] Specifically, the perspective transformation relationship is achieved through the following calibration process: the display device outputs a checkerboard calibration pattern containing known corner coordinates (pixel coordinates are known); the image acquisition terminal acquires the calibration image; the checkerboard corner coordinates in the image are detected using image processing libraries such as OpenCV; the findHomography algorithm is used to calculate the 3×3 homography matrix H from the display coordinate system to the image coordinate system; and H is stored as a system parameter. During the calibration operation phase, the four corner coordinates of each sampled image block in the display coordinate system are mapped to the image coordinate system using H, and the region with a 5% indentation after mapping is taken as the region of interest (ROI) to avoid interference from edge transition areas.

[0045] 2.3 Feedback measurement value extraction.

[0046] Feedback image data is acquired through an image acquisition terminal, and statistical processing is performed on the pixels within the ROI corresponding to each sampled image patch to obtain feedback measurement values. For example, after removing the top and bottom 5% of abnormal pixels in brightness value from all pixels within the ROI, the mean values ​​of the R, G, and B channels are calculated (R... mean G mean B mean The standard deviation of each channel is recorded as a confidence level assessment criterion, and the feedback measurement value is used to characterize the actual imaging result of the sampled patch. This feedback measurement value may include, but is not limited to, the mean value of the region's pixel channels (R0). mean / G mean / B mean The values ​​include: median, representative value after outlier removal, linearized color vector, luminance value L, chromaticity information C, and region variance Var.

[0047] 2.4 Preprocessing and normalization.

[0048] To improve the comparability of feedback image data under different shooting conditions, one or a combination of the following preprocessing operations can be performed on the acquired images: black level correction (subtracting the mean of dark frames), noise reduction, flicker removal, inverse gamma transform (linearizing according to the camera's nominal gamma value), white balance normalization (normalizing the R / G / B values ​​of the neutral gray reference point in the acquired image to an equal scale), exposure normalization, and color space conversion.

[0049] 2.5 Enhanced stability.

[0050] For the same sampled patch, multiple frames of images can be continuously acquired, and mean processing, median processing, weighted fusion, or confidence filtering can be performed on the measurement results of the corresponding ROI to obtain stable feedback measurement values. A confidence score (Conf) can be calculated for each sampled patch, based on factors such as noise level, brightness stability, and multi-frame consistency. Low-confidence samples can be removed or downweighted.

[0051] For example, K ≥ 3 frames are continuously acquired for the same sampling patch (K is the number of frames acquired for each sampling patch), and a weighted average is applied to the ROI measurement results of each frame (the weight is proportional to the confidence score of each frame). The confidence score (Conf) is a comprehensive score based on the following three factors: multi-frame brightness standard deviation. (The smaller the standard deviation, the higher the confidence level), the pixel variance Var within the ROI region (the smaller the variance, the more uniform the data), and the inter-frame consistency of the R / G / B mean values ​​across multiple frames. When Conf < 0.6, the sample is marked as low confidence and its weight is reduced during modeling; when Conf < 0.3, the sample is directly removed and the acquisition of the corresponding sampling patch is retried.

[0052] Step 3: Establish the deviation model.

[0053] Based on the correspondence between sampled input values ​​and feedback measurements, a joint response model from the display input domain to the imaging feedback domain is established, and a compensation mapping relationship for reverse correction of the display output is generated. This step includes sample binding, unified representation, hierarchical modeling, model fusion, and parameter correlation, thereby avoiding the conventional path of compensation based solely on simple difference calculations.

[0054] 3.1 Sample construction and binding.

[0055] For each sampled image patch, its output time sequence identifier, spatial location identifier, and sampling number are used to bind its displayed input value with the ROI feedback measurement value obtained in step 2, thereby forming a sample pair "sampled input value - feedback measurement value". In a preferred embodiment, a sample can be represented as: S = {Input} RGB Measured RGB , t, Region ID Device ID Camera Params}, where t is the timestamp or time sequence marker for the sampling patch corresponding to the sample, Input RGB For sampled input values, Measured RGB To provide feedback on the measured values, Region ID Device is used as a spatial identifier for sampling blocks. ID To display device identification, Camera Params For camera parameter set, Camera Params This includes parameters such as ISO, white balance, exposure, color temperature, gain, shutter speed, and lens filter, which are used for subsequent parameter correlation modeling or model selection.

[0056] 3.2 Unified representation.

[0057] The sampled input values ​​and feedback measurements are mapped to a unified computational space (such as linear RGB space, XYZ space, Lab space or its equivalent transformation space) in order to uniformly represent color shift, brightness response and channel coupling relationship.

[0058] For example, Input RGB With Measured RGB The model is converted from gamma-encoded space to linear RGB space to eliminate the influence of gamma encoding on linear color response analysis; if necessary, it can be further converted to XYZ or CIE Lab space for modeling to characterize the deviation in a perceived uniform color space.

[0059] 3.3 Layered modeling.

[0060] A bias model is constructed based on a sample set using a hierarchical modeling approach, including a global model and a local refinement model. The global model is used to characterize white point shift, channel gain imbalance, and overall brightness response shift; the local refinement model is used to characterize nonlinear biases in low grayscale regions, skin color regions, neutral gray regions, and the white point neighborhood.

[0061] In one implementation, a coarse-grained global mapping model covering the entire color gamut (e.g., a 3×3 color transformation matrix or a coarse-grained 3D LUT) is first established based on the basic sampling points; then, the local areas are refined and corrected using the encrypted sampling points of the low grayscale area and the sensitive color gamut area (e.g., local nonlinear curves, local interpolation or weighted regression).

[0062] Global model establishment. Linearization of all basic sampling points (P=729) Input. RGB The column vectors form a 3×P matrix A, corresponding to Measured RGB The column vectors form a 3×P matrix B (P=729). The 3×3 color transformation matrix M is solved using the least squares method, resulting in a residual ||B||. M·A‖ 2 Minimum: M = B·A ·(A·A ) - ¹, where the superscript T denotes the transpose matrix. In scenarios with nonlinear characteristics, A can be extended to a polynomial eigenvector containing quadratic terms (e.g., [R, G, B, R², G², B², RG, RB, GB, 1]). To establish a multinomial regression model.

[0063] Local refinement model establishment. For the low grayscale region (driving value 0-32, with approximately 16×3=48 points / channel for encrypted sampling) and the encrypted sampling points in the sensitive color gamut region, a local color deviation surface is constructed using radial basis function (RBF) interpolation: f local (x) = Σ i w i ·φ(‖x c i ‖), where f local (x) Local color deviation surface function; i is the index of the encrypted sampling point; φ is the Gaussian kernel function (σ=10 gray levels); x is the input color vector; c i Input for the i-th encrypted sampling point RGB Coordinates; w i The corresponding weights are determined by solving a system of linear equations.

[0064] 3.4 Model Fusion.

[0065] The fusion of local refinement models and global models can be achieved by: locally replacing the corresponding regions in the global model; using the output of the local model as a correction term for the output of the global model; or using a weighting function to smoothly transition between the global output and the local output, thereby forming a composite mapping relationship that balances global consistency and local accuracy. Figure 4 The output values ​​of the global model mapping and the local refined model mapping are displayed.

[0066] For example, the global model output M·Input RGB Using the RBF local refinement model as a baseline, the output is superimposed as a correction value. The final compensated output color value (the fused output color vector) is calculated as follows: Output = M·Input RGB + w local (x)·f local (x), where x is the input color vector, f local (x) is the local color deviation surface function, w local (x) is a smooth transition weight function (linearly transitioning from 1 to 0 near the boundary of the local region) to ensure a smooth fusion of global consistency and local accuracy.

[0067] 3.5 Parameter Correlation and Model Selection.

[0068] In another alternative implementation, the deviation model can establish separate model sets for different image acquisition terminals, different lens filter combinations, or different shooting parameter ranges. When the shooting parameters change, a model matching the current shooting parameters can be selected from the model set, or the two models whose shooting parameter values ​​are closest to the current shooting parameter values ​​can be interpolated to obtain updated compensation parameters; thereby improving the model's adaptability under different parameter conditions.

[0069] Shooting parameters refer to a set of measurable and adjustable parameters related to the working status of an image acquisition terminal (such as a camera), including operating parameters and environmental parameters, such as at least one of white balance parameters, color temperature parameters, ISO parameters, exposure parameters, gain parameters, shutter parameters, lens filter parameters, ambient light brightness parameters, and ambient light color temperature parameters.

[0070] For example, the model set can be built according to the following two dimensions: ISO dimension nodes are 100 / 400 / 800 / 1600 / 3200 (5 levels in total); color temperature dimension nodes are 3200K / 4500K / 5600K / 6500K (4 levels in total), forming 5×4=20 model nodes. Each node stores the global model parameter M and the local refinement model parameter f under the corresponding conditions. localWhen the runtime parameters (ISO=1200, color temperature=5000K) fall between nodes, linear interpolation is performed between ISO=800 (weight 0.5) and ISO=1600 (weight 0.5), and simultaneously between color temperature=4500K (weight 0.36) and color temperature=5600K (weight 0.64). The current compensation parameters are obtained after bilinear interpolation.

[0071] 3.6 Modeling constraints and noise resistance.

[0072] During the construction of the deviation model, neighborhood continuity constraints and regional consistency constraints can be introduced to reduce the impact of single-point measurement noise and outlier sampling on model stability; and confidence-based weighted regression and outlier removal strategies can be used to further enhance robustness.

[0073] For example, a neighborhood continuity constraint is introduced (the first-order difference of the compensation between adjacent grid points does not exceed a preset threshold) to prevent abrupt changes in the 3D LUT; a confidence-based weighted regression is adopted, and the weight of low-confidence samples (Conf < 0.6) is reduced to 0.1 times that of normal samples.

[0074] It should be noted that this invention does not directly compensate based on simple DeltaE difference calculation. Instead, it models the joint response of the display link and the acquisition link through a process of "temporal binding - region extraction - unified representation - hierarchical modeling - model fusion - parameter correlation", thereby avoiding simplifying the core steps to conventional error correction.

[0075] 3.7 Model Output.

[0076] The deviation model output displays the mapping relationship between the input value and the compensation output value. The mapping relationship can be represented as a three-dimensional lookup table (3D LUT), a global color transformation matrix, a channel-specific nonlinear response curve, or a set of models indexed by parameters.

[0077] For example, the final deviation model is output in the form of a 3D LUT with 33×33×33 grid points, covering the 8-bit full color gamut (0~255), and each grid point stores the compensation output value of the R / G / B three channels (float32 precision).

[0078] Step 4: Generate compensation parameters.

[0079] Compensation parameters for correcting the display device output are generated based on the deviation model. These compensation parameters include three-dimensional lookup tables, matrix parameters, channel gain parameters, or combinations thereof. The compensation parameters can be converted to an adaptive format according to different hardware architectures, such as FPGA LUT format, GPU texture lookup tables, or display controller register configuration parameters. The compensation parameters are then written to the hardware processing unit of the display link to complete an initial closed-loop calibration.

[0080] For example, the compensation parameters are based on a 3D LUT with a 33×33×33 grid, and additional channel gain parameters (R_gain, G) can be added. gain B gain This is used for fast white point correction. For FPGA implementation: the float32 3D LUT is quantized into a 12-bit fixed-point format (precision approximately 0.024% of full scale), and stored in Block RAM structure for each of the RGB three channels (approximately 33 bytes per channel). 3 ×12bit≈430Kbit), accessed via trilinear interpolation logic; the write interface uses AXI4-Lite register mapping, supporting single LUT data refresh during the frame blanking period (approximately 700μs @1080p60). For GPU implementation: the 3D LUT is converted to a 32-bit floating-point RGBA texture (format GL). RGB32F ), accessed via trilinear interpolation through the OpenGL or Vulkan texture sampling interface, and color mapping is performed as a post-processing pass in the display pipeline.

[0081] Step 5: Trigger compensation update.

[0082] During the shooting process, the shooting parameters (operational parameters and / or environmental parameters) of the image acquisition terminal are acquired in real time. When a change in the shooting parameters is detected that meets a preset trigger condition, the compensation parameters are updated. The shooting parameters include at least one of the following: white balance parameters, color temperature parameters, ISO parameters, exposure parameters, gain parameters, shutter speed parameters, lens filter parameters, ambient light brightness parameters, and ambient light color temperature parameters.

[0083] The preset triggering conditions can be set to a single threshold strategy, a combined threshold strategy, or a priority triggering strategy. Hysteresis intervals, minimum update intervals, or stable frame count conditions can be introduced to avoid frequent switching of compensation parameters due to slight parameter jitter.

[0084] In one implementation, an update correspondence between "parameter type and compensation dimension" can be established: for example, changes in white balance prioritize updating compensation parameters related to white point and channel gain; changes in exposure or ISO prioritize updating compensation parameters related to brightness response and low grayscale areas; and changes in ambient light brightness or ambient light color temperature prioritize updating comprehensive color gamut compensation parameters.

[0085] In another implementation, without re-sampling the entire dataset, rapid updates can be achieved by selecting a matching model from a pre-established set of models for different discrete shooting parameter values, interpolating the two models whose shooting parameter values ​​are closest to the current shooting parameter values, or performing local parameter updates, thereby reducing the cost of on-site recalibration.

[0086] For example, during shooting, metadata from the image acquisition terminal (metadata embedded via SDI, camera parameters read via USB or Ethernet interface) and / or data from the external ambient light sensor are read on a per-frame or per-second basis.

[0087] Preset trigger condition (using hysteresis interval + stable frame counting strategy): White balance gain (R) gain Or B gain When the change exceeds 5%, the white point and channel gain related compensation parameters are updated; when the ISO changes by more than 1 stop (e.g., from 800 to 1600), the brightness response and low grayscale compensation parameters are updated; when the color temperature changes by more than 300K, the overall color gamut compensation parameters are updated; when the exposure time changes by more than 1 / 3 EV, the overall brightness compensation parameters are updated; when the ambient light brightness changes by more than 10% or the ambient light color temperature changes by more than 500K, the full compensation parameters are updated.

[0088] Hysteresis strategy: Taking ISO change as an example, the upper trigger threshold is level 1, and the lower trigger threshold is level 0.5; after triggering, N consecutive triggers are required. stable =5 frames of parameter change less than 0.5 levels, and the time interval T since the last trigger. min The next update can only be executed after ≥500ms.

[0089] After the update is triggered, the adjacent node models are searched from the two-dimensional model set established in step 3.5 according to the current ISO and color temperature values ​​and bilinear interpolation is performed. The updated compensation parameters are generated within 200ms, without having to re-execute the full sampling and modeling process of steps 1 to 3.

[0090] Step 6: Perform compensation at the hardware level.

[0091] The updated compensation parameters are loaded into the hardware processing unit in the display controller or display link, and mapping processing is performed on the pixel data in the video output path. The compensation parameters can be written to the display controller through double buffer switching, frame boundary synchronization switching, or buffer atomic replacement to reduce the risk of screen flicker during updates; and mapping calculations can be performed at the pixel level or sub-pixel level to meet the implementation requirements of different display hardware architectures.

[0092] The dual-buffering mechanism works as follows: The system maintains two 3D LUT buffers (buffer A and buffer B). The display controller reads one of the active buffers (initially buffer A) in real time to perform color mapping. When a parameter update is triggered, the new compensation parameters are written to the currently inactive buffer (buffer B). During the writing process, the display controller continues to read buffer A without affecting the display of the current frame. After the writing is completed, the buffer pointer is atomically switched at the next vertical sync signal (Vsync) boundary, so that the display controller reads buffer B from the next frame. The original buffer A becomes inactive and waits for the next update. This method ensures that the compensation parameter switching occurs at the frame boundary, avoiding screen tearing or flickering caused by parameter updates within the frame.

[0093] The compensation process is performed in the pixel data output path of the display link: the RGB values ​​of each pixel in each frame of the image are mapped by 3D LUT trilinear interpolation and then output to the display driver to complete real-time color compensation at the pixel level. The end-to-end processing delay does not exceed 1 frame (approximately 16.7ms @60Hz).

[0094] Performance metrics for a typical implementation scenario: The display device is an LED video wall, 2×4 spliced, with a single screen resolution of 1920×1080 and a refresh rate of 60Hz; the image acquisition terminal is a digital cinema camera with a frame rate of 24fps; the average ΔE across the entire color gamut after calibration... 00 Less than 2.0 (target value); Dark areas (L < 20) ΔE 00 Less than 3.0 (target value); parameter trigger update response time less than 200ms; hardware compensation end-to-end latency less than or equal to 1 frame (approximately 16.7ms @ 60Hz).

[0095] Example 2: A color adaptive calibration system for display devices based on real-time image feedback. For example... Figure 5 As shown, the system includes: a standard feature pattern generation module 101, an image feedback acquisition module 103, an adaptive deviation analysis engine 104, a dynamic correction and compensation module 105, a parameter monitoring and triggering module 107, and a hardware loading module 106.

[0096] The standard feature pattern generation module 101 is configured to control the display device 102 to output multiple sampling blocks, grayscale blocks, color bar blocks, or composite color blocks in a predetermined order. To balance sampling efficiency and modeling accuracy, sampling points can be distributed using a three-dimensional mesh structure, and low grayscale areas, neutral gray areas, skin color areas, or areas of interest to the target business can be sampled more densely.

[0097] The image feedback acquisition module 103 is configured to acquire feedback image data from the video output interface, raw data interface, or external acquisition device of the image acquisition terminal, and extract the region of interest from the feedback image data according to the display area position to obtain the feedback measurement value. If necessary, the feedback image data can also be denoised, subjected to inverse gamma transform, white balance normalization, or color space conversion to compare with the sampled input value in a unified computational space.

[0098] The adaptive deviation analysis engine 104 is configured to calculate the deviation between the sampled input value and the feedback measurement value. This deviation can be represented as channel difference, white point offset, comprehensive color difference, or a multidimensional deviation vector. Based on this deviation, a deviation model can be built using interpolation, piecewise fitting, polynomial fitting, weighted regression, or lookup table construction. The engine may include a sample binding unit (for temporal / spatial binding), a unified representation unit (for mapping the display input value and feedback measurement value to the same color space), a hierarchical modeling unit (for building a global model and a local refinement model), a model fusion unit (for fusing the global model and the local refinement model), a parameter association unit (for building a set of models based on discrete shooting parameter values), and a model selection unit (for selecting a matching model based on the current shooting parameter value, or interpolating between the two models whose shooting parameter values ​​are closest to the current shooting parameter value).

[0099] The dynamic correction and compensation module 105 is configured to generate compensation parameters based on the deviation model and output them to the hardware loading module. The compensation parameters may be in the form of a three-dimensional lookup table, a color transformation matrix, a channel-specific nonlinear curve, a local gain parameter, or a combination thereof.

[0100] The parameter monitoring and triggering module 107 is configured to read the metadata and / or external environment parameters of the image acquisition terminal in real time. When the parameter change exceeds the threshold, the compensation parameter is triggered to update; when the parameter change does not exceed the threshold, the current compensation parameter can be kept unchanged to reduce unnecessary frequent refreshes.

[0101] The hardware loading module 106 is configured to write compensation parameters into the hardware processing unit in the display controller or display link. To reduce switching jitter, parameter switching is preferably performed at the frame boundary, and a dual-buffer method is used to achieve seamless switching between old and new parameters.

[0102] Example 3: Multi-display device expansion.

[0103] When multiple display devices participate in shooting simultaneously, a device identifier and initial deviation model can be established for each display device. Alternatively, synchronous sampling, synchronous modeling, and synchronous compensation updates can be performed on multiple display devices under a unified master control node.

[0104] For multiple display devices of the same model, a common basic model can be established first, and then individual fine-tuning can be carried out based on the local deviations of each device. For multiple display devices of different models, device-level models can be established separately, and the output can be uniformly constrained under the same reference color space to improve the imaging consistency between multiple devices.

[0105] In one implementation, the master node can uniformly issue sampling tasks and synchronization trigger signals, and perform frame synchronization control on the loading of compensation parameters for multiple devices to reduce visual flickering or color jumps when switching between multiple screens.

Claims

1. A color adaptive calibration method for display devices based on real-time image feedback, characterized in that, include: The control display device outputs sampled blocks according to a preset color sampling sequence. The color sampling sequence includes global basic sampling points and performs encrypted sampling on low grayscale areas and / or preset sensitive color gamut areas. The low grayscale area is the range of 0 to 32 under 8-bit driving value, and the preset sensitive color gamut area includes at least one of skin color area, neutral gray area or area near white point. The sampling blocks output by the display device are acquired by the image acquisition terminal, feedback image data is obtained, and the temporal and spatial correspondence between the color sampling sequence and the feedback image data is established to extract the feedback measurement value corresponding to each sampling block. The display input value of the sampled image block is bound to the feedback measurement value as a sample pair. A hierarchical deviation model containing a global model and a local refinement model is constructed in a unified computing space. The global model is used to characterize white point offset, channel gain imbalance and overall brightness response offset. The local refinement model is used to characterize nonlinear deviations in low grayscale areas, skin color areas, neutral gray areas or white point neighborhoods. The global model and the local refinement model are then fused. Compensation parameters for correcting the output of the display device are generated based on the fused layered deviation model; The system acquires the shooting parameters of the image acquisition terminal in real time, and updates the compensation parameters when a change in the shooting parameters is detected to meet a preset trigger condition. The updated compensation parameters are loaded into the hardware processing unit in the display controller or display link, and the pixel data is mapped in the video output path.

2. The color adaptive calibration method for display devices based on real-time image feedback according to claim 1, characterized in that, Establishing the temporal and spatial correspondences includes: The sampling patch number corresponding to the current image frame is determined by using the output timing identifier, frame number, synchronization trigger signal or timestamp of the sampling patch; After determining the sampling patch number, the region of interest corresponding to the spatial position of the sampling patch in the image frame is determined by using a preset display layout mapping relationship, positioning mark recognition, geometric correction relationship or perspective transformation relationship, and the pixels in the region of interest are statistically processed to obtain the feedback measurement value.

3. The color adaptive calibration method for display devices based on real-time image feedback according to claim 1, characterized in that, The methods for fusing the global model and the local refinement model include: local replacement, using the output of the local model as a correction term for the output of the global model, or using a weight function to smoothly transition between the global output and the local output.

4. The color adaptive calibration method for display devices based on real-time image feedback according to claim 1, characterized in that, The shooting parameters include at least one of white balance parameters, color temperature parameters, ISO parameters, exposure parameters, gain parameters, shutter parameters, lens filter parameters, ambient light brightness parameters, and ambient light color temperature parameters; the preset triggering conditions include single threshold strategy, combined threshold strategy, or priority triggering strategy, and introduce hysteresis interval, minimum update time interval, or stable frame count conditions.

5. The color adaptive calibration method for display devices based on real-time image feedback according to claim 1, characterized in that, The compensation parameters are updated in the following ways: without re-sampling the entire dataset, the updated compensation parameters are obtained by selecting a model that matches the current shooting parameter value from multiple deviation models pre-established for different shooting parameter values, interpolating the two models whose shooting parameter values ​​are closest to the current shooting parameter value, or performing a local update on the compensation parameters.

6. The color adaptive calibration method for display devices based on real-time image feedback according to claim 1, characterized in that, The compensation parameters are in the form of a three-dimensional lookup table, a global color transformation matrix, a channel-specific nonlinear response curve, or a set of models indexed by parameters. When the compensation parameters are loaded into the hardware processing unit, they are written to the display controller using double buffer switching, frame boundary synchronous switching, or buffer atomic replacement.

7. The color adaptive calibration method for display devices based on real-time image feedback according to any one of claims 1-6, characterized in that, When multiple display devices participate in shooting simultaneously, a unified master control node is used to perform synchronous sampling, synchronous modeling, and synchronous compensation updates on multiple display devices. For multiple display devices of the same model, a common basic model is first established, and then personalized fine-tuning is performed based on the local deviations of each display device. For multiple display devices of different models, device-level models are established separately, and the output is uniformly constrained under the same reference color space to improve the imaging consistency between multiple devices.

8. A color adaptive calibration system for display devices based on real-time image feedback, characterized in that, include: The standard feature pattern generation module is configured to control the display device to output sampling blocks according to a preset color sampling sequence. The color sampling sequence includes global basic sampling points and performs encrypted sampling on low grayscale areas and / or preset sensitive color gamut areas. The low grayscale areas are the range of 0 to 32 under 8-bit driving values, and the preset sensitive color gamut areas include at least one of skin color areas, neutral gray areas, or areas near white points. The image feedback acquisition module is configured to acquire the sampled image blocks output by the display device through the image acquisition terminal, acquire feedback image data, and establish the temporal and spatial correspondence between the color sampling sequence and the feedback image data, so as to extract the feedback measurement values ​​corresponding to each sampled image block. An adaptive deviation analysis engine is configured to bind the display input value of the sampled patch with the feedback measurement value as a sample pair, construct a hierarchical deviation model in a unified computation space that includes a global model and a local refinement model. The global model is used to characterize white point offset, channel gain imbalance and overall brightness response offset, and the local refinement model is used to characterize nonlinear deviations in low grayscale areas, skin color areas, neutral gray areas or white point neighborhoods, and the global model and the local refinement model are fused. The dynamic correction and compensation module is configured to generate compensation parameters for correcting the output of the display device based on the fused layered deviation model; The parameter monitoring and triggering module is configured to acquire the shooting parameters of the image acquisition terminal in real time, and update the compensation parameters when the changes in the shooting parameters meet the preset triggering conditions. The hardware loading module is configured to load the updated compensation parameters into the hardware processing unit in the display controller or display link, and perform mapping processing on the pixel data in the video output path.

9. The color adaptive calibration system for display devices based on real-time image feedback according to claim 8, characterized in that, The adaptive deviation analysis engine includes: The sample binding unit is used to establish the temporal and spatial correspondence between the displayed input values ​​of the sampled plot and the corresponding feedback measurement values; A unified representation unit is used to map the display input values ​​and feedback measurement values ​​to the same color space; Hierarchical modeling units are used to construct global models and local refinement models; A model fusion unit is used to fuse the global model with the local refinement model; And a parameter association unit, used to establish a model set based on discrete shooting parameter values, and select a matching model from the model set according to the current shooting parameter value, or to interpolate the two models whose shooting parameter values ​​are closest to the current shooting parameter value.

Images