A method and system for evaluating the quality of an HDR image based on a large model

Abstract

This invention provides a method and system for evaluating HDR image quality based on a large model. The invention converts the original HDR image to a perceptually uniform color space, separating the luminance and chrominance components. It also generates multi-exposure SDR image sequences and color gamut visualization scatter plots. These multi-exposure SDR image sequences, color gamut visualization scatter plots, and HDR standard encoding are used as multimodal input cues. These multimodal input cues are then input into a pre-trained multimodal large model to obtain the quality evaluation result of the original HDR image to be evaluated. This invention achieves luminance and chrominance decomposition by converting HDR to a perceptually uniform ICtCp space, ensuring the perceptual uniformity of luminance adjustment. Under the premise of fixed chrominance, it generates multi-level multi-exposure images through an optimal information entropy strategy, fully covering the entire luminance range. Color gamut visualization provides an intuitive representation of color distribution, allowing the large model to improve the accuracy of the image quality evaluation results.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and multimedia signal processing, image processing technology, and in particular to a method and system for evaluating HDR image quality based on a large model. Background Technology

[0002] With the rapid development of 4K / 8K ultra-high-definition and HDR display technologies, high-quality video content production has become a core requirement for film and television production, streaming services, and the consumer electronics industry. HDR technology significantly enhances the visual appeal of images by expanding brightness and color gamut, but it also introduces complex perceptual quality evaluation issues.

[0003] Firstly, existing technologies include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and their improved versions in the HDR field (such as HDR-VDP, PU-SSIM, etc.). These methods calculate the degree of image distortion using mathematical models, but their core drawback is that traditional indicators are based on pixel-level error calculations and cannot accurately reflect the perceptual characteristics of the human visual system to HDR content. For example, the perceptual changes brought about by the bright areas and color gamut expansion of HDR images are difficult to capture with simple error measures. Moreover, focusing only on brightness or structural distortion lacks a comprehensive evaluation of HDR-specific attributes (such as dynamic range preservation and color gamut accuracy). Improved methods such as HDR-VDP require manual parameter adjustment to adapt to different display environments, reducing the practicality and consistency of the methods.

[0004] Secondly, in recent years, deep quality assessment models based on convolutional neural networks or Transformers, such as NIMA and LPIPS, have emerged. While these methods have made some progress in SDR image evaluation, they suffer from the following problems in HDR applications: they require large-scale labeled data for training, but subjective labeled data for HDR quality is scarce and costly. Moreover, the trained models often overfit to specific datasets, making it difficult to adapt to different HDR formats (PQ / HLG), different color gamuts (BT.2020 / DCI-P3), and different content types. Furthermore, as "black box" models, they are difficult to provide specific quality issues and improvement suggestions, hindering problem diagnosis in practical applications.

[0005] Third, with the development of multimodal large models, methods using general large models for image analysis have emerged. While these methods possess strong semantic understanding capabilities, they suffer from fundamental flaws in HDR quality evaluation: general large models have not been specifically trained with HDR domain knowledge and lack a deep understanding of HDR technical characteristics, such as dynamic range and color gamut. Moreover, existing large models are primarily trained on SDR images; directly inputting HDR images leads to dynamic range compression and color distortion, resulting in the loss of crucial HDR information.

[0006] Therefore, there is an urgent need to provide a novel HDR image quality evaluation scheme based on a large model. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method and system for evaluating HDR image quality based on a large model.

[0008] In a first aspect, the present invention provides a method for evaluating HDR image quality based on a large model, comprising:

[0009] The original HDR image to be evaluated is acquired, and the original HDR image is converted to a perceptually uniform color space to separate the luminance component and chrominance component.

[0010] Based on the normalized histogram probability distribution of the luminance component and the Shannon information entropy, multiple exposure gains are selected to perform tone mapping processing on the luminance component, generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range.

[0011] Scattered chromaticity coordinates are calculated for the pixels of the original HDR image to be evaluated, and a scatter plot of color gamut visualization with superimposed BT.2020 color gamut reference boundary is generated based on the scattered chromaticity coordinates.

[0012] The multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard encoding are used to form multimodal input prompts. The multimodal input prompts are then input into a pre-trained multimodal large model to obtain the quality assessment results of the original HDR image to be evaluated.

[0013] A structured evaluation report is generated based on the quality assessment results of the original HDR image to be evaluated.

[0014] Preferably, the process involves acquiring the original HDR image to be evaluated, converting the original HDR image to a perceptually uniform color space, and separating the luminance and chrominance components, including:

[0015] Perform inverse PQ calculations on the R, G, and B channels of the original HDR image to be evaluated to obtain a linear light RGB image;

[0016] Based on the BT.2020 transformation matrix, the linear light RGB image is mapped to the CIEXYZ color space to obtain the CIE tristimulus value column vector;

[0017] The response of three types of cone cells in the human retina to light was simulated. The CIE tristimulus value column vector was transformed into the LMS cone cell response space through the CIE CAT02 color adaptation matrix to obtain the LMS cone response vector.

[0018] PQ encoding is applied to the LMS cone response vector to match the nonlinear brightness perception characteristics of the human eye, resulting in a perceptually uniform LMS cone response vector. The perceptually uniform LMS cone response vector is then transformed and decomposed into independent luminance and chrominance components using the ICtCp transformation matrix.

[0019] Preferably, based on the normalized histogram probability distribution of the luminance component and the Shannon information entropy, multiple sets of exposure gains are selected to perform tone mapping processing on the luminance component, generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range, including:

[0020] Calculate the probability distribution of the normalized histogram of the luminance components;

[0021] Calculate the visual information entropy corresponding to each candidate exposure gain based on Shannon information entropy;

[0022] Randomly select s non-repeating exposure gains from the candidate set of exposure gains as an exposure gain combination; calculate the total visual information entropy of each exposure gain combination, and take the exposure gain combination with the largest total visual information entropy as the final exposure gain set;

[0023] The luminance component is tone-mapped using each exposure gain in the final exposure gain set to obtain the stretched luminance component.

[0024] The Reinhard global tone mapping operator is used to perform dynamic range compression on the stretched luminance components to obtain the SDR luminance components;

[0025] The SDR luminance component and the chrominance component are concatenated to form a new ICtCp vector. The RGB color signal is then restored layer by layer using the new ICtCp vector. After scaling, a single SDR image corresponding to the current exposure gain is obtained. The single SDR images corresponding to s exposure gains are then concatenated to obtain a multi-exposure SDR image sequence.

[0026] Preferably, the calculation of the luminance component's luminance normalized histogram probability distribution includes:

[0027] Mapping the luminance components to the [0,1] interval yields the normalized luminance components;

[0028] Set up fixed bins to statistically normalize the luminance component luminance distribution, count the number of pixels at each luminance level, and calculate the normalized probability distribution of pixel occurrence at each luminance level; remove empty luminance levels with no pixel distribution to obtain the normalized histogram probability distribution of the luminance component.

[0029] Preferably, the visual information entropy corresponding to each candidate exposure gain is calculated based on Shannon information entropy, including:

[0030] A preset set of exposure gain candidates covering the highlight, midtone, and shadow regions is provided;

[0031] For each exposure gain in the candidate set of exposure gains, the normalized luminance component is linearly stretched to simulate exposure changes;

[0032] The probability distribution of the brightness components after stretching was recalculated.

[0033] Based on the probability distribution of the stretched luminance components, the visual information entropy of the probability distribution of the stretched luminance is calculated using Shannon information entropy.

[0034] Preferably, based on the probability distribution of the stretched luminance components, the visual information entropy of the probability distribution of the stretched luminance is calculated using Shannon information entropy, and expressed as:

[0035] ;

[0036] In the formula, This represents the total visual information corresponding to the g-th exposure gain; The probability distribution of the k-th level brightness of the brightness component after stretching following the application of exposure gain g.

[0037] Preferably, the scatter chromaticity coordinates of the pixels in the original HDR image to be evaluated are calculated, and a scatter plot of color gamut visualization overlaid with the BT.2020 color gamut reference boundary is generated based on the scatter chromaticity coordinates, including:

[0038] Each effective pixel of the original HDR image to be evaluated is treated as a scatter point, and the chromaticity coordinates of each scatter point are calculated based on the CIE tristimulus value column vector.

[0039] Retrieve the CIExy coordinates corresponding to the red, green, and blue vertices specified in the ITU-R BT.2020 standard, and connect them in the order of red vertex to green vertex, green vertex to blue vertex, and blue vertex back to red vertex to form a closed triangle outline. Set the closed triangle outline to a red dashed line style as a visual reference benchmark.

[0040] Scattered points are plotted based on the chromaticity coordinates of all valid pixels as planar positions, and each scatter point is filled with a gradient color based on the normalized luminance component. The scatter point layer is then set below the visual reference datum to obtain a color gamut visualization scatter plot.

[0041] Secondly, the present invention provides an HDR image quality evaluation system based on a large model, comprising:

[0042] The luminance-chrominance separation module is used to acquire the original HDR image to be evaluated, convert the original HDR image to a perceptually uniform color space, and separate the luminance component and chrominance component.

[0043] The multi-exposure decomposition module is used to select multiple sets of exposure gains based on the normalized histogram probability distribution and Shannon information entropy of the luminance component to perform tone mapping processing on the luminance component, thereby generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range.

[0044] The color gamut visualization module is used to calculate the scatter chromaticity coordinates of the pixels of the original HDR image to be evaluated, and generate a color gamut visualization scatter plot superimposed with the BT.2020 color gamut reference boundary based on the scatter chromaticity coordinates.

[0045] The inference module is used to encode the multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard into multimodal input prompts, and input the multimodal input prompts into a pre-trained multimodal large model to obtain the quality assessment results of the original HDR image to be evaluated.

[0046] The output module is used to generate a structured evaluation report based on the quality assessment results of the original HDR image to be evaluated.

[0047] Thirdly, the present invention provides an electronic device, comprising:

[0048] At least one processor;

[0049] and a memory communicatively connected to the at least one processor;

[0050] The memory stores a computer program that can be executed by the at least one processor, and when the computer program is executed by the at least one processor, it implements the large model-based HDR image quality evaluation method.

[0051] Fourthly, the present invention provides a computer storage medium storing a computer program, which, when executed by a processor, implements the HDR image quality evaluation method based on a large model as described in the embodiments.

[0052] The beneficial technical effects of this invention are as follows:

[0053] 1. This invention converts HDR to a perceptually uniform ICtCp space to achieve luminance and chrominance decomposition, ensuring the perceptual uniformity of luminance adjustment. Under the premise of fixed chrominance, it generates multi-level multi-exposure images through an optimal information entropy strategy, fully covering the entire luminance range. It provides an intuitive representation of color distribution through color gamut visualization, transforming the wide dynamic range and wide color gamut characteristics of HDR into standardized SDR visual materials that can be recognized by large models. This allows large models to analyze image quality defects layer by layer from highlights, midtones, and shadows, improving the accuracy of image quality assessment results.

[0054] 2. This invention achieves luminance and chromaticity decomposition through the ICtCp space, solving the problem of losing HDR highlights, shadows, and wide color gamut details in general large models;

[0055] 3. This invention uses Shannon information entropy to select the optimal exposure gain with the highest total information content, stretching and brightening shadows and suppressing highlights respectively, and then converting them into multiple SDR images through tone mapping; the sequence completely covers the entire dynamic range of HDR, solving the defect that large models cannot resolve the wide dynamic range of HDR;

[0056] 4. This invention only performs exposure stretching and tone mapping on the luminance component, so all generated SDR images have completely consistent colors, only differing in the range of light and dark coverage; when analyzing color distortion in a large model, it can eliminate color interference caused by exposure, accurately distinguish the actual image quality problems such as color cast, insufficient saturation, and color banding that exist in the image itself, and greatly improve the accuracy of color dimension evaluation.

[0057] 5. This invention enables large models to have complete color analysis capabilities based on color gamut visualization scatter plots. Large models can intuitively distinguish the color gamut distribution of highlight, midtone, and shadow pixels through the color gradient of scatter plots, accurately identify unique distortions such as color compression in high-brightness areas and color banding in dark areas, and solve the problem of large deviation between traditional color gamut numerical indicators and human subjective perception. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention;

[0059] Figure 2 This is a structural framework diagram of the system according to an embodiment of the present invention;

[0060] In the diagram, 10 is the brightness and color separation module; 20 is the multi-exposure decomposition module; 30 is the color gamut visualization module; 40 is the inference module; and 50 is the output module. Detailed Implementation

[0061] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0062] like Figure 1 As shown, this embodiment provides a method for evaluating HDR image quality based on a large model, including:

[0063] S1: Obtain the original HDR image to be evaluated, convert the original HDR image to a perceptually uniform color space, and separate the luminance component and chrominance component;

[0064] S2: Based on the normalized histogram probability distribution of the luminance component and the Shannon information entropy, select multiple exposure gains to perform tone mapping processing on the luminance component, and generate a multi-exposure SDR image sequence that fully covers the HDR dynamic range.

[0065] S3: Calculate the scatter chromaticity coordinates of the pixels in the original HDR image to be evaluated, and generate a scatter plot of color gamut visualization based on the scatter chromaticity coordinates and superimposed with the BT.2020 color gamut reference boundary;

[0066] S4: The multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard encoding are converted into multimodal input prompts. The multimodal input prompts are then input into a pre-trained multimodal large model to obtain the quality assessment results of the original HDR image to be evaluated.

[0067] S5: Generate a structured evaluation report based on the quality assessment results of the original HDR image to be evaluated.

[0068] In this embodiment, step S1 involves acquiring the original HDR image to be evaluated, converting the original HDR image to a perceptually uniform color space, and separating the luminance and chrominance components, including:

[0069] S11: Perform inverse PQ calculations on the R, G, and B channels of the original HDR image to be evaluated to obtain a linear light RGB image;

[0070] S12: Based on the BT.2020 transformation matrix, map the linear light RGB image to the CIEXYZ color space to obtain the CIE tristimulus value column vector, i.e.:

[0071] ;

[0072] In the formula, This is a column vector of CIE tristimulus values; This is the BT.2020 transformation matrix; These are the RGB three-channel components of a linear light RGB image;

[0073] In some embodiments, the BT.2020 transformation matrix is:

[0074] .

[0075] S13: Simulating the response of the three types of cone cells in the human retina to light, the CIE tristimulus value column vector is transformed into the LMS cone cell response space using the CIE CAT02 color adaptation matrix, resulting in the LMS cone response vector, i.e.:

[0076] ;

[0077] In the formula, This is the LMS cone response vector; This is the CIE CAT02 color adaptation matrix.

[0078] In some embodiments, the CIECAT02 color adaptation matrix for:

[0079] .

[0080] S14: Apply PQ encoding to the LMS cone response vector to match the nonlinear brightness perception characteristics of the human eye, thereby obtaining a perceptually uniform LMS cone response vector. And the ICtCp transformation matrix is ​​used to transform the perceived uniform LMS cone response vector. The conversion is decomposed into independent luminance and chrominance components, that is:

[0081] ;

[0082] In the formula, For the luminance component; , For chromaticity components, The blue-yellow chromaticity component; The red-green color difference component; Let ItCp be the transformation matrix.

[0083] In this embodiment, the transformation matrix using ICtCp is described. Represented as:

[0084] .

[0085] In this embodiment, in step S2, multiple exposure gains are selected based on the normalized histogram probability distribution of the luminance component and the Shannon information entropy to perform tone mapping processing on the luminance component, generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range, including:

[0086] S21: Calculate the luminance component The probability distribution of the brightness normalized histogram includes:

[0087] Brightness component Mapping to the [0,1] interval yields the normalized luminance component. ;

[0088] Set fixed-block statistical normalized luminance components The brightness distribution is analyzed, the number of pixels at each brightness level is counted, and the normalized probability distribution of pixel occurrence at each brightness level is calculated. Empty brightness levels with no pixel distribution are removed, resulting in the normalized histogram probability distribution of the brightness components, i.e.:

[0089] ;

[0090] In the formula, For the first The probability of brightness level appearing; The number of pixels in each bin; , The width and height of the original HDR image to be evaluated;

[0091] S22: Calculate the visual information entropy corresponding to each candidate exposure gain based on Shannon information entropy, including:

[0092] A preset set of exposure gain candidates covering highlights, midtones, and shadows is used. For each exposure gain within the exposure gain candidate set, the normalized luminance component is... Perform linear stretching to simulate exposure changes, re-statistically analyze the probability distribution of brightness after stretching, and calculate the visual information entropy of the probability distribution of brightness after stretching using Shannon information entropy, i.e.:

[0093] ;

[0094] In the formula, This represents the total visual information corresponding to the g-th exposure gain; The probability distribution of the k-th level brightness of the brightness component after stretching following the application of exposure gain g.

[0095] S23: Randomly select s non-repeating exposure gains from the exposure gain candidate set as an exposure gain combination; calculate the total visual information entropy of each exposure gain combination, and take the exposure gain combination with the maximum total visual information entropy as the final exposure gain set;

[0096] In this embodiment, five non-repeating exposure gains are randomly selected from the exposure gain candidate set as an exposure gain combination.

[0097] S24: Utilize each exposure gain in the final exposure gain set to adjust the luminance component. After performing tone mapping, the stretched luminance component is obtained, i.e.:

[0098] ;

[0099] In the formula, For the first One stretched brightness component; For the final exposure gain set of the Exposure gain.

[0100] S25: The Reinhard global tone mapping operator is used to perform dynamic range compression on the stretched luminance components to obtain the SDR luminance components, i.e.:

[0101] ;

[0102] In the formula, For the first One SDR luminance component;

[0103] S26: The SDR luminance component and the chrominance component are concatenated to form a new ICtCp vector. The new ICtCp vector is used to restore the RGB color signal layer by layer. After scaling, a single SDR image corresponding to the current exposure gain is obtained. The single SDR images corresponding to s exposure gains are concatenated to obtain a multi-exposure SDR image sequence.

[0104] In this embodiment, step S3 involves calculating the scatter chromaticity coordinates of the pixels in the original HDR image to be evaluated, and generating a scatter plot of color gamut visualization overlaid with the BT.2020 color gamut reference boundary based on the scatter chromaticity coordinates, including:

[0105] S31: Treat each valid pixel of the original HDR image to be evaluated as a scatter point, and calculate the chromaticity coordinates of each scatter point based on the CIE tristimulus value column vector; that is:

[0106] ;

[0107] ;

[0108] In the formula, These are the horizontal chromaticity coordinates and the vertical chromaticity coordinates, respectively. , , This is a column vector of CIE tristimulus values.

[0109] In this embodiment, if the sum of the total stimulus values ​​of a single pixel is 0, the corresponding pixel is determined to be a pure black pixel with no brightness, and the corresponding pixel is discarded.

[0110] S32: Retrieve the CIExy coordinates corresponding to the red, green, and blue vertices specified in the ITU-RBT.2020 standard, connect the lines in the order of red vertex to green vertex, green vertex to blue vertex, and blue vertex back to red vertex to form a closed triangle outline, and set the closed triangle outline to a red dashed line style as a visual reference benchmark;

[0111] S33: Draw scatter points based on the chromaticity coordinates of all valid pixels as planar positions, fill each scatter point with a gradient color based on the normalized luminance component, and set the scatter point layer below the visual reference base to obtain a color gamut visualization scatter plot.

[0112] In this embodiment, in step S4, the multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard encoding are converted into multimodal input cues. These multimodal input cues are then input into a pre-trained multimodal large model to obtain the quality assessment result of the original HDR image to be evaluated, including:

[0113] S41: Encode and label the multi-exposure SDR image sequence, specify the exposure gain value corresponding to each SDR image in the prompt text, and classify the observation purpose of each SDR image.

[0114] S42: Perform standardized text interpretation encoding on the color gamut visualization scatter plot; specify the meaning of the scatter distribution range and scatter cluster density in the prompt text;

[0115] S43: Encode all HDR quality evaluation metrics into the prompt text;

[0116] S44: Integrate all prompt text, multi-exposure SDR image sequences, and color gamut visualization scatter plots into a multimodal input prompt;

[0117] S45: Perform inference on the multimodal large model pre-trained with multimodal input prompts to obtain the quality assessment results of the original HDR image to be evaluated.

[0118] In this embodiment, in step S5, the structured evaluation report includes a quality score of 1-100 points, a natural language description of quality problems, and targeted image quality optimization suggestions.

[0119] like Figure 2 As shown, another embodiment of this application provides a large-model-based HDR image quality evaluation system, including:

[0120] The luminance-color separation module 10 is used to acquire the original HDR image to be evaluated, convert the original HDR image to a perceptually uniform color space, and separate the luminance component and chrominance component.

[0121] The multi-exposure decomposition module 20 is used to select multiple exposure gains based on the normalized histogram probability distribution and Shannon information entropy of the luminance component to perform tone mapping processing on the luminance component, thereby generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range.

[0122] The color gamut visualization module 30 is used to calculate the scatter chromaticity coordinates of the pixels of the original HDR image to be evaluated, and generate a color gamut visualization scatter plot with superimposed BT.2020 color gamut reference boundary based on the scatter chromaticity coordinates.

[0123] The inference module 40 is used to encode the multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard into multimodal input prompts, input the multimodal input prompts into a pre-trained multimodal large model, and obtain the quality assessment result of the original HDR image to be evaluated.

[0124] Output module 50 is used to generate a structured evaluation report based on the quality assessment results of the original HDR image to be evaluated.

[0125] In this embodiment, the luminance-chrominance separation module 10 acquires the original HDR image to be evaluated, converts the original HDR image to a perceptually uniform color space, and separates the luminance component and chrominance component, including:

[0126] The brightness and color separation module 10 performs inverse PQ calculations on the R, G, and B channels of the original HDR image to be evaluated to obtain a linear light RGB image.

[0127] The luminance-color separation module 10 maps the linear light RGB image to the CIEXYZ color space based on the BT.2020 transformation matrix, obtaining a CIE tristimulus value column vector, namely:

[0128] ;

[0129] In the formula, This is a column vector of CIE tristimulus values; This is the BT.2020 transformation matrix; These are the RGB three-channel components of a linear light RGB image;

[0130] In some embodiments, the BT.2020 transformation matrix is:

[0131] .

[0132] The brightness-color separation module 10 simulates the light response of three types of cone cells in the human retina. It transforms the CIE tristimulus value column vector into the LMS cone cell response space using the CIE CAT02 color adaptation matrix, obtaining the LMS cone response vector, i.e.:

[0133] ;

[0134] In the formula, This is the LMS cone response vector; This is the CIE CAT02 color adaptation matrix.

[0135] In some embodiments, the CIECAT02 color adaptation matrix for:

[0136] .

[0137] The luminance-color separation module 10 applies PQ encoding to the LMS cone response vector to match the nonlinear luminance perception characteristics of the human eye, thereby obtaining a perceptibly uniform LMS cone response vector. And the ICtCp transformation matrix is ​​used to transform the perceived uniform LMS cone response vector. The conversion is decomposed into independent luminance and chrominance components, that is:

[0138] ;

[0139] In the formula, For the luminance component; , For chromaticity components, The blue-yellow chromaticity component; The red-green color difference component; Let ItCp be the transformation matrix.

[0140] In this embodiment, the transformation matrix using ICtCp is described. Represented as:

[0141] .

[0142] In this embodiment, the multi-exposure decomposition module 20 selects multiple sets of exposure gains based on the normalized histogram probability distribution of the luminance component and Shannon information entropy to perform tone mapping processing on the luminance component, generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range, including:

[0143] The multi-exposure decomposition module 20 calculates the luminance component. The probability distribution of the brightness normalized histogram includes:

[0144] Brightness component Mapping to the [0,1] interval yields the normalized luminance component. ;

[0145] Set fixed-block statistical normalized luminance components The brightness distribution is analyzed, the number of pixels at each brightness level is counted, and the normalized probability distribution of pixel occurrence at each brightness level is calculated. Empty brightness levels with no pixel distribution are removed, resulting in the normalized histogram probability distribution of the brightness components, i.e.:

[0146] ;

[0147] In the formula, For the first The probability of brightness level appearing; The number of pixels in each bin; , The width and height of the original HDR image to be evaluated;

[0148] The multi-exposure decomposition module 20 calculates the visual information entropy corresponding to each candidate exposure gain based on Shannon information entropy, including:

[0149] A preset set of exposure gain candidates covering highlights, midtones, and shadows is used. For each exposure gain within the exposure gain candidate set, the normalized luminance component is... Perform linear stretching to simulate exposure changes, re-statistically analyze the probability distribution of brightness after stretching, and calculate the visual information entropy of the probability distribution of brightness after stretching using Shannon information entropy, i.e.:

[0150] ;

[0151] In the formula, This represents the total visual information corresponding to the g-th exposure gain; The probability distribution of the k-th level brightness of the brightness component after stretching following the application of exposure gain g.

[0152] The multi-exposure decomposition module 20 randomly selects s ​​non-repeating exposure gains from the exposure gain candidate set as an exposure gain combination; and calculates the total visual information entropy of each exposure gain combination, and takes the exposure gain combination with the maximum total visual information entropy as the final exposure gain set;

[0153] In this embodiment, five non-repeating exposure gains are randomly selected from the exposure gain candidate set as an exposure gain combination.

[0154] The multi-exposure decomposition module 20 uses each exposure gain in the final exposure gain set to decompose the luminance component. After performing tone mapping, the stretched luminance component is obtained, i.e.:

[0155] ;

[0156] In the formula, For the first One stretched brightness component; For the final exposure gain set of the Exposure gain.

[0157] The multi-exposure decomposition module 20 uses the Reinhard global tone mapping operator to perform dynamic range compression on the stretched luminance component to obtain the SDR luminance component, i.e.:

[0158] ;

[0159] In the formula, For the first One SDR luminance component;

[0160] The multi-exposure decomposition module 20 concatenates the SDR luminance component and the chrominance component into a new ICtCp vector, uses the new ICtCp vector to restore the RGB color signal layer by layer, and obtains a single SDR image corresponding to the current exposure gain after scaling; and concatenates the single SDR images corresponding to s exposure gains to obtain a multi-exposure SDR image sequence.

[0161] In this embodiment, the color gamut visualization module 30 calculates the scatter chromaticity coordinates of the pixels in the original HDR image to be evaluated, and generates a color gamut visualization scatter plot superimposed with the BT.2020 color gamut reference boundary based on the scatter chromaticity coordinates, including:

[0162] The color gamut visualization module 30 treats each effective pixel of the original HDR image to be evaluated as a scatter point, and calculates the chromaticity coordinates of each scatter point based on the CIE tristimulus value column vector; that is:

[0163] ;

[0164] ;

[0165] In the formula, These are the horizontal chromaticity coordinates and the vertical chromaticity coordinates, respectively. , , This is a column vector of CIE tristimulus values.

[0166] In this embodiment, if the sum of the total stimulus values ​​of a single pixel is 0, the corresponding pixel is determined to be a pure black pixel with no brightness, and the corresponding pixel is discarded.

[0167] The color gamut visualization module 30 retrieves the CIExy coordinates corresponding to the red, green, and blue vertices specified in the ITU-R BT.2020 standard, and connects them in the order of red vertex to green vertex, green vertex to blue vertex, and blue vertex back to red vertex to form a closed triangle outline. The closed triangle outline is set as a red dashed line as a visual reference.

[0168] The color gamut visualization module 30 draws scatter points based on the chromaticity coordinates of all effective pixels as planar positions, fills each scatter point with a gradient color based on the normalized luminance component, and sets the scatter point layer below the visual reference base to obtain a color gamut visualization scatter plot.

[0169] In this embodiment, the inference module 40 encodes the multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard into multimodal input cues, and inputs these multimodal input cues into a pre-trained multimodal large model to obtain the quality assessment result of the original HDR image to be evaluated, including:

[0170] The inference module 40 encodes and labels the multi-exposure SDR image sequence, specifies the exposure gain value corresponding to each SDR image in the prompt text, and classifies the observation purpose of each SDR image.

[0171] The reasoning module 40 performs standardized text interpretation encoding processing on the color gamut visualization scatter plot; and specifies the meaning of the scatter distribution range and scatter cluster density in the prompt text.

[0172] The inference module 40 encodes all HDR quality evaluation indicators into the prompt text.

[0173] The reasoning module 40 integrates all prompt text, multi-exposure SDR image sequences, and color gamut visualization scatter plots into a multimodal input prompt.

[0174] The inference module 40 performs inference on the multimodal large model pre-trained by the multimodal input prompts to obtain the quality assessment result of the original HDR image to be evaluated.

[0175] In this embodiment, the structured evaluation report output by the output module 50 includes a quality score of 1-100, a natural language description of quality problems, and targeted image quality optimization suggestions.

[0176] A third embodiment of this application provides an electronic device, comprising:

[0177] At least one processor;

[0178] and a memory communicatively connected to the at least one processor;

[0179] The memory stores a computer program that can be executed by the at least one processor, and when the computer program is executed by the at least one processor, it implements the large model-based HDR image quality evaluation method.

[0180] The fourth embodiment of this application provides a computer storage medium storing a computer program that, when executed by a processor, implements the HDR image quality evaluation method based on a large model as described in the embodiment.

[0181] In this application, the memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. A processor, coupled to the memory, is used to execute computer programs stored in the memory.

[0182] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. Computer-readable media may include at least: any entity or device capable of carrying the computer program code to a photographic device / electronic device, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0183] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A method for evaluating the quality of HDR images based on a large model, characterized in that, include: The original HDR image to be evaluated is acquired, and the original HDR image is converted to a perceptually uniform color space to separate the luminance component and chrominance component. Based on the normalized histogram probability distribution of the luminance component and the Shannon information entropy, multiple exposure gains are selected to perform tone mapping processing on the luminance component, generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range. Scattered chromaticity coordinates are calculated for the pixels of the original HDR image to be evaluated, and a scatter plot of color gamut visualization with superimposed BT.2020 color gamut reference boundary is generated based on the scattered chromaticity coordinates. The multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard encoding are used to form multimodal input prompts. The multimodal input prompts are then input into a pre-trained multimodal large model to obtain the quality assessment results of the original HDR image to be evaluated. A structured evaluation report is generated based on the quality assessment results of the original HDR image to be evaluated.

2. The HDR image quality assessment method based on a large model according to claim 1, characterized in that, Acquire the original HDR image to be evaluated, convert the original HDR image to a perceptually uniform color space, and separate the luminance and chrominance components, including: Perform inverse PQ calculations on the R, G, and B channels of the original HDR image to be evaluated to obtain a linear light RGB image; Based on the BT.2020 transformation matrix, the linear light RGB image is mapped to the CIEXYZ color space to obtain the CIE tristimulus value column vector; The response of three types of cone cells in the human retina to light was simulated. The CIE tristimulus value column vector was transformed into the LMS cone cell response space through the CIE CAT02 color adaptation matrix to obtain the LMS cone response vector. PQ encoding is applied to the LMS cone response vector to match the nonlinear brightness perception characteristics of the human eye, resulting in a perceptually uniform LMS cone response vector. The perceptually uniform LMS cone response vector is then transformed and decomposed into independent luminance and chrominance components using the ICtCp transformation matrix.

3. The HDR image quality evaluation method based on a large model according to claim 2, characterized in that, Based on the normalized histogram probability distribution of the luminance component and the Shannon information entropy, multiple sets of exposure gains are selected to perform tone mapping processing on the luminance component, generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range, including: Calculate the probability distribution of the normalized histogram of the luminance components; Calculate the visual information entropy corresponding to each candidate exposure gain based on Shannon information entropy; Randomly select s non-repeating exposure gains from the candidate set of exposure gains as an exposure gain combination; calculate the total visual information entropy of each exposure gain combination, and take the exposure gain combination with the largest total visual information entropy as the final exposure gain set; The luminance component is tone-mapped using each exposure gain in the final exposure gain set to obtain the stretched luminance component. The Reinhard global tone mapping operator is used to perform dynamic range compression on the stretched luminance components to obtain the SDR luminance components; The SDR luminance component and the chrominance component are concatenated to form a new ICtCp vector. The RGB color signal is then restored layer by layer using the new ICtCp vector. After scaling, a single SDR image corresponding to the current exposure gain is obtained. The single SDR images corresponding to s exposure gains are then concatenated to obtain a multi-exposure SDR image sequence.

4. The HDR image quality assessment method based on a large model according to claim 3, characterized in that, Calculate the probability distribution of the normalized histogram of the luminance components, including: Mapping the luminance components to the [0,1] interval yields the normalized luminance components; Set up fixed bins to statistically normalize the luminance component luminance distribution, count the number of pixels at each luminance level, and calculate the normalized probability distribution of pixel occurrence at each luminance level; remove empty luminance levels with no pixel distribution to obtain the normalized histogram probability distribution of the luminance component.

5. The HDR image quality assessment method based on a large model according to claim 3, characterized in that, The visual information entropy corresponding to each candidate exposure gain is calculated based on Shannon information entropy, including: A preset set of exposure gain candidates covering the highlight, midtone, and shadow regions is provided; For each exposure gain in the candidate set of exposure gains, the normalized luminance component is linearly stretched to simulate exposure changes; The probability distribution of the brightness components after stretching was recalculated. Based on the probability distribution of the stretched luminance components, the visual information entropy of the probability distribution of the stretched luminance is calculated using Shannon information entropy.

6. The HDR image quality evaluation method based on a large model according to claim 5, characterized in that, Based on the probability distribution of the stretched luminance components, the visual information entropy of the probability distribution of the stretched luminance is calculated using Shannon information entropy, and is expressed as: ； In the formula, This represents the total visual information corresponding to the g-th exposure gain; The probability distribution of the k-th level brightness of the brightness component after applying exposure gain g and stretching.

7. The HDR image quality evaluation method based on a large model according to claim 2, characterized in that, The process involves calculating the chromaticity coordinates of the pixels in the original HDR image to be evaluated using scatter plots, and generating a scatter plot of color gamut visualization overlaid with the BT.2020 color gamut reference boundary based on these coordinates. Each effective pixel of the original HDR image to be evaluated is treated as a scatter point, and the chromaticity coordinates of each scatter point are calculated based on the CIE tristimulus value column vector. Retrieve the CIExy coordinates corresponding to the red, green, and blue vertices specified in the ITU-R BT.2020 standard, and connect them in the order of red vertex to green vertex, green vertex to blue vertex, and blue vertex back to red vertex to form a closed triangle outline. Set the closed triangle outline to a red dashed line style as a visual reference benchmark. Scattered points are plotted based on the chromaticity coordinates of all valid pixels as planar positions, and each scatter point is filled with a gradient color based on the normalized luminance component. The scatter point layer is then set below the visual reference datum to obtain a color gamut visualization scatter plot.

8. A large-model-based HDR image quality assessment system, characterized in that, include: The luminance-chrominance separation module is used to acquire the original HDR image to be evaluated, convert the original HDR image to a perceptually uniform color space, and separate the luminance component and chrominance component. The multi-exposure decomposition module is used to select multiple sets of exposure gains based on the normalized histogram probability distribution and Shannon information entropy of the luminance component to perform tone mapping processing on the luminance component, thereby generating a multi-exposure SDR image sequence that fully covers the HDR dynamic range. The color gamut visualization module is used to calculate the scatter chromaticity coordinates of the pixels of the original HDR image to be evaluated, and generate a color gamut visualization scatter plot superimposed with the BT.2020 color gamut reference boundary based on the scatter chromaticity coordinates. The inference module is used to encode the multi-exposure SDR image sequence, color gamut visualization scatter plot, and HDR standard into multimodal input prompts, and input the multimodal input prompts into a pre-trained multimodal large model to obtain the quality assessment results of the original HDR image to be evaluated. The output module is used to generate a structured evaluation report based on the quality assessment results of the original HDR image to be evaluated.

9. An electronic device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, wherein the computer program, when executed by the at least one processor, implements the large model-based HDR image quality evaluation method as described in any one of claims 1-7.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the large-model-based HDR image quality evaluation method as described in any one of claims 1-7.

Images