A large-format mural hyperspectral image visualization method and system

Abstract

The present application relates to the technical field of image processing and digital visualization, and discloses a large-format mural hyperspectral image visualization method and system. The method adopts a spectral-to-RGB conversion method based on the CIE colorimetric system, combines a color card-driven adaptive color correction mechanism, and uses image pyramid grading dynamic rendering technology, thereby achieving significant progress in the visualization of large-format mural hyperspectral data, and achieving the comprehensive effect of accurate color restoration, smooth image loading, and good interactive experience. Compared with the traditional pseudo-color mapping and static image browsing scheme, the present application saves the time for manual parameter adjustment and the cost of professional equipment, improves the color reconstruction accuracy and system operation efficiency, and provides a standardized and automated solution for high-fidelity display and digital protection of cultural relic hyperspectral images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and digital visualization, and particularly relates to a large-format mural hyperspectral image visualization method and system. BACKGROUND

[0002] Hyperspectral imaging (HSI) technology can obtain information of an object in both spatial and spectral dimensions, and is widely used in digital protection of cultural relics due to its non-destructive and accurate characteristics. However, there are two technical bottlenecks in the visualization of current large-format mural hyperspectral data:

[0003] Inaccurate color restoration: traditional methods mostly use subjective selection of three primary colors to construct pseudo-color images, lack of theoretical support for spectral-to-color conversion, resulting in color distortion and poor accuracy of reconstructed images;

[0004] Visualization performance bottleneck: the mural has a huge format (often up to billions of pixels), and existing image display and interaction systems cannot achieve smooth rendering and scaling on ordinary devices, hindering the application and promotion of digital achievements.

[0005] The above problems seriously restrict the promotion and popularization of hyperspectral images in the field of cultural heritage visualization. It can be seen that, in the prior art, it is difficult to perform efficient and accurate color mapping and visualization processing on large-format murals. SUMMARY

[0006] The present application provides a large-format mural hyperspectral image visualization method and system, which solves the problem that it is difficult to perform efficient and accurate color mapping and visualization processing on large-format murals in the prior art.

[0007] In order to achieve the above purpose, the present application realizes the technical scheme as follows:

[0008] In a first aspect, the present application provides a large-format mural hyperspectral image visualization method, comprising:

[0009] S1: reading hyperspectral data of a mural and performing preprocessing;

[0010] S2: converting pixel reflectance spectra of the preprocessed hyperspectral data into CIEXYZ chrominance values;

[0011] S3: performing RGB image generation and color adaptation processing based on the CIEXYZ chrominance values, and performing standard gamma correction on the processing result to generate an image displayable by a device;

[0012] S4: performing color correction on the image;

[0013] S5: performing image pyramid generation and dynamic loading based on the result of color correction to obtain a final visualization image.

[0014] In a second aspect, the application provides a large-format mural hyperspectral image visualization system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method of the first aspect.

[0015] Advantages:

[0016] The application provides a large-format mural hyperspectral image visualization method, which adopts a spectral-to-RGB conversion method based on the CIE colorimetric system, combines a color card-driven adaptive color correction mechanism, and uses an image pyramid hierarchical dynamic rendering technology, thereby achieving significant progress in large-format mural hyperspectral data visualization, and achieving the comprehensive effects of accurate color restoration, smooth image loading, and good interactive experience. Compared with the traditional pseudo-color mapping and static image browsing scheme, the application saves the time for manual parameter adjustment and the cost of professional equipment, improves the color reconstruction accuracy and system running efficiency, and provides a standardized and automated solution for high-fidelity display and digital protection of cultural relic hyperspectral images. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is one of the flowcharts of a large-format mural hyperspectral image visualization method according to an embodiment of the application;

[0018] Figure 2 is another flowchart of a large-format mural hyperspectral image visualization method according to an embodiment of the application;

[0019] Figure 3 is a calculation flowchart of hyperspectral data to RGB image according to an embodiment of the application;

[0020] Figure 4 is an adaptive color correction flowchart according to an embodiment of the application. DETAILED DESCRIPTION

[0021] The technical solutions of the application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0022] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the meanings as understood by a person of ordinary skill in the art to which the present application pertains. The terms "first", "second", and similar terms used in the present application do not denote any order, quantity, or importance, but are used to distinguish different components. Similarly, the terms "one" or "a" or similar terms do not denote a quantity restriction, but mean at least one.

[0023] It should be understood that the hyperspectral image visualization method for large-format murals in the embodiments of the present application can be applied to the processing of hyperspectral images of cultural relics such as large-format murals. Here, only examples are given, and no limitation is made.

[0024] Please refer to Figure 1 The present application provides a hyperspectral image visualization method for large-format murals, comprising:

[0025] S1: reading hyperspectral data of a mural and performing preprocessing;

[0026] S2: converting the pixel reflectance spectrum of the preprocessed hyperspectral data into CIEXYZ chrominance values;

[0027] S3: generating an RGB image and performing color adaptation processing based on the CIEXYZ chrominance values, and performing standard gamma correction on the processing result to generate an image displayable by a device;

[0028] S4: performing color correction on the image;

[0029] S5: generating an image pyramid and dynamically loading to obtain a final visualization image based on the results of color correction.

[0030] The hyperspectral image visualization method for large-format murals described above uses a spectral-to-RGB conversion method based on the CIE color system, an adaptive color correction mechanism combined with a color card driver, and an image pyramid grading dynamic rendering technology, which has made significant progress in the visualization of large-format mural hyperspectral data, and achieved a comprehensive effect of accurate color restoration, smooth image loading, and good interactive experience. Compared with the traditional pseudo-color mapping and static image browsing scheme, the present application saves the time for manual parameter adjustment and the cost of professional equipment, improves the color reconstruction accuracy and system running efficiency, and provides a standardized and automated solution for high-fidelity display and digital protection of cultural relic hyperspectral images.

[0031] In the following, as Figure 2 shown, the steps of the hyperspectral image visualization method for large-format murals described above are described in detail with a complete example:

[0032] Step 1, hyperspectral data reading and preprocessing:

[0033] In this application, multi-thread I / O and parallel computing technology are adopted to realize HSI data block loading, radiation correction and band resampling, and to unify the band resolution required by the standard colorimetric system.

[0034] Step 2, CIEXYZ chromaticity value calculation:

[0035] Based on the light source (such as D65, A light source, etc.) and the observer's viewing angle (2° or 10°), the built-in CIE standard color matching function (CMF) and light source spectral power distribution (SPD) data are called to calculate the CIEXYZ value of each pixel using the following formula:

[0036] ;

[0037] Wherein, is the reflectivity spectrum of HSI data, is the light source SPD, , , is the CMF function, is the normalization coefficient, is the spectral interval, and (X, Y, Z) is the CIEXYZ value of the pixel.

[0038] Step 3, color adaptation and RGB conversion:

[0039] As shown in Figure 3 , since the actual acquisition light source may not be consistent with the standard light source used by the target RGB space (such as sRGB), color adaptation conversion and color space mapping are required. Use the color adaptation matrix to convert XYZ from the source light source (such as A light source) to the target light source (such as D65), and the calculation formula is:

[0040] ;

[0041] Wherein , , and , , are the XYZ chromaticity values under the target light source and the source light source respectively, is the color adaptation transformation matrix, , , are the color adaptation coefficients of the light source.

[0042] Then map XYZ to RGB through the RGB conversion matrix, and the formula is as follows:

[0043] ;

[0044] In the formula, M is the conversion matrix for calculating the RGB value from the XYZ chromaticity value, 、 、 Linear RGB values.

[0045] Convert linear RGB values to non-linear display RGB, such as gamma correction rules under sRGB standard:

[0046] ;

[0047] ;

[0048] .

[0049] Step 4, color correction:

[0050] To solve the color cast problem in complex shooting environments (such as mixed light sources, low illumination), to improve the color accuracy of the image, a "fit-application" two-stage color correction mechanism based on color cards is adopted, as shown in Figure 4 .

[0051] High-precision color correction (CCM) matrix solving is performed in the fitting stage, and the specific steps are as follows:

[0052] Eliminate the non-linear response of the camera sensor through the inverse gamma curve, and restore the true physical light relationship. Automatically detect the color card color block in the image and extract the color value, and give higher optimization weight to the color block that is easy to distort the color gamut (such as high saturation red / blue color block):

[0053] ;

[0054] Where is the initial color difference of the ith color block.

[0055] Solve the CCM matrix by minimizing the weighted color difference, and the optimization target is as follows:

[0056] ;

[0057] Where, is the color difference, is the color correction matrix, are the red, green, and blue color values in the RGB color space, is the color value of the standard color card in the Lab color space.

[0058] The mixed optimization strategy is adopted in the optimization, CMA-ES (covariance matrix adaptation evolution strategy) global search and L-BFGS-B (quasi-Newton method with bound constraints) local optimization are used. Real-time closed-loop verification is carried out in the optimization process, the CCM is applied to the color card data, the color difference is calculated in real time, the weight and the optimization parameter are iteratively adjusted to the minimum color difference, and finally the optimal CCM matrix and linearization parameter are obtained.

[0059] In the application stage, image dynamic correction is carried out, and the specific steps are as follows:

[0060] The linearization parameter in the fitting stage is used to perform inverse gamma correction linearization processing on the RGB image to be corrected, and the CCM matrix obtained in the fitting stage is used to correct the color deviation:

[0061] ;

[0062] In the formula, 、 、 The red, green and blue color channel values of the pixel after correction by the CCM matrix.

[0063] Gamma correction is applied again to output the color corrected RGB image, and a color correction configuration file (containing light source information, CCM matrix, gamma parameter, etc.) is output synchronously, which can be used repeatedly for subsequent images (which can be images not shot with color cards).

[0064] Step 4, image pyramid generation and dynamic loading:

[0065] In order to realize real-time browsing and efficient interaction of large-format images on conventional devices, the present application adopts the Deep Zoom Image (DZI) format to construct the image pyramid structure.

[0066] The slice size (such as 256x256 pixels) and the overlapping pixels (such as 8 pixels) are set, and the interpolation algorithm (such as bicubic interpolation) is used for layered reconstruction, and the corrected RGB image obtained in the previous step is generated into multi-resolution images according to the scaling factor:

[0067] Level 0: original resolution (such as 100,000x50,000 pixels);

[0068] Level N: scaled to resolution.

[0069] Using viewport area-based request scheduling, loading image slices on demand, building an adaptive rendering engine, integrating coordinate mapping and image buffering mechanism, supporting sub-pixel level detail positioning, the rendering response delay is less than 0.5 seconds. This step realizes the "second-level loading + real-time scaling + high-precision interaction" of the image of the order of 100 million pixels, significantly reduces the load of the terminal device, and improves the usability and display effect of the hyperspectral image.

[0070] The embodiment of the present application further provides a large-format mural hyperspectral image visualization system, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps corresponding to the above method when executing the computer program.

[0071] The large-format mural hyperspectral image visualization system described above can implement each embodiment of the large-format mural hyperspectral image visualization method described above and achieve the same beneficial effects, which will not be described here.

[0072] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the existing technology according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A method for visualization of large-format mural hyperspectral images, characterized by, The method comprises the following steps: S1: reading hyperspectral data of a mural and preprocessing the hyperspectral data; S2: converting pixel reflectance spectra of the preprocessed hyperspectral data into CIEXYZ chromaticity values; S3: generating an RGB image and performing color adaptation based on the CIEXYZ chromaticity values, and performing standard gamma correction on the processing result to generate an image displayable by equipment; S4: performing color correction on the image; S5: generating an image pyramid and dynamically loading to obtain a final visualized image based on the result of color correction; The S4 comprises: A color correction mechanism based on a fitting-application two-stage color correction mechanism is used for color correction, comprising the following steps: The following steps are performed in the fitting stage: Eliminate the nonlinear response of the camera sensor by inverse gamma curve, automatically detect the color card color block in the image and extract the color value, give higher optimization weight to the color gamut prone to distortion , satisfy the following relationship: ； In the formula, is the initial color difference of the i-th color block; Optimization is performed by minimizing a weighted color difference sum to solve a CCM matrix, wherein the optimization target is as follows: ； wherein is the color difference, is the color correction matrix, are the red, green, and blue color values in the RGB color space, respectively, is the color value of the standard color card in the Lab color space; In the optimization, a hybrid optimization strategy is used, a covariance matrix self-adaptive evolutionary strategy is used for global search, a quasi-Newton method with boundary constraints is used for local optimization, real-time closed-loop verification is performed in the optimization process, the CCM matrix is applied to the color card data, the color difference is calculated in real time, the weight and the optimization parameter are iteratively adjusted based on the optimization target to the minimum color difference, and finally the optimal CCM matrix and linearization parameter are obtained; The following steps are performed in the application stage: The linearization parameter of the fitting stage is used to perform inverse gamma correction linearization processing on the RGB image to be corrected, the CCM matrix obtained in the fitting stage is used to correct color deviation, and the following relationship is satisfied: ； In the formula, , , are the red, green and blue color channel values of the pixel after correction by the CCM matrix. Gamma correction is applied again to output the color-corrected RGB image.

2. The method of visualization of large-format mural hyperspectral images according to claim 1, characterized in that, The S1 comprises: Multi-thread I / O and parallel computing technology are used for block loading, radiation correction and band resampling processing of the hyperspectral data, and the processing result is unified to the band resolution required by the standard colorimetric system to obtain the preprocessed hyperspectral data.

3. The method of visualization of large-format mural hyperspectral images according to claim 1, characterized in that, The S2 comprises: Based on the light source and the observer's perspective, the CIE standard color matching function and the light source spectral power distribution data are called to calculate the CIEXYZ value of each pixel, and the following relationship is satisfied: ； wherein is the hyperspectral data reflectance spectrum, is the light source spectral power distribution data, , , is the matching function, is the normalization coefficient, is the spectral interval, (X, Y, Z) is the pixel's CIE XYZ values.

4. The method of visualization of large-format mural hyperspectral images according to claim 1, characterized in that, The S3 comprises: The XYZ is converted from the source light source to the target light source using the color adaptation matrix, and the following relationship is satisfied: ； wherein , , and , , are the XYZ tristimulus values under the target light source and the source light source, respectively, is the color adaptation transform matrix, , , are the color adaptation coefficients of the target light source, , , are the color adaptation coefficients of the source light source; The XYZ under the target light source is mapped to RGB through an RGB conversion matrix, and the following relationship is satisfied: ； where M is a conversion matrix from XYZ colorimetric values to RGB values, , , are linear RGB values; The linear RGB value is converted into a non-linear display RGB, and the gamma correction under the sRGB standard is performed, and the rules are as follows: ； ； 。 5. The method of visualization of large-format mural hyperspectral images according to claim 1, characterized in that, The method further comprises: when the color-corrected RGB image is output, a color correction configuration file is synchronously output, and the color correction configuration file contains light source information, a CCM matrix and gamma parameters.

6. The method of visualization of large-format mural hyperspectral images according to claim 1, characterized in that, The S5 comprises: An image pyramid structure is constructed using a depth scaling image DZI format, the slice size and the overlapping pixels are set, an interpolation algorithm is used for layered reconstruction, and the color-corrected RGB image is generated into multi-level resolution images according to the scaling factor; A viewport area-based request scheduling is used to load image slices on demand, an adaptive rendering engine is constructed, rendering is performed using a coordinate mapping and image buffering mechanism, and a final visualized image is obtained.

7. A large-format mural hyperspectral image visualization system comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 6.

Images