Method and apparatus for recovering and reconstructing a light source spectrum based on a hyperspectral image

The method uses Gaussian convolution and dictionary-based decomposition to quickly and accurately reconstruct light source spectra in hyperspectral images, addressing computational and scene limitations of existing methods and enhancing hyperspectral imaging applications.

JP7590011B2Active Publication Date: 2024-11-26SHEN ZHEN HYPERNANO OPTICS TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2022554342
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-09-08
Publication Date
2024-11-26
Estimated Expiration
2040-09-08

AI Technical Summary

Technical Problem

Current hyperspectral image analysis methods struggle to accurately reconstruct light source spectra in unknown environments due to computational intensity, time constraints, and limited application scenes, especially when there are few colors in the image.

Method used

A method involving Gaussian convolution, spectral enhancement, and dictionary-based decomposition to reconstruct light source spectra using hyperspectral images, utilizing a pre-stored light source spectrum dictionary and iterative approximation to achieve accurate reconstruction.

Benefits of technology

The method enables fast and accurate restoration of light source spectra, even in monochromatic or color-poor scenes, expanding the application scope of hyperspectral imaging and improving analysis efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007590011000023
    Figure 0007590011000023
  • Figure 0007590011000024
    Figure 0007590011000024
  • Figure 0007590011000025
    Figure 0007590011000025
Patent Text Reader

Abstract

This application discloses a method and apparatus for recovering and reconstructing a light source spectrum based on a hyperspectral image, which involves processing the hyperspectral image to obtain a first maximum spectrum, matching the first maximum spectrum with a set of light source spectrum basis vectors in a light source spectrum dictionary, linearly decomposing the first maximum spectrum to obtain a dictionary set, obtaining the maximum value for each wavelength range, and performing iterative approximation. Therefore, a simpler calculation process can be used to quickly recover the light source spectrum of a shooting environment for a single hyperspectral image, and the recovery effect for monochromatic or low-color image scenes is still high, thus approaching the actual light source spectrum. After the light source spectrum is obtained, this information can be further used for various applications. This eliminates the need to obtain light source spectrum information in advance or to measure the light source using additional devices and components, thereby shortening data collection time, improving analysis efficiency, greatly expanding the application scenarios of hyperspectral imaging, and improving application flexibility and possibilities.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present application relates to the field of hyperspectral analysis, and in particular to a method and apparatus for recovering and reconstructing a light source spectrum based on a hyperspectral image. [Background technology]

[0002] Hyperspectral imaging technology can simultaneously obtain image information and spectral information, and can be combined with machine vision technology to distinguish objects and also perform spectral analysis based on the spectrum, so hyperspectral imaging technology is a new technology with great potential. The spectral analysis function in hyperspectral imaging technology comes from hyperspectral images and can collect spectral information emitted by materials with different wavelengths, and these spectral information directly reflect the physical and chemical components of objects and other information. Combined with information such as image identification and area selection, hyperspectral imaging technology can realize the full automation of target detection-component judgment-result output.

[0003] At present, when extracting the spectral information of a material, the general hyperspectral image analysis method needs to know the light source spectrum information of the shooting environment in advance, but it cannot always be read directly from the image. In the screen, the object being shot partially reflects the light source information it carries based on its reflectance, but due to the difference in absorption of different wavelength ranges of each color, the reflected spectrum loses some wavelength range energy from the original light source to different degrees. When there are enough different colors, white areas, or regular reflection areas on the screen, the light source spectrum can be restored by certain calculations. However, these conditions are often not met or these areas cannot be easily found, and at this time, obtaining the light source spectrum from the image becomes an intractable problem.

[0004] In the open application scene of unknown light source, currently, there are several algorithm methods that can restore the light source spectrum by one hyperspectral photograph, such as the light source restoration method based on two-color model, or the light source restoration method based on sparse matrix. The main limitations of these methods are, firstly, that the amount of calculation is large, the requirements for computing power are high or it takes a long time, and it is not possible to realize immediacy; secondly, that the application scene is limited, and the light source spectrum cannot be accurately restored when the image has few kinds of colors, and generally, the image needs close to 10 kinds or more colors. These two problems are currently the biggest technical problems faced by the restoration of hyperspectral light source. Currently, the industry has not yet proposed a method based on hyperspectral or spectral analysis applied to multiple kinds of scenes that can accurately open and reconstruct or restore the light source spectrum in unknown light source scenes in real time.

[0005] In view of this, it is of great importance to design a method for quickly and accurately recovering and reconstructing the light source spectrum based on hyperspectral images. Summary of the Invention [Problem to be solved by the invention]

[0006] The above-mentioned conventional methods for obtaining light source spectrum have problems such as poor effect, high limitations, and limitations in application scenarios. To solve the above problems, the embodiments of the present application provide a method and apparatus for restoring and reconstructing light source spectrum based on hyperspectral image.

[0007] In a first aspect, embodiments of the present application provide a method for recovering and reconstructing a light source spectrum based on a hyperspectral image, the method comprising: Step S1: obtain a hyperspectral image, denoted as I(x,y,ω), where x, y and ω respectively denote the width, height and wavelength of the hyperspectral image; and perform Gaussian convolution on the hyperspectral image I(x,y,ω) to obtain a Gaussian blurred image B(x,y,ω), where B(x,y,ω)=I(x,y,ω)*g(x,y), where g(x,y) is a Gaussian convolution kernel matrix; Step S2: subtract the Gaussian blurred image B(x,y,ω) from the hyperspectral image I(x,y,ω) to obtain the enhanced image E(x,y,ω), where E(x,y,ω)=I(x,y,ω)-B(x,y,ω); The maximum value of each wavelength range in the enhanced image E(x, y, ω) is obtained, and the first maximum spectrum R(ω) after visual enhancement is obtained. R(ω)=max x,y (E(x,y,ω)), where max x,y Step S3 showing that (E(x,y,ω)) searches for the maximum value in the x and y dimensions for E(x,y,ω); Step S4: searching and matching the first maximum spectrum R(ω) with a light source spectrum basis vector set D(n, m, ω) in a pre-stored light source spectrum dictionary to obtain a potential light source spectrum vector set P(k, ω) after decomposition and merging, where n is the vector number in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n, m, ω), m is the group number of a related vector group consisting of related basis vectors in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n, m, ω), and k is the vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k, ω) after matching and merging; The first maximum spectrum R(ω) is projected onto the potential light source spectrum vector set P(k,ω) to obtain the intensity vector b(k) of each basis vector, and the formula L 0 Based on (ω)=P(k,ω)*b(k), the initial predicted light source spectrum L 0 (ω) and L 0Step S5 of obtaining a second maximum spectrum R'(ω) by taking the maximum values ​​in each wavelength range of both R(ω) and R(ω); Step S5 of replacing the first maximum spectrum R(ω) with the second maximum spectrum R'(ω) is repeated until the color temperature change value of the second maximum spectrum R'(ω) becomes smaller than a predetermined threshold value in step S6; The initial predicted light source spectrum L just obtained 0 and step S7 of normalizing (ω) to obtain a restored reconstructed light source spectrum L(ω).

[0008] In some embodiments, the step of constructing the illuminant spectrum dictionary comprises: Step S8 of obtaining general light source spectrum information, and extracting waveform vectors having the same wavelength range and wavelength range interval from the light source spectrum information to form a first waveform vector set; Step S9: extracting complete peak-trough information of the waveform of the light source of unspecified spectrum or color temperature, and extracting waveform vectors of the same wavelength range and wavelength range interval from the peak-trough information to form a second waveform vector set; and step S10 of marking each waveform vector belonging to the same light source in the first waveform vector set and the second waveform vector set as a simultaneously related and coexisting basis vector sequence to construct an index to form a light source spectrum dictionary, the light source spectrum dictionary consisting of a light source spectrum basis vector set D(n, m, ω).

[0009] In constructing and using the light source spectrum dictionary, a dynamic color temperature variable basis vector combination method is used, so that the calculated light source spectrum has higher flexibility and adaptability, and is closer to the actual light source spectrum.

[0010] In some embodiments, in step S9, a multi-Gaussian peak fitting method is used to separate and extract each complete peak-trough information of the waveform of the light source with an unspecified spectrum or color temperature, which has high extraction efficiency and low computational complexity.

[0011] In some embodiments, step S4 specifically includes: S41, performing linear regression of the first maximum spectrum R(ω) on the light source spectrum basis vector set D(n, m, ω) in the light source spectrum dictionary, and combining them according to relevance to form a new light source spectrum basis vector set D'(k', m', ω), where k' is the vector number in the light source spectrum dictionary of the new light source spectrum basis vector set D'(k', m', ω), and m' is the group number of the related vector group in the light source spectrum dictionary of the new light source spectrum basis vector set D'(k', m', ω); and merging and adding all other basis vectors with the same group number as m' in the light source spectrum dictionary to the light source spectrum basis vector set D'(k',m',ω) obtained by decomposition to obtain a potential light source spectrum vector set P(k,ω), where k is the vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k,ω).

[0012] The first maximum spectrum directly obtained from the hyperspectral image is matched and searched in the light source spectrum dictionary using the light source spectrum dictionary construction method, so as to obtain a more accurate spectrum or a spectrum closer to the actual light source.

[0013] In some embodiments, the higher-ranked basis vectors arranged in descending order of relevance obtained in the new light source spectrum basis vector set D'(k',m',ω) are at most two or three kinds of light source components, which is easy to calculate and can improve the calculation efficiency.

[0014] In some embodiments, the linear decomposition includes a least squares linear decomposition, which computes the most relevant illuminant spectral basis vectors.

[0015] In some embodiments, the formula for the two-dimensional Gaussian kernel used in the Gaussian convolution process is:

number

[0016] A blurred hyperspectral image is obtained by Gaussian convolution processing, and an enhanced image is obtained by subtracting the Gaussian blurred image from the original image, making the processed image more similar to the perception of the human eye.

[0017] In some embodiments, g(x,y) is replaced by an i×i numeric matrix, which is easier to compute in practical applications.

[0018] In some embodiments, the color temperature change value of the second maximum spectrum R'(ω)

number

number

number

number

[0019] In a second aspect, embodiments of the present application further provide an apparatus for recovering and reconstructing a light source spectrum based on a hyperspectral image, comprising: an image blurring module for obtaining a hyperspectral image, denoted as I(x,y,ω), where x, y and ω respectively denote the width, height and wavelength of the hyperspectral image, and performing Gaussian convolution on the hyperspectral image I(x,y,ω) to obtain a Gaussian blurred image B(x,y,ω), where B(x,y,ω)=I(x,y,ω)*g(x,y), where g(x,y) is a Gaussian convolution kernel matrix; an image enhancement module, which is used to subtract the Gaussian blurred image B(x,y,ω) from the hyperspectral image I(x,y,ω) to obtain an enhanced image E(x,y,ω), where E(x,y,ω)=I(x,y,ω)-B(x,y,ω); The maximum value of each wavelength range in the enhanced image E(x, y, ω) is obtained to obtain the first maximum spectrum R(ω) after visual enhancement, where R(ω)=max x,y (E(x,y,ω)), where max x,y a first maximum spectrum acquisition module indicating that (E(x,y,ω)) searches for the maximum in the x and y dimensions for E(x,y,ω); a spectrum matching module for searching and matching the first maximum spectrum R(ω) with a light source spectrum basis vector set D(n,m,ω) in a pre-stored light source spectrum dictionary to obtain a potential light source spectrum vector set P(k,ω) after decomposition and merging, where n is the vector number in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n,m,ω), m is the group number of a related vector group consisting of related basis vectors in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n,m,ω), and k is the vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k,ω) after matching and merging; The first maximum spectrum R(ω) is projected onto the potential light source spectrum vector set P(k,ω) to obtain the intensity vector b(k) of each basis vector, and the formula L 0Based on (ω)=P(k,ω)*b(k), the initial predicted light source spectrum L 0 (ω) and L 0 a light source spectrum initial prediction module, which is used to obtain a second maximum spectrum R'(ω) by taking the maximum value in each wavelength range of both R(ω) and R(ω); a spectrum iteration module, which is used for repeatedly executing the light source spectrum initial prediction module by replacing the first maximum spectrum R(ω) with the second maximum spectrum R'(ω) until a color temperature change value of the second maximum spectrum R'(ω) is smaller than a predetermined threshold; The initial predicted light source spectrum L just obtained 0 and a normalization module that is used to normalize (ω) to obtain a reconstructed light source spectrum L(ω).

[0020] In a third aspect, an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method according to any one of the implementations of the first aspect.

[0021] This application discloses a method and device for restoring and reconstructing a light source spectrum based on a hyperspectral image, which can quickly and accurately restore the light source spectrum of the shooting environment using a simpler calculation process in one hyperspectral image, and the restoration effect for monochromatic or less colored image scenes is still high, and thus approaches the actual light source spectrum. After obtaining the light source spectrum, the information can be further used for different applications, such as extracting and analyzing the reflection spectrum and absorption spectrum of a material, and further analyzing the material components, which does not require the pre-obtaining of light source spectrum information or the measurement of the light source by additional devices and components, thereby shortening the data collection time, improving the analysis efficiency, greatly expanding the application scenarios of hyperspectral imaging, and improving the application flexibility and possibilities. [Brief description of the drawings]

[0022] The drawings are included to provide a further understanding of the embodiments, and are incorporated in and made a part of this specification. The drawings illustrate the embodiments, and together with the description serve to explain the principles of the present application. Other embodiments and many of the expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Elements of the drawings are not necessarily drawn to scale. Like reference numerals indicate corresponding like parts. [Figure 1] FIG. 1 is a block diagram of an exemplary device to which an embodiment of the present application is applied. [Diagram 2] 1 is a schematic flowchart of a method for recovering and reconstructing a light source spectrum based on a hyperspectral image in an embodiment of the present application. [Diagram 3] 1 is a schematic flowchart of a step of constructing an illuminant spectrum dictionary in a method for restoring and reconstructing an illuminant spectrum based on a hyperspectral image in an embodiment of the present application. [Figure 4] 1 is a flowchart of step S4 of a method for recovering and reconstructing a light source spectrum based on a hyperspectral image in an embodiment of the present application. [Diagram 5] FIG. 1 is a schematic diagram of an apparatus for restoring and reconstructing a light source spectrum based on a hyperspectral image in an embodiment of the present application. [Figure 6] FIG. 2 is a structural schematic diagram of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in more detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments of the present application, any other embodiments that a person skilled in the art can obtain without inventive efforts fall within the scope of protection of the present application.

[0024] FIG. 1 shows an exemplary configuration 100 of a method for recovering and reconstructing a light source spectrum based on a hyperspectral image or an apparatus for recovering and reconstructing a light source spectrum based on a hyperspectral image to which an embodiment of the present application is applied.

[0025] 1, the device configuration 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0026] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104, such as to send and receive messages, etc. Various applications, such as data processing type applications, file processing type applications, etc., may be installed on terminal devices 101, 102, 103.

[0027] The terminal devices 101, 102, and 103 may be hardware or software. If the terminal devices 101, 102, and 103 are hardware, they may be various electronic devices, including but not limited to smartphones, tablet computers, laptops, and desktop computers. If the terminal devices 101, 102, and 103 are software, they may be installed in the electronic devices listed above. They may be realized in multiple software or software modules (e.g., software or software modules for providing distributed services), or in a single software or software module. No specific limitations are provided here.

[0028] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, 103. The background data processing server may process the acquired files or data to generate a processing result.

[0029] In addition, the method for restoring and reconstructing a light source spectrum based on a hyperspectral image according to an embodiment of the present application may be executed by the server 105 or by the terminal devices 101, 102, and 103, and correspondingly, the device for restoring and reconstructing a light source spectrum based on a hyperspectral image may be installed in the server 105 or in the terminal devices 101, 102, and 103.

[0030] It should be understood that the number of terminal devices, networks and servers in Fig. 1 are merely schematic. Any number of terminal devices, networks and servers may be included according to the needs of the implementation. If the processed data does not need to be obtained remotely, the above configuration of the devices may not include a network and only require a server or terminal device.

[0031] As shown in FIG. 2 , an embodiment of the present application provides a method for recovering and reconstructing a light source spectrum based on a hyperspectral image, Step S1: obtain a hyperspectral image, denoted as I(x,y,ω), where x, y and ω respectively denote the width, height and wavelength of the hyperspectral image; and perform Gaussian convolution on the hyperspectral image I(x,y,ω) to obtain a Gaussian blurred image B(x,y,ω), where B(x,y,ω)=I(x,y,ω)*g(x,y), where g(x,y) is a Gaussian convolution kernel matrix; Step S2: subtract the Gaussian blurred image B(x,y,ω) from the hyperspectral image I(x,y,ω) to obtain the enhanced image E(x,y,ω), where E(x,y,ω)=I(x,y,ω)-B(x,y,ω); The maximum value of each wavelength range in the enhanced image E(x, y, ω) is obtained, and the first maximum spectrum R(ω) after visual enhancement is obtained. R(ω)=max x,y (E(x,y,ω)), where max x,y Step S3 showing that (E(x,y,ω)) searches for the maximum value in the x and y dimensions for E(x,y,ω); Step S4: searching and matching the first maximum spectrum R(ω) with a light source spectrum basis vector set D(n, m, ω) in a pre-stored light source spectrum dictionary to obtain a potential light source spectrum vector set P(k, ω) after decomposition and merging, where n is the vector number in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n, m, ω), m is the group number of a related vector group consisting of related basis vectors in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n, m, ω), and k is the vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k, ω) after matching and merging; The first maximum spectrum R(ω) is projected onto the potential light source spectrum vector set P(k,ω) to obtain the intensity vector b(k) of each basis vector, and the formula L 0 Obtain the initial predicted light source spectrum L0(ω) based on (ω) = P(k,ω) * b(k), and 0 Step S5 of obtaining a second maximum spectrum R'(ω) by taking the maximum values ​​in each wavelength range of both R(ω) and R(ω); Step S6 repeats step S5 by substituting the second maximum spectrum R'(ω) for the first maximum spectrum R(ω) until the color temperature change value of the second maximum spectrum R'(ω) becomes smaller than a predetermined threshold value; The initial predicted light source spectrum L just obtained 0 and step S7 of normalizing (ω) to obtain a restored reconstructed light source spectrum L(ω).

[0032] In a specific embodiment, as shown in FIG. 3, the step of constructing the light source spectrum dictionary is specifically as follows: Step S8 of obtaining general light source spectrum information, and extracting waveform vectors having the same wavelength range and wavelength range interval from the light source spectrum information to form a first waveform vector set; Step S9: extracting complete peak-trough information of the waveform of the light source of unspecified spectrum or color temperature, and extracting waveform vectors of the same wavelength range and wavelength range interval from the peak-trough information to form a second waveform vector set; and step S10 of marking each waveform vector belonging to the same light source in the first waveform vector set and the second waveform vector set as a simultaneously related and coexisting basis vector sequence to construct an index to form a light source spectrum dictionary, the light source spectrum dictionary consisting of a light source spectrum basis vector set D(n, m, ω).

[0033] The common representative light sources acquired in step S8 include sunlight, black body radiation curve or incandescent lamp with different color temperatures, different representative fluorescent light spectrum sequences F2-F12, and the current more common LED light source. For some artificial light sources with unspecified spectrum or color temperature, such as LED, the color temperature and spectrum will drift due to the change in the luminous intensity of the excitation chip and phosphor. Therefore, in a specific embodiment, in step S9, each complete peak-trough information of the waveform of the light source with unspecified spectrum or color temperature is separated and extracted by a multi-Gaussian peak fitting method. Each complete peak-trough information is treated as an independent basis vector to extract waveform vectors with the same wavelength range and wavelength range interval, thereby forming a second waveform vector set and adding it to the light source spectrum dictionary. In the construction and use of the light source spectrum dictionary, a basis vector combination method with dynamic color temperature change is used, so that the calculated light source spectrum has higher versatility and adaptability, and is closer to the actual light source spectrum. In addition, each waveform vector separated from the same light source is marked as a basis vector sequence that needs to be related and coexist at the same time. That is, when the search matching of the light source spectrum restoration algorithm generates a matching ratio involving one waveform of the set of bases, other waveforms in the same set of sequences are automatically added as potential light source spectrum matching basis vectors. When a new light source spectrum that does not exist in the light source spectrum dictionary is found, the process of adding new light source spectrum information to the light source spectrum dictionary according to steps S8 to S10 and building an index is repeated, where the index includes an index of each vector and an index of the associated vector sequence of each set. In other optional embodiments, the light source spectrum dictionary may be built in other ways. After building a more complete light source spectrum dictionary, the light source spectrum of the captured hyperspectral image can be restored, and steps S1 to S7 are carried out.

[0034] In a specific embodiment, the formula of the two-dimensional Gaussian kernel used in the Gaussian convolution process in step S1 is:

number

[0035] In step S3, the maximum value of each wavelength range in the enhanced image E(x, y, ω) is obtained to obtain the first maximum spectrum R(ω), the first maximum spectrum R(ω) is theoretically close to the light source spectrum in part, but due to changes in light intensity and object distance, and different object surfaces have different reflections and absorptions of light of different wavelengths, the shape of the first maximum spectrum R(ω) changes, and compared with the light source spectrum, the wavelength range waveform is concave and missing, and the fewer colors in the shooting screen, the more severe the missing. Therefore, in order to compensate for such missing, step S4 is used.

[0036] There are various methods for matching the acquired first maximum spectrum R(ω) with the light source spectrum dictionary, and the following description focuses on the specific method used in this embodiment. In a specific embodiment, as shown in FIG. 4, step S4 is specifically as follows: S41, performing linear regression of the first maximum spectrum R(ω) on the light source spectrum basis vector set D(n, m, ω) in the light source spectrum dictionary, and combining them according to relevance to form a new light source spectrum basis vector set D'(k', m', ω), where k' is the vector number in the light source spectrum dictionary of the new light source spectrum basis vector set D'(k', m', ω), and m' is the group number of the related vector group in the light source spectrum dictionary of the new light source spectrum basis vector set D'(k', m', ω); and merging and adding all other basis vectors with the same group number as m' in the light source spectrum dictionary to the light source spectrum basis vector set D'(k',m',ω) obtained by decomposition to obtain a potential light source spectrum vector set P(k,ω), where k is the vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k,ω).

[0037] In a specific embodiment, the top basis vectors arranged in descending order of relevance obtained in the most relevant illuminant spectrum basis vector set D'(k',m',ω) are at most two or three kinds of illuminant components. This is because this assumption can be satisfied in many natural or artificial light source scenes. The linear decomposition in step S41 includes least squares linear decomposition. Take the top three most relevant illuminant spectrum basis vector sets D'(k',m',ω) after calculating by least squares linear decomposition. After obtaining k' and m', add all other (i.e., the top three unrelated) basis vectors in the same set as m' to the potential illuminant spectrum basis vector set to obtain a new potential illuminant spectrum vector set P(k,ω), where k is the number in the pre-constructed illuminant spectrum dictionary of the potential illuminant spectrum vector set P(k,ω).

[0038] In a specific embodiment, in step S6, the color temperature change value of the second maximum spectrum R'(ω) is calculated.

number

number

number

number

number

[0039] Further referring to FIG. 5, as a realization of the methods shown in the above figures, the present application provides an embodiment of an apparatus for recovering and reconstructing a light source spectrum based on a hyperspectral image, which embodiment of the apparatus corresponds to the embodiment of the method shown in FIG. 2, and which can be specifically applied to various electronic devices.

[0040] As shown in FIG. 5, an embodiment of the present application further provides an apparatus for recovering and reconstructing a light source spectrum based on a hyperspectral image, an image blurring module 1, which obtains a hyperspectral image, denoted as I(x,y,ω), where x, y and ω respectively represent the width, height and wavelength of the hyperspectral image, and performs Gaussian convolution on the hyperspectral image I(x,y,ω) to obtain a Gaussian blurred image B(x,y,ω), where B(x,y,ω)=I(x,y,ω)*g(x,y), where g(x,y) is a Gaussian convolution kernel matrix; an image enhancement module 2, which is used to subtract the Gaussian blurred image B(x,y,ω) from the hyperspectral image I(x,y,ω) to obtain an enhanced image E(x,y,ω), where E(x,y,ω)=I(x,y,ω)-B(x,y,ω); The maximum value of each wavelength range in the enhanced image E(x, y, ω) is obtained to obtain the first maximum spectrum R(ω) after visual enhancement, where R(ω)=max x,y(E(x,y,ω)), where max x,y a first maximum spectrum acquisition module 3, which indicates that (E(x,y,ω)) searches for the maximum in the x and y dimensions for E(x,y,ω); a spectrum matching module 4 for searching and matching the first maximum spectrum R(ω) with a light source spectrum basis vector set D(n,m,ω) in a pre-stored light source spectrum dictionary to obtain a potential light source spectrum vector set P(k,ω) after decomposition and merging, where n is the vector number in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n,m,ω), m is the group number of a related vector group consisting of related basis vectors in the light source spectrum dictionary of each basis vector in the light source spectrum basis vector set D(n,m,ω), and k is the vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k,ω) after matching and merging; The first maximum spectrum R(ω) is projected onto the potential light source spectrum vector set P(k,ω) to obtain the intensity vector b(k) of each basis vector, and the formula L 0 Based on (ω)=P(k,ω)*b(k), the initial predicted light source spectrum L 0 (ω) and L 0 a light source spectrum initial prediction module 5, which is used to obtain a second maximum spectrum R'(ω) by taking the maximum value in each wavelength range of both R(ω) and R(ω); a spectrum iteration module 6, which is used for repeatedly executing the light source spectrum initial prediction module by replacing the first maximum spectrum R(ω) with the second maximum spectrum R'(ω) until the color temperature change value of the second maximum spectrum R'(ω) is smaller than a predetermined threshold; The initial predicted light source spectrum L just obtained 0 and a normalization module 7, which is used to normalize (ω) to obtain a reconstructed light source spectrum L(ω).

[0041] In a specific embodiment, the illuminant spectrum dictionary constructing device is specifically configured to: A light source spectrum information collecting module is used for obtaining general light source spectrum information, and extracting waveform vectors having the same wavelength range and wavelength range interval from the light source spectrum information to form a first set of waveform vectors; A peak-trough information extracting module is used for separating and extracting each complete peak-trough information of the waveform of a light source with an unspecified spectrum or color temperature, and extracting waveform vectors with the same wavelength range and wavelength range interval from the peak-trough information respectively to form a second waveform vector set; and a dictionary construction module which is used to mark each waveform vector belonging to the same light source in the first waveform vector set and the second waveform vector set as a basis vector sequence that is simultaneously related and coexisting to construct an index to form a light source spectrum dictionary, wherein the light source spectrum dictionary is composed of a light source spectrum basis vector set D(n, m, ω).

[0042] In a specific embodiment, the formula of the two-dimensional Gaussian kernel used in the Gaussian convolution process in the image blurring module 1 is:

number

[0043] In the first maximum spectrum acquisition module 3, the maximum value of each wavelength range in the enhanced image E(x,y,ω) is acquired to obtain the first maximum spectrum R(ω), the first maximum spectrum R(ω) is theoretically close to the light source spectrum in part, but due to changes in light intensity and object distance, and different object surfaces have different reflections and absorptions of light of different wavelengths, the shape of the first maximum spectrum R(ω) changes, and compared with the light source spectrum, the wavelength range waveform is concave and missing, and the fewer colors in the shooting screen, the more severe the missing. Therefore, a spectrum matching module 4 is used to compensate for such missing.

[0044] There are various methods for matching the acquired first maximum spectrum R(ω) with the light source spectrum dictionary, and the following description focuses on the specific method used in this embodiment. In this embodiment, the spectrum matching module 4 specifically: a linear decomposition module for linearly regressively decomposing the first maximum spectrum R(ω) in the light source spectrum basis vector set D(n,m,ω) in the light source spectrum dictionary, and combining them according to relevance to form a new light source spectrum basis vector set D'(k',m',ω), where k' is the vector number of the new light source spectrum basis vector set D'(k',m',ω) in the light source spectrum dictionary, and m' is the group number of the related vector group in the light source spectrum dictionary of the new light source spectrum basis vector set D'(k',m',ω); a dictionary set acquisition module, which is used to merge and add all other basis vectors with the same group number as m' in the light source spectrum dictionary to a new light source spectrum basis vector set D'(k',m',ω) obtained by linear decomposition to obtain a potential light source spectrum vector set P(k,ω), where k is the vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k,ω).

[0045] In a specific embodiment, the higher-level basis vectors arranged in descending order of relevance obtained in the most relevant illuminant spectrum basis vector D'(k',m',ω) are at most two or three kinds of illuminant components. This is because this assumption can be satisfied in many natural or artificial light source scenes. The linear decomposition in the linear decomposition module includes least squares linear decomposition. After calculating by least squares linear decomposition, the top three most relevant illuminant spectrum basis vector set D'(k',m',ω) is taken. After obtaining the top three most relevant k' and m', all other basis vectors in the same set as m' are merged into the extracted illuminant spectrum basis vector set D'(k',m',ω) to obtain a potential illuminant spectrum vector set P(k,ω), where k is the basis vector number in the illuminant spectrum dictionary of the potential illuminant spectrum vector set P(k,ω).

[0046] In a specific embodiment, in the spectrum repetition module 6, the color temperature change value of the second maximum spectrum R'(ω)

number

number

number

number

number

[0047] This application discloses a method and device for restoring and reconstructing a light source spectrum based on a hyperspectral image, which can quickly and accurately restore the light source spectrum of the shooting environment using a simpler calculation process in one hyperspectral image, and the restoration effect for monochromatic or less colored image scenes is still high, and thus approaches the actual light source spectrum. After obtaining the light source spectrum, the information can be further used for different applications, such as extracting and analyzing the reflection spectrum and absorption spectrum of a material, and further analyzing the material components, which does not require the pre-obtaining of light source spectrum information or the measurement of the light source by additional devices and components, thereby shortening the data collection time, improving the analysis efficiency, greatly expanding the application scenarios of hyperspectral imaging, and improving the application flexibility and possibilities.

[0048] 6 is a structural schematic diagram of a computer device 600 suitable for implementing an electronic device of an embodiment of the present application (e.g., the server or terminal device shown in FIG. 1). The electronic device shown in FIG. 6 is merely an example and does not limit the functions and scope of use of the embodiment of the present application.

[0049] 6, the computer device 600 includes a central processing unit (CPU) 601 and a graphic processor (GPU) 602, and can execute various appropriate operations and processes based on programs stored in a read-only memory (ROM) 603 or programs loaded from a storage unit 609 into a random access memory (RAM) 604. The RAM 604 further stores various programs and data required for operating the device 600. The CPU 601, the GPU 602, the ROM 603, and the RAM 604 are connected to each other via a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.

[0050] An input unit 607 including a keyboard, a mouse, etc., an output unit 608 including a liquid crystal display (LCD), etc. and a loudspeaker, etc., a storage unit 609 including a hard disk, etc., and a communication unit 610 including a network interface card such as a LAN card and a modem are connected to the I / O interface 606. The communication unit 610 executes communication processing via a network such as the Internet. A drive 611 may be connected to the I / O interface 606 as necessary. A removable medium 612, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 611 as necessary, and a computer program read from the drive 611 is thereby installed in the storage unit 609 as necessary.

[0051] In particular, according to an embodiment of the present disclosure, the above processes described with reference to the flowcharts may be implemented as a computer software program. For example, an embodiment of the present disclosure may include a computer program product, the computer program product may include a computer program carried on a computer-readable medium, the computer program including a program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed via a network by a communication unit 610 and / or installed from a removable medium 612. When the computer program is executed by a central processing unit (CPU) 601 and a graphics processor (GPU) 602, it performs the above-mentioned functions limited in the method of the present application.

[0052] It should be noted that the computer readable medium described herein may be a computer readable signal medium or a computer readable medium or any combination of both. The computer readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus or device, or any combination of the above. Further examples of computer readable medium may include, but are not limited to, an electrical connection having one or more conductors, a notebook computer magnetic disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer readable medium may be any tangible medium that contains a program or on which a program is stored, and the program may be used by or in combination with an instruction execution apparatus, apparatus or device. And, in the present application, a computer readable signal medium may include a data signal propagating in baseband or as part of a carrier, the data signal carrying the computer readable program code. Such a propagated data signal may take many forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium, which may transmit, propagate, or transmit a program for use by or in connection with an instruction execution apparatus, apparatus, or device. The program code contained in the computer-readable medium may be transmitted by any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0053] Computer program code for carrying out the operations of the present application may be compiled in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and even conventional procedural programming languages ​​such as "C" or similar. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected via the Internet using an Internet Service Provider).

[0054] The flowcharts and block diagrams in the drawings illustrate possible system configurations, functions, and operations of the apparatus, method, and computer program product according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or part of code, which includes one or more executable instructions for implementing a specified logical function. It should be noted that in some alternative implementations, the functions depicted in the blocks may occur in a different order than the order depicted in the drawings. For example, two blocks shown in succession may actually be essentially executed in parallel, or may be executed in the reverse order, depending on the functions involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be realized by a dedicated hardware-based device that performs the specified functions or operations, or may be realized by a combination of dedicated hardware and computer instructions.

[0055] The modules related to the embodiments of the present application may be implemented in the form of software or hardware. The modules described may be installed in a processor.

[0056] The above description is merely a description of the preferred embodiment and the technical principles used in the present application. As will be understood by those skilled in the art, the scope of the present application is not limited to the technical solution consisting of the specific combination of the above technical features, but also includes other technical solutions that are formed by any combination of the above technical features or their equivalent features without departing from the concept of the present application. For example, it also includes technical solutions that are formed by substituting the above features with technical features having similar functions (including but not limited to those disclosed in the present application) with each other.

Claims

1. A method for recovering and reconstructing a light source spectrum based on a hyperspectral image, comprising: a step S1 of acquiring a hyperspectral image, recording a value at a certain wavelength ω at coordinates (x, y) of a point on an x-y plane formed by a width x direction and a height y direction in the hyperspectral image having a predetermined width and height as I(x, y, ω), and performing Gaussian convolution processing on I(x, y, ω) at all wavelengths with a Gaussian convolution kernel matrix g(x, y) to acquire a value B(x, y, ω) after Gaussian convolution processing corresponding to I(x, y, ω) at a certain wavelength ω; Step S2: subtract B(x,y,ω) from I(x,y,ω) corresponding to all wavelengths to obtain E(x,y,ω), where E(x,y,ω)=I(x,y,ω)-B(x,y,ω); The maximum value of each wavelength in E(x, y, ω) is obtained, and a first maximum spectrum R(ω) after visual enhancement is obtained, where R(ω)=max x,y (E(x, y, ω)), where max x,y Step S3, denoting that (E(x,y,ω)) searches for the maximum value in the x and y directions for E(x,y,ω); Step S4 of linearly regressing the first maximum spectrum R(ω) on a light source spectrum basis vector set D(n, m, ω) in a light source spectrum dictionary and combining them according to relevance to form a new light source spectrum basis vector set D'(k', m', ω), where k' is a vector number in the light source spectrum dictionary of the new light source spectrum basis vector set D'(k', m', ω), and m' is a group number of a related vector group in the light source spectrum dictionary of the new light source spectrum basis vector set D'(k', m', ω), and merging and adding all other basis vectors with the same group number as m' in the light source spectrum dictionary to the new light source spectrum basis vector set D'(k', m', ω) to obtain a potential light source spectrum vector set P(k, ω), where k is a vector number in the light source spectrum dictionary of each basis vector in the potential light source spectrum vector set P(k, ω); The first maximum spectrum R(ω) is projected onto the potential light source spectrum vector set P(k,ω) to obtain an intensity vector b(k) of each basis vector, and the potential light source spectrum vector set P(k,ω) at all wavelengths is convolved with the intensity vector b(k) of each basis vector to obtain an initial predicted light source spectrum L 0 (ω) is obtained, and L 0 Step S5 of taking the maximum value at each wavelength of both R(ω) and R(ω) to obtain a second maximum spectrum R′(ω); step S6 of repeating step S5 by substituting the second maximum spectrum R'(ω) for the first maximum spectrum R(ω) until a color temperature change value of the second maximum spectrum R'(ω), i.e., a color temperature change value between the previous second maximum spectrum R'(ω) and the current second maximum spectrum R'(ω), becomes smaller than a predetermined threshold value; The initial predicted light source spectrum L just obtained 0 and step S7 of normalizing L(ω) to obtain a reconstructed light source spectrum L(ω). A method for recovering and reconstructing a light source spectrum based on a hyperspectral image, comprising:

2. The step of constructing the light source spectrum dictionary is specifically: A step S8 of acquiring light source spectrum information, extracting spectral waveforms having the same wavelength range and wavelength interval from the light source spectrum information as waveform vectors, and forming a first waveform vector set; Step S9: Separating and extracting complete peak-trough information of the waveform of a light source with an unspecified spectrum or color temperature, the color temperature or spectrum of which changes during use; and extracting waveform vectors of the same wavelength range and wavelength interval from the peak-trough information to form a second waveform vector set; and a step S10 of marking each waveform vector belonging to the same light source in the first waveform vector set and the second waveform vector set as a simultaneously related and coexisting basis vector sequence to construct an index to form the light source spectrum basis vector set D(n, m, ω), and the light source spectrum dictionary is composed of the light source spectrum basis vector set D(n, m, ω). The method for recovering and reconstructing a light source spectrum based on a hyperspectral image according to claim 1 .

3. In step S9, the complete peak-trough information of the waveform of the light source of unspecified spectrum or color temperature is separated and extracted by a multi-Gaussian peak fitting method. The method for recovering and reconstructing a light source spectrum based on a hyperspectral image according to claim 2 .

4. The higher-ranked basis vectors arranged in the new light source spectrum basis vector set D'(k',m',ω) in descending order of relevance obtained are at most two or three light source components. The light source spectrum based on the hyperspectral image according to claim 1 is How to rebuild restore.

5. The linear regression involves a least squares linear decomposition. The light source spectrum based on the hyperspectral image according to claim 1 is How to rebuild restore.

6. The formula for the two-dimensional Gaussian kernel used in the Gaussian convolution process is and where σ is the standard deviation of the Gaussian function The light source spectrum based on the hyperspectral image according to claim 1 is How to rebuild restore.

7. The g(x, y) is replaced with an i×i numeric matrix. The light source spectrum based on the hyperspectral image according to claim 6 is How to rebuild restore.

8. The color temperature change value of the second maximum spectrum R'(ω) is the formula is calculated based on, where: is the initial predicted light source spectrum L 0 (ω) is the color temperature corresponding to is the color temperature of the second maximum spectrum R'(ω) The light source spectrum based on the hyperspectral image according to claim 1 is How to rebuild restore.

9. A computer-readable storage medium on which a computer program is stored, When the program is executed by a processor, the program according to any one of claims 1 to 8 is How to achieve this A computer-readable storage medium comprising:

Citation Information

Patent Citations

  • Measuring and constructing method of broadband light source spectrum distribution function and self-correlation function

    CN104502071A

  • Virtual multi-source spectral reconstruction method based on color constancy

    CN108020519A

  • Computing method of spectral index value of laser beam, arithmetic unit of spectral index value of laser beam, and measuring device of spectral waveform

    JP2004271498A

  • Spectral reconstruction

    WO2017066825A1