Spectrum recovery method, spectrum recovery device, and electronic device

Abstract

The present application relates to a spectrum recovery method, device and electronic equipment. The spectrum recovery method comprises: obtaining a light energy response signal matrix output by a photosensitive chip of the spectrum imaging device and a standard spectrum; determining a basic element recovery function and a response signal vector of the basic element recovery function based on the light energy response signal matrix, the basic element recovery function using a predetermined pixel value of the photosensitive chip and pixel values around the predetermined pixel value to recover a spectral image value of a predetermined channel corresponding to the basic element recovery function; obtaining a recovery tensor, a product of the recovery tensor and the response signal vector being equal to an output of the basic element recovery function based on the response signal vector; and obtaining a recovered spectral image based on the product of the recovery tensor and the response signal vector. In this way, the spectrum recovery is fast, easy to parallel operation and high in recovery accuracy.

Description

Technical Field

[0001] This application relates to the field of spectral chip technology, and more specifically, to a spectral recovery method, apparatus, and electronic device. Background Technology

[0002] Spectral imaging technology is a technique that organically combines spectral detection and imaging. It can image an object under different spectra, simultaneously obtaining the geometric shape information and spectral characteristics of the object being detected. Spectral imaging technology has become an important tool for Earth observation and deep space exploration, and is widely used in fields such as agricultural and forestry production, mineral resource exploration, cultural relic detection, marine remote sensing, environmental monitoring, disaster prevention and mitigation, and military reconnaissance.

[0003] In practical applications, as spectral resolution increases, analytical methods are used for spectral reconstruction in computational spectral chips. The advantage of analytical methods is that they can directly perform inverse problem calculations. The disadvantage is the difficulty in matrix inversion when high reconstruction resolution is required. In computational spectral chips, the number of structural units often reaches tens or even hundreds of thousands. The matrix elements to be solved represent the contribution of pixels to the spectral wavenumber, thus forming a large matrix unsuitable for inversion operations.

[0004] Therefore, there is a need to provide an improved spectral recovery method. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a spectral restoration method, a spectral restoration apparatus, and an electronic device, which restore spectral images by introducing a standard spectrum and a restoration tensor, thereby achieving fast, easily parallelizable, and highly accurate spectral restoration.

[0006] According to one aspect of this application, a spectral restoration method is provided, comprising: acquiring a light energy response signal matrix and a standard spectrum output by a photosensitive chip of a spectral imaging device; determining a basic element recovery function and a response signal vector of the basic element recovery function based on the light energy response signal matrix, wherein the basic element recovery function uses predetermined pixel values ​​of the photosensitive chip and nearby pixel values ​​to restore the spectral image value of a corresponding predetermined channel; acquiring a restoration tensor, wherein the product of the restoration tensor and the response signal vector is equal to the output of the basic element recovery function based on the response signal vector; and obtaining a restored spectral image based on the product of the restoration tensor and the response signal vector.

[0007] In the above spectral restoration method, the light energy response signal matrix is ​​represented as a matrix B including two dimensions: image width w and image height h. The standard spectrum has a dimension of l and is set to minimize the distance between the product of the true value tensor of the spectral image received by the spectral imaging device and the standard spectrum and the tensor of the spectral image to be restored.

[0008] In the above spectral restoration method, the standard spectrum is denoted as s, and the channel standard spectrum corresponding to the k-th channel of the standard spectrum is denoted as s. k , so that:

[0009] x k →O(i,j)s k

[0010] Where, x k O(i,j) is the spectral image value of the k-th channel of a certain spectral pixel, and O(i,j) is the tensor of the true value of the spectral curve of a certain spectral pixel, where → indicates that the Euclidean distance between tensors is minimized.

[0011] In the above spectral restoration method, the basic element restoration function uses pixel values ​​that are a predetermined threshold p away from the predetermined pixel in both width and height, and the response signal vector of the predetermined pixel is denoted as... Represented as:

[0012]

[0013] In the above spectral restoration method, the restoration tensor is denoted as C, and the channel restoration vector in the restoration tensor used to restore the value of the k-th channel of the predetermined pixel is denoted as c. k ,have:

[0014]

[0015] in, The basic element recovery function is based on the response signal vector. The output of .

[0016] In the above-described spectral restoration method, the process of solving the restoration tensor includes: establishing a first constraint equation based on the restoration tensor, the spectral response tensor block corresponding to the fundamental element restoration function of the spectral imaging device, and the standard spectrum; establishing a second constraint equation based on the restoration tensor and the spectral response tensor block; and obtaining the restoration tensor based on the first constraint equation and the second constraint equation.

[0017] In the above spectral restoration method, the first constraint equation is the channel restoration vector c. k The spectral response tensor block in the spectral response tensor that corresponds to the fundamental element recovery function. and the channel standard spectrum s k The product of these is equal to one, expressed as:

[0018]

[0019] in, The spectral response tensor of the spectral imaging device is used to recover X. i,j The spectral response tensor block A(ip:i+p,jp:j+p,:) has its first and second orders rearranged to the same order, forming a tensor block of shape (a 2 The matrix is ​​a matrix of (l), and the spectral response tensor is represented as a tensor A including three dimensions: image width w, image height h, and calibration resolution l.

[0020] In the above spectral restoration method, the second constraint equation is the channel restoration vector c. k The spectral response tensor block in the spectral response tensor that corresponds to the fundamental element recovery function. The product of a unit vector and a unit vector is constrained to 0, which is expressed as:

[0021]

[0022] Where e is a unit vector.

[0023] In the above spectral restoration method, obtaining the restored tensor based on the first and second constraint equations includes: obtaining the restored tensor based on the first, second, and third constraint equations, wherein the third constraint equation is the Tikhonov matrix and the channel restoration vector c. k The product of the product of the second norm and the Lagrange multiplier λ of the regularization term is constrained to 0, and is expressed as:

[0024] λ‖(Dc k )‖2→0.

[0025] In the above spectral restoration method, solving the first, second, and third constraint equations to obtain the restored tensor includes: multiplying the first constraint equation by a sensitivity coefficient and adding it to the second and third constraint equations to obtain a joint equation, expressed as:

[0026]

[0027] Differentiating and finding the zeros of the joint equation, it can be expressed as:

[0028] g′(c k ) = 0;

[0029] The channel recovery vector is obtained as follows:

[0030]

[0031] Iterate through the above steps to obtain the complete recovery tensor as follows:

[0032]

[0033] In the above spectral restoration method, the second constraint equation is the channel restoration vector c. k The spectral response tensor block corresponding to the fundamental element recovery function in the spectral response tensor The product of is constrained to have a L2 norm of 0, expressed as:

[0034]

[0035] In the above spectral restoration method, obtaining the restored tensor based on the first and second constraint equations includes obtaining the restored tensor based on the first, second, and third constraint equations, and includes: multiplying the first constraint equation by a sensitivity coefficient and adding it to the second and third constraint equations to obtain a joint equation, expressed as:

[0036]

[0037] Differentiating and finding the zeros of the joint equation, it can be expressed as:

[0038] g′(c k ) = 0;

[0039] The channel recovery vector is obtained as follows:

[0040]

[0041] In the above spectral recovery method, the first constraint equation is expressed as:

[0042]

[0043] Among them, f k It is a neural network consisting of connection layers and activation layers, and It is the spectral response tensor block A(ip:i+p,jp:j+p,:) corresponding to the fundamental element recovery function in the spectral response tensor. The first and second orders of this tensor block are rearranged to the same order, forming a tensor block with shape (a 2 A matrix of ,l).

[0044] In the above spectral recovery method, the second constraint equation is expressed as:

[0045]

[0046] In the above spectral recovery method, obtaining the recovered tensor based on the first constraint equation and the second constraint equation includes:

[0047] The recovered tensor is obtained based on the first constraint equation, the second constraint equation, and the third constraint equation, wherein the third constraint equation is expressed as:

[0048]

[0049]

[0050] Where N is the noise element, which is a random number matrix of shape (a,a) with expectation of 0 and following a Gaussian distribution.

[0051] In the above spectral recovery method, a fourth constraint equation is established:

[0052]

[0053] In the above-described spectral restoration method, solving the first constraint equation, the second constraint equation, and the third constraint equation to obtain the restored tensor includes: training the neural network based on the dataset to perform spectral restoration through the trained neural network.

[0054] According to another aspect of this application, a spectral restoration apparatus is provided, comprising: a data acquisition unit for acquiring a light energy response signal matrix and a standard spectrum output by a photosensitive chip of a spectral imaging device; a response signal unit for determining a basic element recovery function and a response signal vector of the basic element recovery function based on the light energy response signal matrix, wherein the basic element recovery function uses predetermined pixel values ​​of the photosensitive chip and nearby pixel values ​​to restore the spectral image value of a corresponding predetermined channel; a recovery tensor unit for acquiring a recovery tensor, wherein the product of the recovery tensor and the response signal vector is equal to the output of the basic element recovery function based on the response signal vector; and a spectral restoration unit for obtaining a restored spectral image based on the product of the recovery tensor and the response signal vector.

[0055] According to another aspect of this application, an electronic device is provided, comprising: a processor; and a memory storing computer program instructions that, when the processor is executed, cause the processor to perform the spectral recovery method as described above.

[0056] According to another aspect of this application, a computer-readable storage medium is provided, wherein computer program instructions are stored thereon, which, when executed by a computing device, are operable to perform the spectral recovery method as described above.

[0057] The spectral restoration method, spectral restoration device, and electronic device provided in this application can restore spectral images by introducing standard spectra and restoration tensors, thereby achieving fast, easy-to-parallel-computation, and high-precision spectral restoration. Attached Figure Description

[0058] Various other advantages and benefits of this application will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0059] Figure 1 The illustration shows a schematic configuration diagram of a spectral imaging device according to an embodiment of this application;

[0060] Figure 2 The illustration shows a schematic diagram of establishing a standard spectrum of the RGB color gamut in the spectral recovery method according to an embodiment of this application;

[0061] Figure 3 The illustration shows a flowchart of a spectral recovery method according to an embodiment of this application;

[0062] Figure 4 The figure shows a block diagram of a spectral recovery apparatus according to an embodiment of this application;

[0063] Figure 5 A block diagram of an electronic device according to an embodiment of this application is illustrated. Detailed Implementation

[0064] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0065] Application Overview

[0066] The spectral restoration method according to the embodiments of this application is applied to a spectral imaging device. Figure 1 The illustration shows a schematic configuration diagram of a spectral imaging device according to an embodiment of this application. Figure 1 As shown, in the spectral imaging device according to the embodiments of this application, the optical system is optional, and it may be an optical system such as a lens assembly or a homogenizing assembly. The filter structure is a broadband filter structure in the frequency domain or wavelength domain. The pass spectra of different wavelengths of the filter structure are not completely the same at different locations. The filter structure can be a metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS (microelectromechanical systems), FP etalon, cavity layer, waveguide layer, diffraction element, or other structures or materials with filtering properties. For example, in the embodiments of this application, the filter structure can be the light modulation layer in Chinese Patent CN201921223201.2.

[0067] The image sensor (i.e., photodetector array) can be a CMOS image sensor (CIS), CCD, array photodetector, etc. Additionally, optional data processing units can be MCUs, CPUs, GPUs, FPGAs, NPUs, ASICs, etc., which can export the data generated by the image sensor for external processing.

[0068] For example, after the image sensor measures the light intensity information, it is transmitted to the data processing unit for reconstruction calculation. This process is described in detail below:

[0069] Let x(λ) denote the intensity signals of the incident light at different wavelengths λ, and let T(λ) denote the transmission spectrum curve of the filter structure. The filter (filter structure) has m sets of structural units, and the transmission spectrum of each set of structural units is different. Overall, the filter structure can be denoted as T. i (λ)(i=1,2,3,…,m). Each group of structural units has a corresponding physical pixel below it, which detects the light intensity bi modulated by the filtered light structure. In a specific embodiment of this application, one physical pixel corresponds to one group of structural units, but it is not limited to this. In other embodiments, multiple physical pixels may be grouped together to correspond to one group of structural units. Therefore, in the spectral imaging device according to the embodiments of this application, multiple groups of structural units constitute a "spectral pixel". Furthermore, the present invention can use at least one spectral pixel to reconstruct an image. It should be noted that the effective transmission spectrum (the transmission spectrum used for spectral reconstruction is called the effective transmission spectrum) T of the filter structure. i(λ) The number of structural units may not be the same as the number of structural units. The transmission spectrum of the filter structure is set, tested or calculated manually according to certain rules based on the needs of identification or recovery (for example, the transmission spectrum obtained by each structural unit through testing is the effective transmission spectrum). Therefore, the number of effective transmission spectra of the filter structure may be less than the number of structural units, or even more than the number of structural units. In this modified embodiment, a certain transmission spectrum curve is not necessarily determined by a group of structural units.

[0070] The relationship between the spectral distribution of incident light and the measurements from the image sensor can be expressed by the following formula:

[0071] b i =∫x(λ)*T i (λ)*R(λ)dλ

[0072] After discretization, we get:

[0073] b i =Σ(x(λ)*T i (λ)*R(λ))

[0074] Where R(λ) is the response of the image sensor, denoted as:

[0075] A i (λ)=T i (λ)*R(λ),

[0076] The above equation can then be extended into matrix form:

[0077]

[0078] Among them, b i (i = 1, 2, 3, ..., m) represents the response of the image sensor after the light under test passes through the filter structure. These correspond to the light intensity measurements of the image sensor for each of the m structural units. When one physical pixel corresponds to one structural unit, it can be understood as the light intensity measurements corresponding to m "physical pixels," which is a vector of length m. A represents the system's response to light of different wavelengths, determined by the transmittance of the filter structure and the quantum efficiency of the image sensor. A is a matrix, where each row vector corresponds to a set of structural units responding to incident light of different wavelengths. Here, the incident light is sampled discretely and uniformly, with a total of n sampling points. The number of columns in A is the same as the number of sampling points for the incident light. Here, x(λ) represents the light intensity of the incident light at different wavelengths λ, which is the incident light spectrum to be measured.

[0079] In some embodiments, unlike the above embodiments, the filter structure can be directly formed on the upper surface of the image sensor, such as quantum dots, nanowires, etc. It directly forms the filter structure or material (nanowires, quantum dots, etc.) in the photosensitive area of ​​the sensor. Taking the filter structure as an example, it can be understood that when the raw materials of the image sensor are processed to form the image sensor, a filter structure is formed on the upper surface of the raw materials. The transmission spectrum and the response of the image sensor are integrated, that is, it can be understood that the response of the detector and the transmission spectrum are the same curve. In this case, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following formula:

[0080] b i =Σ(x(λ)*R i (λ))

[0081] That is, in this embodiment, the transmission spectrum A i (λ)=R i (λ)

[0082] Furthermore, it can also be a combination of the two embodiments described above, that is, at least one filter structure for modulating incident light is provided on the image sensor with the filter structure. It can be understood that the image sensor (i.e., photodetector array) in the first embodiment, which can be a CMOS image sensor (CIS), CCD, array photodetector, etc., can be replaced with an image sensor with an integrated filter structure in the second embodiment.

[0083] At this point, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following formula:

[0084] b i =∫x(λ)*T i (λ)*R i (λ)dλ

[0085] After discretization, we get:

[0086] b i =Σ(x(λ)*T i (λ)*R i (λ))

[0087] That is, in this embodiment, A i (λ)=T i (λ)*R i (λ)

[0088] Exemplary methods

[0089] If we consider multi-channel spectral imaging, the imaging principle of each spectral pixel in a spectral imaging device, such as a snapshot spectral camera, can be expressed by the following equation:

[0090] AX = B

[0091] Here, X represents the spectral image tensor to be recovered by the algorithm, generally composed of three orders: w, h, and c. w is the image width in pixels; h is the image height in pixels; and c is the number of spectral image channels output by the spectral imaging device (e.g., c = 3 for RGB images). A is the pre-calibrated spectral response tensor of the spectral imaging device, composed of three orders: w, h, and l. Dimension l is the calibration resolution, that is, the number of spectral data channels for the target spectral band given by the calibration device. This tensor A characterizes the transmittance of the photosensitive chip (i.e., the spectral chip), such as the structure on the image sensor as described above, to monochromatic light of different wavelengths. A(i,j,o) represents the value of the number in the i-th row, j-th column, and o-th layer of tensor A, i.e., the transmittance of pixel (i,j) to the o-th monochromatic light, and A(:,:,o) represents the total transmittance of the photosensitive chip specifically for the o-th monochromatic light. In addition, B is the light energy response signal matrix given by the photosensitive chip of the spectral imaging device, which consists of two orders, w and h.

[0092] Therefore, in this embodiment of the application, spectral recovery is to find F given A, such that:

[0093] argmin[E(X,F(B))]

[0094] Where E represents a certain error function. In image processing, SSIM (Structural Similarity) is usually used as a function to measure error, while F(B) is the spectral recovery function.

[0095] To obtain the spectral image tensor X using spectral reconstruction methods, this application introduces a new concept in the field of spectral reconstruction: the "standard spectrum," denoted as s, with dimension l, and the standard spectrum corresponding to the k-th channel is sk. k This refers to the standard spectral vector. Furthermore, these standard spectral vectors are stacked as column vectors to form a matrix S, consisting of two orders, l and c. In a spectral imaging device, each channel X(:,:,k) of X corresponds to a standard spectrum s. k .

[0096] here, Figure 2 The illustration shows a schematic diagram of establishing a standard spectrum of the RGB color gamut in the spectral recovery method according to an embodiment of this application.

[0097] like Figure 2 As shown, it illustrates the reference to the CIE 1931 standard. The curve (i.e., the tristimulus value curve, which can also be written as X, Y, Z).

[0098] Assuming the observation matrix has an observation range of αnm to βnm and an observation precision of δnm, then the standard spectrum has a total of There are 1 element. Therefore, the three standard spectra can be calculated as follows:

[0099]

[0100]

[0101]

[0102] Where λ = α + δ(j-1).

[0103] Note that the following conditions must be met when establishing a standard spectrum:

[0104]

[0105] ||s||2=1

[0106] Furthermore, if we denote the tensor of the true spectral image received by the spectral imaging device as O, which consists of three orders w, h, and l, representing the spectrum of the initial incident light at each physical pixel, then we have:

[0107] X(i,j,k)→O(i,j)s k

[0108] Where X(i,j,k) can also be denoted as x k Generally speaking, x refers to the spectral image value of the k-th channel of a given spectral pixel, i.e., x k = X(i,j,k), where i and j are generic. Furthermore, O(i,j) generically refers to the tensor of the true value of the spectral curve of a certain spectral pixel, where i and j are also generic. The symbol "→" indicates that it should be as close as possible to, and if used between tensors, it indicates that their Euclidean distance should be as small as possible.

[0109] As mentioned above, obtaining X requires the aid of the spectral reconstitution function, i.e.:

[0110] F(B) = X

[0111] therefore:

[0112] F(B)→OS

[0113] Based on this, the objective of the spectral recovery method according to the embodiments of this application is transformed into finding a function F such that:

[0114] ‖F(B)-OS‖2→0

[0115] Specifically, as described above, the main functional structure of the spectral imaging device according to the embodiments of this application is a photosensitive chip (i.e., a spectral chip) covered with a modulation layer. The photosensitive chip has w wide and h high photosensitive units, each of which can independently respond to light irradiation. It can be assumed that the response of the photosensitive unit to light is positively linearly correlated with the light energy. A modulation layer composed of various structures is covered on the photosensitive chip, and this is defined as a spectral chip. This modulation layer causes different photosensitive units on the photosensitive chip to have different responses to light with the same energy but different spectra. This change in response is determined by A, i.e.:

[0116]

[0117] The above expression represents the accumulation of tensor A*O at order l. Here, * denotes generalized multiplication. When used between tensors, it takes the smallest tensor order and multiplies its elements. For higher orders, a broadcast strategy is used to multiply elements by elements.

[0118] Here, tensor A is determined by the modulation layer on the photosensitive chip, and A(i,j) is the transmission spectrum on the photosensitive unit (i,j). In the following text, the term "physical pixel" is used to refer to the smallest imaging unit on the photosensitive chip, which can be composed of one or more physical pixels. This invention is preferably illustrated using a one-to-one example.

[0119] In the multispectral image X finally output by the spectral restoration method according to the embodiments of this application, each X(i,j) is defined as a "spectral pixel". As the name suggests, a spectral pixel is a geometric pixel in a multispectral image. Further, it can be understood that the modulation layer has at least one modulation unit, the modulation unit corresponds to at least one physical pixel, the modulation unit and the physical pixel constitute a structural pixel, and at least one structural pixel constitutes a spectral pixel; each modulation unit has a corresponding transmission spectrum curve, that is, the transmission spectrum curve is determined by the modulation unit, and the transmission spectrum curve further constitutes a transmission spectrum tensor A. It should be noted that the transmission spectrum tensor A can be obtained by calibration, or it can be calculated by human calculation, or it can be obtained by other methods. Further, the spectral chip of the spectral imaging device in this invention has at least two spectral pixels, and there are at least two spectral pixels with different corresponding structures or modulation effects on light. X(i,j,k), that is, the value of a certain channel in a certain spectral pixel, can be defined as a "basic element". The basic element is the smallest element required by the spectral restoration method according to the embodiments of this application. In parallel computing, the restoration operation of each basic element corresponds to one thread.

[0120] To obtain the value of a basic element, we need to use the value of its corresponding physical pixel and the values ​​of its nearby physical pixels, that is:

[0121]

[0122] The selection of p depends on the actual situation. i,j,k Defined as the basic primitive recovery function. Once f is determined... i,j,k By determining the function F as described above, spectral reconstruction can be achieved. Therefore, in the following text, f will be used to represent f when referring to a specific channel of a specific spectral pixel. i,j,k ,use This refers to the photosensitive chip response signal vector corresponding to a specific spectral pixel, that is, It is the photosensitive chip response signal vector used to determine a specific spectral pixel. It is transformed into a vector B(ip:i+p,jp:j+p). When referring to a specific spectral pixel in general, it can be used... Therefore, the above formula is to find f such that it satisfies:

[0123]

[0124] The function f can be of any differentiable form. When it is a linear function, it takes the following form:

[0125]

[0126] Here, in the spectral recovery method according to the embodiments of this application, a new concept, "recovery tensor," is introduced, for example, denoted as C, which is composed of w, h, a 2 It consists of four orders: a, c, and 'a'. Here, a is the side length of the physical pixel array used to recover any spectral pixel, measured in pixels. Furthermore, c... k The recovery tensor C is the vector used to recover the value of the k-th channel of a specific pixel, denoted as C(i,j,:,k); where i and j are generic terms. Furthermore, · represents a vector or matrix product; when used between vectors, it's an inner product; when used between matrices, it's a matrix multiplication. If used between tensors of different orders, the smallest first or second order is used for the vector inner product or matrix multiplication; for higher orders, a broadcast strategy is employed.

[0127] As mentioned above, in a spectral imaging device, each channel X(:,:,k) of X corresponds to a standard spectrum s. k Under the premise that the total light energy remains at unit energy, if the actual spectrum O(i,j,*) received by the spectral imaging device at a certain spectral pixel (i,j) is completely consistent with the standard spectrum corresponding to channel k, then the spectral pixel value of the corresponding channel of the image output by the imaging device should be 1. Of course, those skilled in the art will understand that a spectral pixel value of 1 is not a limitation and can be other constants. Thus, the following limiting equation (1) is obtained:

[0128]

[0129] here, For use in recovering X i,j The spectral response tensor block A(ip:i+p,jp:j+p,:) should have its first and second orders rearranged to the same order, forming a tensor block of shape (a 2 A matrix of (i,j), where each pixel (i,j) is defined as...

[0130] It can be understood that a standard spectrum is a standard filter on each channel. When the actual spectrum completely overlaps with the transmission spectrum of this filter, the value on that channel should be 1. This standard spectrum can be a Gaussian filter with a specific wavelength at its center and a fixed peak width; or it can be any defined shape (such as RGB tristimulus values).

[0131] In addition to this, for s k For all other spectra, the response should be as small as possible. Therefore, the constraint equation (2) is obtained:

[0132]

[0133] Where e is a vector in which all elements are 1.

[0134] In addition, to prevent overfitting, the Tikhonov method can be used to adjust the parameter vector c. k By applying constraints, we obtain the constraint equation (3):

[0135] λ‖(Dc k )‖2→0

[0136] Where D is the Tikhonov matrix, a diagonal matrix, which is usually taken as the identity matrix, and λ is the Lagrange multiplier for the regularization term. Furthermore, if overfitting prevention is not required, λ can be set to 0.

[0137] By using a solver to solve the above three equations, we can obtain c. k The value of .

[0138] Considering the limited computational speed of the solver, in the spectral recovery method according to the embodiments of this application, a more efficient method can be proposed to solve the above set of equations, that is, to integrate the three constraint equations into one:

[0139]

[0140] To allow the constraint equation (1) to be included in g, a sensitivity multiplier α is introduced to characterize the spectral recovery sensitivity requirement. A higher α results in higher sensitivity, but correspondingly reduces model robustness and spectral accuracy. Therefore, α needs to be validated in practical applications. Methods for determining α include, but are not limited to, using a leave-one-out strategy to train from a known spectral dataset; or empirically setting it based on the error characteristics of known calibration data and actual requirements.

[0141] Furthermore, as mentioned above, when λ = 0, the integrated constraint equations may also include only the first and second constraint equations as described above.

[0142] In order to make g(c k To minimize it, we should find its zero point after differentiating it, that is:

[0143] g′(c k ) = 0

[0144] To simplify calculations, the integrated formula can be adjusted as follows:

[0145]

[0146] Solving the equation, we get c k Method for finding:

[0147]

[0148] Extending this algorithm to the entire image yields:

[0149]

[0150] The final recovery matrix C can then be used to obtain the hyperspectral image using the following formula:

[0151]

[0152] The spectral restoration method according to the embodiments of this application has the advantages of high speed, easy parallel computation, and high restoration accuracy, and it is feasible as a spectral imaging restoration algorithm.

[0153] In a modified embodiment, considering that the constraint equation (2) is required to be more stringent in the following form, but model fitting will also be more difficult:

[0154]

[0155] The final constraint equations are:

[0156]

[0157] Take the derivative, and set the derivative to zero:

[0158] g′(c k ) = 0

[0159] To facilitate the calculation of modulation constraint equations

[0160]

[0161] Solving for:

[0162]

[0163] Figure 3 The illustration shows a flowchart of a spectral recovery method according to an embodiment of this application.

[0164] like Figure 3 As shown, the spectral recovery method according to an embodiment of this application includes the following steps.

[0165] Step S110: Obtain the light energy response signal matrix and standard spectrum output by the photosensitive chip of the spectral imaging device.

[0166] That is, obtain the light energy response signal matrix B as described above, where B includes two dimensions: image width w and image height h.

[0167] That is, in the spectral restoration method according to the embodiments of this application, the light energy response signal matrix is ​​represented as a matrix B including two dimensions: image width w and image height h.

[0168] Furthermore, the dimension of the standard spectrum is the same as the calibration resolution of the spectral response tensor of the pre-calibrated spectral imaging device, i.e., it is also 1. The standard spectrum is set such that the distance between the product of the true value tensor of the spectral image received by the spectral imaging device and the standard spectrum is minimized and the tensor of the spectral image to be recovered is minimized.

[0169] X(i,j,k)→O(i,j)s k

[0170] Here, the standard spectrum is denoted as s, and the standard spectrum corresponding to the k-th channel is denoted as sk. k X(i,j,k) can also be denoted as x k Generally speaking, x refers to the spectral image value of the k-th channel of a given spectral pixel, i.e., x k = X(i,j,k), where i and j are generic. Furthermore, O(i,j) generically refers to the tensor of the true spectral image value of a certain spectral pixel, where i and j are also generic. The symbol "→" indicates that it should be as close as possible to, and if used between tensors, it indicates that their Euclidean distance should be as small as possible.

[0171] That is, in the spectral recovery method according to the embodiments of this application, the standard spectrum is represented as s, and the channel standard spectrum corresponding to the k-th channel of the standard spectrum is represented as s. k , so that:

[0172] x k →O(i,j)s k

[0173] Where, x k O(i,j) is the spectral image value of the k-th channel of a certain pixel, and O(i,j) is the tensor of the true spectral image value of a certain pixel. → indicates that the Euclidean distance between tensors is minimized.

[0174] Step S120: Determine the basic element recovery function and the response signal vector of the basic element recovery function based on the light energy response signal matrix. The basic element recovery function uses the predetermined pixel value of the photosensitive chip and the pixel value of its vicinity to recover the spectral image value of the corresponding predetermined channel.

[0175] That is, based on the formula described above:

[0176]

[0177] Obtain the response signal vector

[0178] That is, in the spectral restoration method according to the embodiments of this application, the basic element restoration function uses pixel values ​​that are at a predetermined distance of a predetermined threshold from the predetermined pixel in both width and height, and the response signal vector of the predetermined pixel is denoted as...

[0179] Step S130: Obtain the recovery tensor, the product of the recovery tensor and the response signal vector is equal to the output of the basic element recovery function based on the response signal vector.

[0180] That is, in the spectral restoration method according to the embodiments of this application, the restoration tensor is denoted as C, and the channel restoration vector in the restoration tensor used to restore the value of the k-th channel of the predetermined pixel is denoted as c. k Then we have:

[0181]

[0182] in, The basic element recovery function is based on the response signal vector. The output of .

[0183] Step S140: The recovered spectral image is obtained based on the product of the recovery tensor and the response signal vector.

[0184] That is, in the spectral restoration method according to the embodiments of this application, the pixels of the spectral image are:

[0185]

[0186] As described above, in this embodiment of the application, the recovery tensor can be solved by establishing constraint equations. The specific process includes: establishing a first constraint equation based on the recovery tensor, the spectral response tensor block corresponding to the basic element recovery function of the spectral imaging device, and the standard spectrum; establishing a second constraint equation based on the recovery tensor and the spectral response tensor block; and obtaining the recovery tensor based on the first constraint equation and the second constraint equation.

[0187] The first constraint equation is the constraint equation (1) as described above:

[0188]

[0189] in, For use in recovering X i,j The spectral response tensor block A(ip:i+p,jp:j+p,:) should have its first and second orders rearranged to the same order, forming a tensor block of shape (a 2 A matrix of (i,j), where each pixel (i,j) is defined as...

[0190] Therefore, in the spectral recovery method according to the embodiments of this application, the first limiting equation is the channel recovery vector c. k The spectral response tensor block in the spectral response tensor that corresponds to the fundamental element recovery function. and the channel standard spectrum s k The product of is equal to one.

[0191] Furthermore, the second constraint equation is the constraint equation (2) as described above:

[0192]

[0193] Where e is a vector in which all elements are 1, i.e., a unit vector.

[0194] Therefore, in the spectral recovery method according to the embodiments of this application, the second limiting equation is the channel recovery vector c. k The spectral response tensor block in the spectral response tensor that corresponds to the fundamental element recovery function. The product constraint with the unit vector is 0.

[0195] Thus, based on the first constraint equation and the second constraint equation, the recovered tensor is obtained.

[0196] Furthermore, as described above, preferably, in the spectral recovery method according to the embodiments of this application, the solution can be obtained by integrating the first constraint equation and the second constraint equation.

[0197] Furthermore, preferably, when solving the recovery tensor, the recovery tensor is further constrained to prevent overfitting to obtain a third constraint equation.

[0198] The third constraint equation is the constraint equation (3) as described above:

[0199] λ‖(Dc k )‖2→0

[0200] Therefore, in the spectral recovery method according to the embodiments of this application, the third limiting equation is the Tikhonov matrix and the channel recovery vector c. k The product of the second norm of the product and the Lagrange multiplier λ of the regular term is constrained to 0.

[0201] Thus, obtaining the recovered tensor based on the first and second constraint equations includes solving the first, second, and third constraint equations to obtain the recovered tensor.

[0202] Specifically, the first constraint equation is first multiplied by the sensitivity coefficient, and then added to the second and third constraint equations to obtain the joint equation. Next, the channel recovery vector is obtained by solving for the zeros of the derivative of the joint equation. Finally, the above steps are iteratively repeated to obtain the entire recovery tensor.

[0203] Therefore, in the spectral recovery method according to the embodiments of this application, solving the first constraint equation, the second constraint equation, and the third constraint equation to obtain the recovered tensor includes:

[0204] Multiplying the first constraint equation by the sensitivity coefficient and adding it to the second and third constraint equations yields the joint equation, expressed as:

[0205]

[0206] Differentiating and finding the zeros of the joint equation, it can be expressed as:

[0207] g′(c k ) = 0

[0208] The channel recovery vector is obtained as follows:

[0209]

[0210] Iterate through the above steps to obtain the complete recovery tensor as follows:

[0211]

[0212] Furthermore, in a variant embodiment, the second limiting equation is the channel recovery vector c. k The spectral response tensor block corresponding to the fundamental element recovery function in the spectral response tensor The product of is constrained to have a L2 norm of 0, expressed as:

[0213]

[0214] Then, multiplying the first constraint equation by the sensitivity coefficient and adding it to the second and third constraint equations yields the joint equation, expressed as:

[0215]

[0216] The channel recovery vector is obtained as follows:

[0217]

[0218] Here, we can see that the value of the channel recovery vector is based on the spectral response tensor block corresponding to the fundamental element recovery function. and standard spectrum k Therefore, for a given spectral imaging device, the spectral response tensor can be pre-calibrated and a standard spectrum can be acquired to determine the channel recovery vector. Thus, when performing spectral imaging, the response signal vector can be determined based on the light energy response signal matrix, thereby obtaining the recovered spectral image.

[0219] Therefore, by introducing a standard spectrum and a recovery tensor to recover the spectral image, a spectral recovery method that is fast, easy to parallelize, and has high recovery accuracy according to the embodiments of this application is realized.

[0220] Furthermore, in the embodiments of this application, a method based on a neural network model can also be used to restore the spectral image.

[0221] Specifically, when using a neural network for recovery, that is:

[0222]

[0223] Where f is a neural network composed of connection layers and activation layers. To allow the neural network to acquire as much information as possible, the information from the response matrix needs to be further incorporated, i.e.:

[0224]

[0225] According to the definition of standard spectrum as described above, when The entire covered region (ip:i+p,jp:j+p) is covered by a spectrum of s. k When the light shines, That is, the first limiting equation is:

[0226]

[0227] At the same time, considering the need to minimize errors, the second constraint equation should be satisfied:

[0228]

[0229] Furthermore, for neural networks, regularization terms can no longer be used to control overfitting. Therefore, noise factors are introduced. The data is strengthened to obtain the third constraint equation:

[0230]

[0231]

[0232] Here, N represents the noise element, which is a random number matrix with an expected value of 0 and follows a Gaussian distribution. The farther the element is from the center of the matrix, the greater the variance of the random number. The specific parameters can be selected according to the actual training situation.

[0233] In addition, other common data augmentation methods can be used to control model overfitting. However, these two constraints alone are insufficient for more complex neural network models to learn effectively. Therefore, in this embodiment, data from a hyperspectral database can be used to further train the model, i.e., a fourth constraint equation can be established:

[0234]

[0235] Where t is the true value of the spectral image.

[0236] Based on the above constraint equations, a dataset is created to train f, thereby achieving spectral imaging restoration. In the embodiments of this application, the neural networks include, but are not limited to: FC (fully connected), CNN (convolutional neural network), RNN (recurrent neural network), ResNet, attention (attention neural network), transformer (transformer neural network), and their variations.

[0237] That is, in the spectral recovery method according to the embodiments of this application, the first limiting equation is expressed as:

[0238]

[0239] Among them, f kIt is a neural network consisting of connection layers and activation layers.

[0240] Furthermore, in the above-described spectral recovery method, the second constraint equation is expressed as:

[0241]

[0242] Furthermore, in the above-described spectral recovery method, obtaining the recovered tensor based on the first and second constraint equations includes: obtaining the recovered tensor based on the first, second, and third constraint equations, wherein the third constraint equation is expressed as:

[0243]

[0244]

[0245] Where N is the noise element, which is a random number matrix of shape (a,a) with expectation of 0 and following a Gaussian distribution.

[0246] Furthermore, the above-mentioned spectral recovery method further includes establishing a fourth constraint equation:

[0247]

[0248] Accordingly, in the above-described spectral restoration method, solving the first constraint equation, the second constraint equation, and the third constraint equation to obtain the restoration tensor includes: training the neural network based on the dataset to perform spectral restoration through the trained neural network.

[0249] Indicative device

[0250] Figure 4 A block diagram of a spectral recovery apparatus according to an embodiment of this application is shown.

[0251] like Figure 4 As shown, the spectral restoration apparatus 200 according to an embodiment of this application includes: a data acquisition unit 210, configured to acquire the light energy response signal matrix and standard spectrum output by the photosensitive chip of the spectral imaging device; a response signal unit 220, configured to determine a basic element recovery function and a response signal vector of the basic element recovery function based on the light energy response signal matrix, wherein the basic element recovery function uses predetermined pixel values ​​of the photosensitive chip and nearby pixel values ​​to restore the spectral image value of its corresponding predetermined channel; a recovery tensor unit 230, configured to acquire a recovery tensor, wherein the product of the recovery tensor and the response signal vector is equal to the output of the basic element recovery function based on the response signal vector; and a spectral restoration unit 240, configured to obtain the restored spectral image based on the product of the recovery tensor and the response signal vector.

[0252] Here, those skilled in the art will understand that the specific functions and operations of each unit and module in the above-described spectral recovery device 200 have been referenced above. Figure 3 The spectral recovery method is described in detail here, and therefore, its repeated description will be omitted.

[0253] As described above, the spectral recovery device 200 according to the embodiments of this application can be implemented in various terminal devices, such as servers for high-resolution spectral recovery, or various spectrometers and spectral imaging devices. In one example, the spectral recovery device 200 according to the embodiments of this application can be integrated into the terminal device as a software module and / or a hardware module. For example, the spectral recovery device 200 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the spectral recovery device 200 can also be one of many hardware modules of the terminal device.

[0254] Alternatively, in another example, the spectral recovery device 200 and the terminal device can also be separate devices, and the spectral recovery device 200 can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0255] Exemplary electronic devices

[0256] Below, for reference Figure 5 This describes an electronic device according to embodiments of the present application.

[0257] Figure 5 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0258] like Figure 5 As shown, the electronic device 10 includes one or more processors 11 and memory 12.

[0259] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0260] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the spectral recovery methods of the various embodiments of this application described above and / or other desired functions. Various contents such as response signal data and standard spectral data may also be stored in the computer-readable storage medium.

[0261] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0262] For example, the input device 13 can be, for example, a keyboard, a mouse, etc.

[0263] The output device 14 can output various information to the outside, such as spectral reconstruction results. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0264] Of course, for the sake of simplicity, Figure 5 Only some of the components of the electronic device 10 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0265] Exemplary computer program products and computer-readable storage media

[0266] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the spectral recovery methods according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0267] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0268] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the spectral recovery methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.

[0269] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0270] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0271] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0272] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0273] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0274] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A spectral reconstruction method, characterized in that, include: Acquire the light energy response signal matrix and standard spectrum output by the photosensitive chip of the spectral imaging device; The basic element recovery function and the response signal vector of the basic element recovery function are determined based on the light energy response signal matrix. The basic element recovery function uses the predetermined pixel value of the photosensitive chip and the pixel value of its vicinity to recover the spectral image value of the corresponding predetermined channel. Obtain the recovery tensor, the product of the recovery tensor and the response signal vector is equal to the output of the basic element recovery function based on the response signal vector; as well as The recovered spectral image is obtained based on the product of the recovered tensor and the response signal vector; The process of solving the recovery tensor includes: Based on the recovery tensor, a first constraint equation is established between the spectral response tensor block of the spectral response tensor of the spectral imaging device and the basic element recovery function, and the standard spectrum. A second constraint equation is established based on the recovery tensor and the spectral response tensor block; and The recovered tensor is obtained based on the first constraint equation and the second constraint equation; The light energy response signal matrix is ​​represented as a matrix B including two dimensions: image width w and image height h. The standard spectrum has a dimension of l and is set to minimize the distance between the product of the true value tensor of the spectral image received by the spectral imaging device and the standard spectrum and the tensor of the spectral image to be recovered. The standard spectrum is denoted as s, and the channel standard spectrum corresponding to the k-th channel of the standard spectrum is denoted as s. k So that: x k →O(i,j)s k , where x k O(i,j) is the spectral image value of the k-th channel of a certain spectral pixel, and O(i,j) is the tensor of the true value of the spectral curve of a certain spectral pixel, and → indicates that the Euclidean distance between tensors is minimized. The basic meta-recovery function uses pixel values ​​that are a predetermined threshold p away from the predetermined pixel in both width and height, and the response signal vector of the predetermined pixel is denoted as... Represented as:

2. The spectral recovery method as described in claim 1, wherein, The recovery tensor is denoted as C, and the channel recovery vector in the recovery tensor used to recover the value of the k-th channel of the predetermined pixel is denoted as c. k ,have: in, The basic element recovery function is based on the response signal vector. The output of .

3. The spectral recovery method as described in claim 1, wherein, The first constraint equation is the channel recovery vector c k The spectral response tensor block in the spectral response tensor that corresponds to the fundamental element recovery function. and the channel standard spectrum s k The product of these is equal to one, expressed as: in, The spectral response tensor of the spectral imaging device is used to recover X. i,j The spectral response tensor block A(ip:i+p,jp:j+p,:) has its first and second orders rearranged to the same order, forming a tensor block of shape (a 2 The matrix is ​​a matrix of (l), and the spectral response tensor is represented as a tensor A including three dimensions: image width w, image height h, and calibration resolution l.

4. The spectral recovery method as described in claim 3, wherein, The second constraint equation is the channel recovery vector c. k The spectral response tensor block in the spectral response tensor that corresponds to the fundamental element recovery function. The product of a unit vector and a vector is constrained to 0, which is expressed as: Where e is a unit vector.

5. The spectral recovery method as described in claim 4, wherein, The recovered tensor obtained based on the first constraint equation and the second constraint equation includes: The recovered tensor is obtained based on the first, second, and third constraint equations, wherein the third constraint equation is the Tikhonov matrix and the channel recovery vector c. k The product of the product of the second norm and the Lagrange multiplier λ of the regularization term is constrained to 0, and is expressed as: λ‖(Dc k )‖2→0。 6. The spectral recovery method as described in claim 5, wherein, Solving the first, second, and third constraint equations to obtain the recovered tensor includes: Multiplying the first constraint equation by the sensitivity coefficient and adding it to the second and third constraint equations yields the joint equation, expressed as: Differentiating and finding the zeros of the joint equation, it can be expressed as: g ′ (c k )=0; The channel recovery vector is obtained as follows: Iterate through the above steps to obtain the complete recovery tensor as follows:

7. The spectral recovery method as described in claim 3, wherein, The second constraint equation is the channel recovery vector c. k The spectral response tensor block corresponding to the fundamental element recovery function in the spectral response tensor The product of is constrained to have a L2 norm of 0, expressed as:

8. The spectral recovery method as described in claim 7, wherein, The recovered tensor obtained based on the first and second constraint equations includes the recovered tensor obtained based on the first, second, and third constraint equations, and includes: Multiplying the first constraint equation by the sensitivity coefficient and adding it to the second and third constraint equations yields the joint equation, expressed as: Differentiating and finding the zeros of the joint equation, it can be expressed as: g ′ (c k )=0; The channel recovery vector is obtained as follows:

9. The spectral recovery method as described in claim 1, wherein, The first constraint equation is expressed as: Among them, f k It is a neural network consisting of connection layers and activation layers, and It is the spectral response tensor block A(ip:i+p,jp:j+p,:) corresponding to the fundamental element recovery function in the spectral response tensor. The first and second orders of this tensor block are rearranged to the same order, forming a tensor block with shape (a 2 A matrix of ,l).

10. The spectral recovery method as described in claim 9, wherein, The second constraint equation is expressed as:

11. The spectral recovery method as described in claim 10, wherein, The recovered tensor obtained based on the first constraint equation and the second constraint equation includes: The recovered tensor is obtained based on the first constraint equation, the second constraint equation, and the third constraint equation, wherein the third constraint equation is expressed as: Where N is the noise element, which is a random number matrix of shape (a,a) with expectation of 0 and following a Gaussian distribution.

12. The spectral recovery method as described in claim 11, further comprising: establishing a fourth constraint equation:

13. The spectral recovery method as described in claim 12, wherein, Solving the first, second, and third constraint equations to obtain the recovered tensor includes: The neural network is trained based on the dataset to perform spectral recovery using the trained neural network.

14. A spectral recovery device, characterized in that, include: The data acquisition unit is used to acquire the spectral response tensor of a pre-calibrated spectral imaging device, the light energy response signal matrix output by the photosensitive chip of the spectral imaging device, and the standard spectrum. A response signal unit is used to determine a basic element recovery function and a response signal vector of the basic element recovery function based on a light energy response signal matrix. The basic element recovery function uses a predetermined pixel value of the photosensitive chip and the pixel values ​​near it to recover the spectral image value of its corresponding predetermined channel. A recovery tensor unit is used to obtain a recovery tensor, the product of which and the response signal vector are equal to the output of the basic element recovery function based on the response signal vector; as well as A spectral reconstruction unit is used to obtain the reconstructed spectral image based on the product of the reconstruction tensor and the response signal vector; The process of solving the recovery tensor includes: Based on the recovery tensor, a first constraint equation is established between the spectral response tensor block of the spectral response tensor of the spectral imaging device and the basic element recovery function, and the standard spectrum. A second constraint equation is established based on the recovery tensor and the spectral response tensor block; and The recovered tensor is obtained based on the first constraint equation and the second constraint equation; The light energy response signal matrix is ​​represented as a matrix B including two dimensions: image width w and image height h. The standard spectrum has a dimension of l and is set to minimize the distance between the product of the true value tensor of the spectral image received by the spectral imaging device and the standard spectrum and the tensor of the spectral image to be recovered. The standard spectrum is denoted as s, and the channel standard spectrum corresponding to the k-th channel of the standard spectrum is denoted as s. k So that: x k →O(i,j)s k , where x k O(i,j) is the spectral image value of the k-th channel of a certain spectral pixel, and O(i,j) is the tensor of the true value of the spectral curve of a certain spectral pixel, and → indicates that the Euclidean distance between tensors is minimized. The basic meta-recovery function uses pixel values ​​that are a predetermined threshold p away from the predetermined pixel in both width and height, and the response signal vector of the predetermined pixel is denoted as... Represented as:

Images