Spectromodulation microstructure screening method, device, equipment, medium, program product and computing spectral imaging and spectral analysis system based on wide spectrum measurement base

Abstract

This invention provides a method, apparatus, device, medium, program product, and computational spectral imaging and spectral analysis system for screening spectral modulation microstructures based on a broadband measurement basis, relating to the field of optical technology. The method includes: acquiring the transmission spectra of multiple candidate spectral modulation microstructures; determining a sparse matrix based on the spectral application scenario; and performing inverse design based on the sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures, iteratively screening at least one target spectral modulation microstructure as the optimal combination of spectral modulation microstructures. The inverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix, which is determined based on the sparse matrix. Through this method, the optimal combination of spectral modulation microstructures can possess task adaptability for specific spectral application scenarios, which is beneficial for ensuring the system's spectral reconstruction performance and spectral feature extraction performance in spectral application scenarios.

Description

Technical Field

[0001] This invention relates to the field of optical technology, and in particular to a method, apparatus, device, medium, program product, and computational spectral imaging and spectral analysis system for screening spectral modulation microstructures based on a broadband measurement basis. Background Technology

[0002] Spectral imaging and analysis technologies can provide crucial spectral information for applications such as material identification, biological detection, and industrial inspection. Traditional spectral imaging and analysis systems generally use dispersive elements such as gratings and prisms, and operate in a scanning or push-broom manner, resulting in large system size, complex structure, and limited imaging speed.

[0003] To achieve system miniaturization and snapshot imaging, the industry has proposed a computational spectral imaging and spectral analysis scheme based on broadband modulation. The core idea of ​​this scheme is to integrate several spectral modulation microstructures (i.e., broadband modulation microstructures) in front of the system detector. These spectral modulation microstructures can generate characteristic spectral codes for the incident light of the object under test at different pixels. Based on these characteristic spectral codes, the complete spectral information of the incident light can be calculated and reconstructed.

[0004] In computational spectral imaging and spectral analysis schemes based on broadband modulation, the system integrates a variety of different spectral modulation microstructures. The spectral response performance of different spectral modulation microstructures varies. In order to improve the spectral resolution of spectral imaging and the accuracy of spectral analysis, it is necessary to rationally design all the spectral modulation microstructures integrated in the system to obtain the optimal combination of spectral modulation microstructures.

[0005] Currently, the design methods for spectral modulation microstructures are mainly divided into two categories: forward design and inverse design. Forward design usually requires generating a large number of candidate spectral modulation microstructures and using a preset objective function to evaluate the transmission spectrum combination performance of different spectral modulation microstructures one by one, and then selecting the spectral modulation microstructure combination that meets the requirements. Inverse design, based on the preliminary evaluation results, introduces search strategies such as genetic algorithms and simulated annealing algorithms, and iteratively updates and filters the transmission spectrum combinations of different spectral modulation microstructures to approximate the optimal spectral modulation microstructure combination.

[0006] In existing methods for screening spectral modulation microstructures, whether using forward or reverse design, the objective function is typically constructed based on the mathematical relationships between the transmission spectra of different spectral modulation microstructures. First, a measurement matrix is ​​built based on the combination of transmission spectra of different spectral modulation microstructures. Then, the objective function is constructed based on indices such as the average column coherence and average row coherence of the measurement matrix. While these indices can describe the discriminative power of different spectral modulation microstructures in their spectral responses, they essentially only reflect the mathematical properties of the measurement matrix and do not strictly correspond to the spectral reconstruction and feature extraction performance of the system in real-world spectral applications. Therefore, the spectral modulation microstructure combinations obtained through existing objective function screening lack task adaptability for specific spectral application scenarios and are prone to affecting the spectral reconstruction and feature extraction performance of the system in those scenarios.

[0007] In summary, existing methods for screening spectral modulation microstructures lack task adaptability for specific spectral application scenarios, making it difficult to guarantee the spectral reconstruction performance and spectral feature extraction performance of computational spectral imaging and spectral analysis systems in specific spectral application scenarios. Summary of the Invention

[0008] This invention provides a method, apparatus, device, medium, program product, and computational spectral imaging and spectral analysis system for screening spectral modulation microstructures based on a broadband measurement basis. This addresses the shortcomings of existing spectral modulation microstructure screening methods, which lack task adaptability for specific spectral application scenarios and struggle to guarantee the spectral reconstruction and spectral feature extraction performance of computational spectral imaging and spectral analysis systems in specific spectral application scenarios.

[0009] This invention provides a method for screening spectral modulation microstructures based on broadband measurement bases, comprising: acquiring the transmission spectra of multiple candidate spectral modulation microstructures; determining a sparse matrix based on the spectral application scenario; the sparse matrix being used to characterize spectral prior information related to the spectral application scenario; and performing inverse design based on the sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures, iteratively screening at least one target spectral modulation microstructure from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures; wherein, the inverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix, and the sensing matrix is ​​determined based on the sparse matrix; the objective function is determined based on the following steps: determining the measurement matrix; the measurement matrix being constructed based on the transmission spectra of multiple candidate spectral modulation microstructures; determining the sensing matrix based on the measurement matrix and the sparse matrix; the sensing matrix including multiple columns, each column representing a broadband measurement base; and determining the objective function based on each broadband measurement base of the sensing matrix; wherein, the objective function is determined based on the maximum column coherence between the broadband measurement bases in the sensing matrix.

[0010] According to the present invention, a method for screening spectral modulation microstructures based on a broadband measurement basis is provided. Based on a sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures, the method performs inverse design according to an objective function, iteratively screening at least one target spectral modulation microstructure from multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. The method further includes: if the spectral application scenario is a spectral reconstruction scenario, acquiring the incident light of the object under test; determining the detector measurement value of the incident light under each target spectral modulation microstructure; and reconstructing the incident spectrum of the incident light based on the detector measurement value of the incident light under each target spectral modulation microstructure.

[0011] According to the present invention, a method for screening spectral modulation microstructures based on a broadband measurement basis reconstructs the incident spectrum of the incident light based on the detector measurement values ​​of the incident light under each target spectral modulation microstructure. The method includes: determining the detector measurement error of each target spectral modulation microstructure; reconstructing a sparse vector based on the detector measurement values ​​of the incident light under each target spectral modulation microstructure, the sensing matrix, and the detector measurement error of each target spectral modulation microstructure, according to compressed sensing theory; and reconstructing the incident spectrum of the incident light based on the sparse vector.

[0012] According to the present invention, a method for screening spectral modulation microstructures based on a broadband measurement basis is provided. Based on a sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures, and through inverse design according to an objective function, at least one target spectral modulation microstructure is iteratively screened from multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. The method further includes: if the spectral application scenario is a material identification and classification scenario, then collecting the incident light of the object to be tested; determining the detector measurement value of the incident light under each target spectral modulation microstructure; and based on the detector measurement value of the incident light under each target spectral modulation microstructure, performing material identification and classification on the object to be tested to determine the material category of the object to be tested.

[0013] This invention also provides a spectral modulation microstructure screening device based on a broadband measurement basis, comprising: a microstructure transmission spectrum solving unit for obtaining the transmission spectra of multiple candidate spectral modulation microstructures; a sparse matrix solving unit for determining a sparse matrix based on the spectral application scenario; the sparse matrix being used to characterize spectral prior information related to the spectral application scenario; and a transmission spectrum iterative screening unit for performing inverse design based on the sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures according to an objective function, iteratively screening at least one target spectral modulation microstructure from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. The reverse design process aims to minimize an objective function, which is determined based on the maximum column coherence of the sensing matrix, which is based on a sparse matrix. The objective function is determined by the following steps: determining the measurement matrix; constructing the measurement matrix based on the transmission spectra of multiple candidate spectral modulation microstructures; determining the sensing matrix based on the measurement matrix and the sparse matrix; the sensing matrix includes multiple columns, each representing a broadband measurement basis; and determining the objective function based on each broadband measurement basis of the sensing matrix. The objective function is determined based on the maximum column coherence between the broadband measurement bases in the sensing matrix.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described methods for screening spectral modulation microstructures based on a broadband measurement basis.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for screening spectral modulation microstructures based on a broadband measurement basis.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described methods for screening spectral modulation microstructures based on a broadband measurement basis.

[0017] The present invention also provides a computational spectral imaging and spectral analysis system, comprising an optical lens, a spectral modulation layer, an image sensor layer, a circuit readout layer, and a computing unit arranged sequentially. The spectral modulation layer includes at least one superpixel unit, and each superpixel unit includes at least one broadband modulation microstructure. The broadband modulation microstructure is a target spectral modulation microstructure using broadband modulation, and the target spectral modulation microstructure is obtained by screening using any of the above-mentioned spectral modulation microstructure screening methods based on broadband measurement basis.

[0018] The present invention provides a method, apparatus, device, medium, program product, and computational spectral imaging and spectral analysis system for screening spectral modulation microstructures based on a broadband measurement basis. First, a sparse matrix is ​​determined according to the spectral application scenario. The sparse matrix is ​​used to characterize spectral prior information related to the spectral application scenario. A sensing matrix is ​​then determined based on the sparse matrix, and an objective function is constructed based on the maximum column coherence of the sensing matrix. This introduces spectral prior information related to the spectral application scenario into the objective function. Finally, based on the sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures, inverse design is performed according to the objective function to iteratively screen at least one target spectral modulation microstructure from multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. Because spectral prior information related to the spectral application scenario is introduced into the objective function beforehand, the optimal combination of spectral modulation microstructures obtained by screening has… The system's adaptability to specific spectral application scenarios helps ensure the spectral reconstruction and feature extraction performance of the subsequent computational spectral imaging and analysis system in real-world spectral applications. Furthermore, the reverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix. Since a lower maximum column coherence of the sensing matrix results in stronger noise resistance, selecting spectral modulation microstructures with this objective improves the noise resistance of the system, reduces noise interference during spectral imaging and analysis, and further guarantees the spectral reconstruction and feature extraction performance of the computational spectral imaging and analysis system. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the spectral modulation microstructure screening method based on a broadband measurement basis provided by the present invention.

[0021] Figure 2 This is a schematic diagram of the computational spectral imaging and spectral analysis system provided by the present invention.

[0022] Figure 3 This is a schematic diagram of the spectral modulation microstructure screening device based on a broadband measurement basis provided by the present invention.

[0023] Figure 4This is one of the schematic diagrams of the working process of the spectral modulation microstructure screening device based on broadband measurement basis provided by the present invention.

[0024] Figure 5 This is the second schematic diagram of the workflow of the spectral modulation microstructure screening device based on broadband measurement basis provided by the present invention.

[0025] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] Please see Figures 1 to 2 , Figure 1 This is a flowchart illustrating the spectral modulation microstructure screening method based on a broadband measurement basis provided by the present invention. Figure 2 This is a schematic diagram of the computational spectral imaging and spectral analysis system provided by the present invention.

[0028] like Figure 1 As shown, the spectral modulation microstructure screening method based on broadband measurement basis includes steps S110 to S130, and the specific details of each step are as follows:

[0029] S110: Obtain the transmission spectra of multiple candidate spectral modulation microstructures.

[0030] S120: Determine the sparse matrix based on the spectral application scenario.

[0031] Sparse matrices are used to characterize spectral prior information relevant to spectral application scenarios.

[0032] S130: Based on the transmission spectrum of a sparse matrix and multiple candidate spectral modulation microstructures, reverse design is performed according to the objective function to iteratively select at least one target spectral modulation microstructure from multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures.

[0033] The reverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix, which is determined based on the sparse matrix.

[0034] The objective function is determined based on the following steps: determining the measurement matrix; the measurement matrix is ​​constructed based on the transmission spectra of multiple candidate spectral modulation microstructures; determining the sensing matrix based on the measurement matrix and the sparse matrix; the sensing matrix includes multiple columns, each column representing a broadband measurement basis; determining the objective function based on each broadband measurement basis of the sensing matrix; wherein the objective function is determined based on the maximum column coherence among the broadband measurement basis in the sensing matrix.

[0035] To facilitate understanding, the principle of a computational spectral imaging and spectral analysis system based on broadband modulation will be introduced here first.

[0036] like Figure 2 As shown, the computational spectral imaging and spectral analysis system based on broadband modulation includes an optical lens, a spectral modulation layer, an image sensor layer, a circuit readout layer, and a computing unit arranged sequentially.

[0037] The spectral modulation layer includes at least one superpixel unit, and each superpixel unit includes multiple different broadband modulation microstructures. The broadband modulation microstructures are spectral modulation microstructures using broadband modulation, and the spectral modulation microstructures can be realized by various materials or components such as dye filters, multilayer interference films, subwavelength gratings, and metasurfaces.

[0038] Spectroscopic applications mainly include spectral reconstruction and material identification and classification. In practical spectral applications, optical lenses transmit the incident light from the object under test to the spectral modulation layer. Each superpixel unit in the spectral modulation layer can achieve different spectral modulations on the light incident on that unit through different spectral modulation microstructures. The image sensor layer is adjacent to the spectral modulation layer and can detect the spectrum modulated by each spectral modulation microstructure, obtaining the integral result of each spectral modulation microstructure with respect to the light intensity of each wavelength, which is used as the detector measurement value of the incident light under each spectral modulation microstructure. The circuit readout layer can use the detector measurement value of the incident light under each spectral modulation microstructure as the detection result and transmit the detection result to the computing unit. The computing unit can perform spectral reconstruction or execute material identification and classification tasks based on spectral features according to the detection results.

[0039] In practical spectral applications, the principle of broadband modulation can be expressed by the following formula:

[0040] ;

[0041] in, For one The vector, This indicates the number of spectrally modulated microstructures contained in a superpixel unit. This represents the detector measurement value of the incident light under each spectral modulation microstructure in the superpixel unit; For one The matrix, Indicates the number of spectral channels. Represents the measurement matrix, measurement matrix It is constructed based on the transmission spectra of different spectral modulation microstructures, and the measurement matrix is... Each row in the table represents the transmission spectrum of a spectral modulation microstructure within a superpixel unit; For one The vector, This represents the incident spectrum of the light incident on the superpixel unit; For one The vector, This represents the detector measurement error for each spectral modulation microstructure.

[0042] Taking the spectral reconstruction scenario as an example, after obtaining the detection results... Then, based on the detection results and the known measurement matrix Reconstructing the incident spectrum However, in practical spectral applications, Typically much smaller (i.e., satisfy) This leads to an undetermined problem in the reconstruction of the incident spectrum, which is not conducive to solving the spectral reconstruction problem. Therefore, it is necessary to introduce prior knowledge of the incident spectrum into the solution process.

[0043] According to compressed sensing theory, the incident spectrum It can be expressed by the following formula:

[0044] ;

[0045] in, For one A matrix of these is called a sparse matrix, which is used to represent spectral prior information related to spectral application scenarios. For one A vector with a certain density is called a sparse vector; This represents the number of channels in the sparse region, which varies depending on the application of the spectrum. The value can be very small, for example , , The value can also be very large, for example .

[0046] Compressed sensing theory states that in sparse vectors of The number of non-zero elements in the array is much smaller than that of the array. Then the above incident spectrum The physical meaning of the formula is: incident spectrum It can be done through sparse matrices The mapping is transformed into a sparse vector. The representation, due to sparse vectors The number of non-zero elements is much smaller than Therefore, the number of unknowns in the solution process can be greatly reduced.

[0047] Based on the two formulas above, the following formula can be derived:

[0048] ;

[0049] in, This is called the sensing matrix, which is composed of sparse matrices. and measurement matrix It's confirmed.

[0050] According to the above formula, the incident spectrum The reconstruction process can be transformed into: first, based on the detection results... and the known sensing matrix (Because of the measurement matrix) sparse matrix All are known quantities, so the sensing matrix (Also known quantities), reconstructing sparse vectors In reconstructing sparse vectors Then, according to the formula Reconstructing the incident spectrum Due to sparse vectors The number of non-zero elements is much smaller than The spectral reconstruction problem is actually an overdetermined problem, which can be effectively solved using various algorithms such as convex optimization and neural networks.

[0051] Sparse matrix Used to characterize spectral prior information related to spectral application scenarios, sparse matrices are employed in different spectral application scenarios. They differ. Traditional methods only consider the measurement matrix when designing the objective function. The correlation of the transmission spectrum does not consider the sparse matrix. The prior conditions represented cannot guarantee that computational spectral imaging and spectral analysis systems can achieve optimal performance in various practical spectral application scenarios.

[0052] Based on this, this embodiment proposes a reverse design method for spectral modulation microstructures based on broadband measurement basis. It introduces spectral prior information related to spectral application scenarios, uses broadband measurement basis theory to evaluate the spectral feature extraction performance of the transmission spectrum combination of spectral modulation microstructures, and designs a new objective function, thereby providing a theoretical basis for the screening of spectral modulation microstructures.

[0053] Specifically, for sparse vectors The reconstruction process, its reconstruction accuracy is determined by the sensor matrix. The characteristics determine that the sensing matrix It includes multiple columns, each column representing a broadband measurement basis, and the sensing matrix. The multiple columns can be represented as follows: , Represents the sensing matrix The A broadband measurement basis, sparse vector The reconstruction process is actually equivalent to using a broadband measurement base. Establish sparse vectors With the detection results The mapping.

[0054] In sparse vectors In spectral applications with only one non-zero element, assuming sparse vectors The middle of The elements are non-zero elements, and their corresponding detection results It can be represented as:

[0055] ;

[0056] in, For the sensor matrix in the above scenario The value of .

[0057] When sparse vectors When the characteristics change, The value will also change, in order to accurately distinguish different... Sparse vectors under value The values ​​are different The detection result corresponding to the value They should be linearly independent, and the detection results The lower the coherence between them, the lower the noise that the computational spectral imaging and analysis system can tolerate. The higher the detector measurement error (i.e., the higher the noise resistance) of each spectral modulation microstructure, the stronger the noise resistance.

[0058] Based on the above theory, in the design process of the objective function in this embodiment, the sparse matrix is ​​first determined according to the spectral application scenario. sparse matrix It is a fixed quantity determined by the application scenario of spectroscopy, used to characterize the spectral prior information related to the application scenario of spectroscopy.

[0059] Furthermore, due to Then, based on the sparse matrix Determine the sensing matrix And based on the sensing matrix The objective function is constructed based on the maximum column coherence. Thus in the objective function The paper incorporates spectral prior information relevant to spectral application scenarios.

[0060] Wherein, objective function The expression is as follows:

[0061] ;

[0062] in, For one The matrix, Indicates the number of spectral channels. Represents the measurement matrix, measurement matrix It is constructed based on the transmission spectra of multiple candidate spectral modulation microstructures, and the measurement matrix is... Each row in the table represents the transmission spectrum of a candidate spectral modulation microstructure; For one A matrix of these is called a sparse matrix, which is used to represent spectral prior information related to spectral application scenarios. This is a function to find the maximum value. and sparse vector Two distinct non-zero elements in; The sensing matrix, the sensing matrix From sparse matrix and measurement matrix The sensing matrix is ​​obtained through dot product. It includes multiple columns, with each column representing a broadband measurement basis; Represents the sensing matrix The A broadband measurement base; Represents the sensing matrix The A broadband measurement base; and They are and The L2 norm; For the objective function The core of this constraint is to incorporate spectral prior information (i.e., sparse matrix) related to the spectral application scenario. This is directly related to the design of spectral modulation microstructures.

[0063] The above objective function The physical meaning is: for the sensing matrix Any two columns (denoted as) and The objective function is to maximize the column coherence of the two broadband measurement bases represented by the given information. What is actually being calculated is the sensing matrix in a specific spectral application scenario. Maximum column coherence among various broadband measurement bases.

[0064] The reverse design process (i.e., the screening process for spectral modulation microstructures) should aim to minimize the aforementioned objective function; that is, the reverse design process should minimize the sensing matrix. The maximum column coherence is the optimization objective. Since the lower the maximum column coherence of the sensing matrix, the better the sensing matrix... The higher the discriminative power of different columns (different columns correspond to different sparse features), the stronger the noise resistance of the computational spectral imaging and spectral analysis system. Therefore, selecting spectral modulation microstructures with minimizing the maximum column coherence of the sensing matrix as the optimization objective is beneficial to improving the noise resistance of the computational spectral imaging and spectral analysis system, reducing noise interference in the spectral imaging and spectral analysis processes, and further ensuring the spectral reconstruction performance and spectral feature extraction performance of the computational spectral imaging and spectral analysis system.

[0065] Furthermore, in determining the objective function Then, based on this objective function Perform reverse engineering.

[0066] Specifically, the transmission spectra of all candidate spectral modulation microstructures are first obtained from the transmission spectrum database, and then a suitable sparse matrix is ​​determined according to the spectral application scenario. This is to provide prior spectral information relevant to spectral application scenarios.

[0067] Furthermore, based on sparse matrices And the transmission spectra of multiple candidate spectral modulation microstructures, based on the objective function Reverse engineering is performed, and multiple candidate spectral modulation microstructures are iteratively screened using a pre-defined algorithm. The pre-defined algorithm aims to minimize the objective function. To optimize the objective, and thus select the options that make the objective function... The optimal transmission spectrum combination that achieves the minimum value (i.e., the optimal measurement matrix) And, each candidate spectral modulation microstructure corresponding to the optimal transmission spectrum combination is taken as the target spectral modulation microstructure, to obtain the optimal spectral modulation microstructure combination.

[0068] Optionally, the preset algorithm includes, but is not limited to, genetic algorithm, simulated annealing algorithm, etc.

[0069] It should be noted that in the above iterative screening process, due to That is, the sensing matrix Only by sparse matrix and measurement matrix Determined, while sparse matrix It is a fixed quantity, therefore the actual reverse design process only applies to the measurement matrix. The actual reverse design process involves selecting several candidate spectral modulation microstructures from all available candidates and combining their transmission spectra to obtain a measurement matrix. According to this measurement matrix Calculate whether the current combination of candidate spectral modulation microstructures can satisfy the objective function. If the minimum value cannot be obtained, repeat the above filtering and calculation steps until the objective function is found. The optimal transmission spectrum combination that achieves the minimum value.

[0070] Furthermore, after obtaining the optimal combination of spectral modulation microstructures, this optimal combination can be integrated into the spectral modulation layer of a computational spectral imaging and spectral analysis system for application in subsequent spectral reconstruction tasks or material identification and classification tasks, utilizing the aforementioned objective function. The optimal combination of spectral modulation microstructures obtained through screening enables computational spectral imaging and spectral analysis systems to effectively characterize actual spectral reconstruction scenarios or material identification and classification scenarios based on spectral features.

[0071] The spectral modulation microstructure screening method based on a broadband measurement basis provided in this embodiment first determines a sparse matrix according to the spectral application scenario. The sparse matrix is ​​used to characterize the spectral prior information related to the spectral application scenario. A sensing matrix is ​​then determined based on the sparse matrix, and an objective function is constructed based on the maximum column coherence of the sensing matrix. This introduces spectral prior information related to the spectral application scenario into the objective function. Finally, based on the sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures, inverse design is performed according to the objective function to iteratively screen at least one target spectral modulation microstructure from the multiple candidate spectral modulation microstructures as the optimal spectral modulation microstructure combination. Because spectral prior information related to the spectral application scenario is introduced into the objective function beforehand, the selected optimal spectral modulation microstructure combination possesses the capability to perform tasks specific to the spectral application scenario. Adaptability helps ensure the spectral reconstruction and feature extraction performance of the subsequent computational spectral imaging and analysis system in real-world spectral application scenarios. Simultaneously, the reverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix. This means the reverse design process optimizes by minimizing the maximum column coherence of the sensing matrix. Since a lower maximum column coherence of the sensing matrix results in stronger noise resistance for the computational spectral imaging and analysis system, minimizing the maximum column coherence of the sensing matrix during the selection of spectral modulation microstructures improves the noise resistance of the computational spectral imaging and analysis system, reduces noise interference during spectral imaging and analysis, and further ensures the spectral reconstruction and feature extraction performance of the computational spectral imaging and analysis system.

[0072] In some embodiments, based on the transmission spectra of a sparse matrix and multiple candidate spectral modulation microstructures, and after inverse design according to an objective function, at least one target spectral modulation microstructure is iteratively selected from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures, the method further includes: if the spectral application scenario is a spectral reconstruction scenario, then acquiring the incident light of the object under test; determining the detector measurement value of the incident light under each target spectral modulation microstructure; and reconstructing the incident spectrum of the incident light based on the detector measurement value of the incident light under each target spectral modulation microstructure.

[0073] Specifically, after determining the optimal combination of spectral modulation microstructures, this optimal combination of spectral modulation microstructures can be integrated into the spectral modulation layer of the computational spectral imaging and spectral analysis system.

[0074] Furthermore, in the use of computational spectral imaging and spectral analysis systems, if the actual spectral application scenario is a spectral reconstruction scenario, the incident light of the object under test can be collected through an optical lens.

[0075] Furthermore, by using the image sensor layer, the detector measurements of the incident light under each target spectral modulation microstructure are determined, thus obtaining the detection results. Detection results For one The vector, This indicates the number of target spectral modulation microstructures.

[0076] Furthermore, based on the detector measurements (i.e., detection results) of the incident light under the spectral modulation microstructure of each target. ), to reconstruct the incident spectrum of the incident light.

[0077] In some embodiments, reconstructing the incident spectrum of the incident light based on the detector measurements of the incident light under each target spectral modulation microstructure includes: determining the detector measurement error of each target spectral modulation microstructure; reconstructing a sparse vector based on the detector measurements of the incident light under each target spectral modulation microstructure, the sensing matrix, and the detector measurement error of each target spectral modulation microstructure, according to compressed sensing theory; and reconstructing the incident spectrum of the incident light based on the sparse vector.

[0078] Specifically, in determining the detector measurements (i.e., detection results) of the incident light under the spectral modulation microstructure of each target. Then, based on the transmission spectrum of each target spectral modulation microstructure in the optimal spectral modulation microstructure combination, the measurement matrix corresponding to the optimal spectral modulation microstructure combination is determined. Then, based on the measurement matrix corresponding to the optimal spectral modulation microstructure combination... and sparse matrices suitable for spectral reconstruction scenarios Through formula Calculate the sensing matrix .

[0079] Furthermore, based on the detection results Sensing matrix and the detector measurement error of each target spectral modulation microstructure Through formula Calculate the sparse vector .

[0080] Furthermore, based on the sparse matrix Sparse vectors Through formula The incident spectrum of the incident light was reconstructed. .

[0081] For example, in single-peak spectrum reconstruction scenarios, sparse matrices A diagonal matrix with sparse vectors With only one non-zero element, which represents the location and intensity of the single peak, the objective function... The physical meaning can be interpreted as: selecting the optimal combination of transmission spectra (i.e., the optimal measurement matrix). This ensures that the coherence between detector measurements under different spectral modulation microstructures and single-peak inputs at different wavelengths is as low as possible.

[0082] In some embodiments, based on the transmission spectra of a sparse matrix and multiple candidate spectral modulation microstructures, and after inverse design according to an objective function, at least one target spectral modulation microstructure is iteratively selected from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. The method further includes: if the spectral application scenario is a material identification and classification scenario, then collecting the incident light of the object to be tested; determining the detector measurement value of the incident light under each target spectral modulation microstructure; and based on the detector measurement value of the incident light under each target spectral modulation microstructure, performing material identification and classification on the object to be tested to determine the material category of the object to be tested.

[0083] Specifically, after determining the optimal combination of spectral modulation microstructures, this optimal combination of spectral modulation microstructures can be integrated into the spectral modulation layer of the computational spectral imaging and spectral analysis system.

[0084] Furthermore, in the use of computational spectral imaging and spectral analysis systems, if the actual spectral application scenario is a material identification and classification scenario, the incident light of the object to be tested can be collected through an optical lens.

[0085] Furthermore, by using the image sensor layer, the detector measurements of the incident light under each target spectral modulation microstructure are determined, thus obtaining the detection results. Detection results For one The vector, This indicates the number of target spectral modulation microstructures.

[0086] Furthermore, based on the detector measurements (i.e., detection results) of the incident light under the spectral modulation microstructure of each target. The test object is identified and classified to determine its material category.

[0087] For example, in the scenario of substance identification and classification based on spectral features, sparse vectors A sparse matrix contains only one non-zero element, which represents the possible material category of the object being measured. Each row represents the spectral curve corresponding to the object being measured, where the objective function is... The physical meaning can be interpreted as: selecting the optimal combination of transmission spectra (i.e., the optimal measurement matrix). This ensures that the coherence between detector measurements under different spectral modulation microstructures is as low as possible when the system faces incident light from different types of objects to be measured.

[0088] The spectral modulation microstructure screening method based on a broadband measurement basis provided in this embodiment transforms the objective function from "mathematical structure optimization" to "task performance-driven optimization" by introducing spectral prior information related to the spectral application scenario during the reverse design stage. This contrasts with existing technologies that rely solely on transmission spectrum combinations (i.e., measurement matrices). The selection method for constructing the objective function based on row coherence or column coherence differs from that used in this embodiment. The objective function constructed in this embodiment can directly reflect the spectral feature discrimination capability of the spectral modulation microstructure combination in actual spectral reconstruction scenarios or material identification and classification scenarios. This makes the reverse design process closely related to the actual performance of the computational spectral imaging and spectral analysis system. Thus, while ensuring the feasibility of the spectral modulation microstructure combination, it improves the spectral reconstruction resolution of the computational spectral imaging and spectral analysis system, reduces reconstruction errors, and enhances the robustness of the system under low signal-to-noise ratio conditions.

[0089] Furthermore, the spectral modulation microstructure screening method based on a broadband measurement basis provided in this embodiment has good versatility and scalability, and can be adapted to a variety of spectral modulation microstructures (spectral modulation microstructures include, but are not limited to, dye filters, multilayer interference films, metasurfaces, etc.), and the sparse matrix can be flexibly replaced according to different spectral application scenarios. This enables computational spectral imaging and spectral analysis systems to perform high-resolution detection of incident light in specific wavelength bands, broadband feature recognition, and classification of specific target substances. Addressing the problem that traditional forward or reverse design methods cannot take into account the differences in actual spectral application scenarios, the spectral modulation microstructure screening method in this embodiment can automatically adapt to the spectral statistical characteristics of different spectral application scenarios, thereby improving the task adaptability of spectral modulation microstructure transmission spectrum combinations.

[0090] In summary, the spectral modulation microstructure screening method based on broadband measurement basis provided in this embodiment not only improves the actual performance limit of spectral modulation microstructure combinations, but also provides a unified, scalable, task-driven spectral modulation microstructure design paradigm, laying the foundation for the application of miniaturized, snapshot-like spectral imaging systems in precision detection, biosensing, and intelligent recognition.

[0091] This invention also provides a spectral modulation microstructure screening device based on a broadband measurement basis. Please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram of the spectral modulation microstructure screening device based on a broadband measurement basis provided by the present invention. Figure 3As shown, in this embodiment, the spectral modulation microstructure screening device based on broadband measurement basis includes a microstructure transmission spectrum solving unit, a sparse matrix solving unit, and a transmission spectrum iterative screening unit.

[0092] The microstructure transmission spectrum solving unit is used to obtain the transmission spectra of multiple candidate spectral modulation microstructures.

[0093] The sparse matrix solving unit is used to determine the sparse matrix based on spectral application scenarios.

[0094] Sparse matrices are used to characterize spectral prior information relevant to spectral application scenarios.

[0095] The transmission spectrum iterative screening unit is used to perform inverse design based on the transmission spectrum of sparse matrix and multiple candidate spectral modulation microstructures according to the objective function, and iteratively screen at least one target spectral modulation microstructure from multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures.

[0096] The reverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix, which is determined based on the sparse matrix.

[0097] The objective function is determined based on the following steps: determining the measurement matrix; the measurement matrix is ​​constructed based on the transmission spectra of multiple candidate spectral modulation microstructures; determining the sensing matrix based on the measurement matrix and the sparse matrix; the sensing matrix includes multiple columns, each column representing a broadband measurement basis; determining the objective function based on each broadband measurement basis of the sensing matrix; wherein the objective function is determined based on the maximum column coherence among the broadband measurement basis in the sensing matrix.

[0098] In some embodiments, based on the transmission spectra of a sparse matrix and multiple candidate spectral modulation microstructures, and after inverse design according to an objective function, at least one target spectral modulation microstructure is iteratively selected from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures, the method further includes: if the spectral application scenario is a spectral reconstruction scenario, then acquiring the incident light of the object under test; determining the detector measurement value of the incident light under each target spectral modulation microstructure; and reconstructing the incident spectrum of the incident light based on the detector measurement value of the incident light under each target spectral modulation microstructure.

[0099] In some embodiments, reconstructing the incident spectrum of the incident light based on the detector measurements of the incident light under each target spectral modulation microstructure includes: determining the detector measurement error of each target spectral modulation microstructure; reconstructing a sparse vector based on the detector measurements of the incident light under each target spectral modulation microstructure, the sensing matrix, and the detector measurement error of each target spectral modulation microstructure, according to compressed sensing theory; and reconstructing the incident spectrum of the incident light based on the sparse vector.

[0100] In some embodiments, based on the transmission spectra of a sparse matrix and multiple candidate spectral modulation microstructures, and after inverse design according to an objective function, at least one target spectral modulation microstructure is iteratively selected from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. The method further includes: if the spectral application scenario is a material identification and classification scenario, then collecting the incident light of the object to be tested; determining the detector measurement value of the incident light under each target spectral modulation microstructure; and based on the detector measurement value of the incident light under each target spectral modulation microstructure, performing material identification and classification on the object to be tested to determine the material category of the object to be tested.

[0101] Specifically, in the workflow of the spectral modulation microstructure screening device, the transmission spectra of all candidate spectral modulation microstructures can first be obtained from the transmission spectrum database through the microstructure transmission spectrum solving unit, so as to subsequently construct the measurement matrix. This establishes the solution space.

[0102] Optionally, the transmission spectrum of different candidate spectral modulation microstructures can be obtained in different ways.

[0103] For example, if the candidate spectral modulation microstructure is a dye filter, its transmission spectrum can be determined by material selection and experimental measurement; if the candidate spectral modulation microstructure is a multilayer interference film or a subwavelength grating, its transmission spectrum can be determined by parameter scanning and electromagnetic simulation; if the candidate spectral modulation microstructure is a metasurface, its transmission spectrum can be determined by electromagnetic simulation or neural network prediction.

[0104] Furthermore, through the sparse matrix solving unit, a sparse matrix suitable for the actual spectral application scenario is selected. This is to provide prior spectral information relevant to spectral application scenarios.

[0105] Optionally, sparse matrix It can be determined through various methods, such as empirically designed methods and neural network prediction methods based on iterative training of datasets.

[0106] Furthermore, the transmission spectra of all candidate spectra provided by the microstructure transmission spectrum solving unit and the sparse matrix provided by the sparse matrix solving unit are used to modulate the transmission spectra of the microstructure. As input data to the transmission spectrum iterative screening unit, the transmission spectrum iterative screening unit can determine the objective function based on the broadband measurement basis theory. As an optimization objective, the optimal combination of spectral modulation microstructures is selected iteratively from all candidate spectral modulation microstructures through reverse design.

[0107] Optionally, the reverse design process can be implemented using a variety of preset algorithms, including but not limited to genetic algorithms and simulated annealing algorithms.

[0108] To facilitate understanding, this invention also provides two examples of using a spectral modulation microstructure screening device based on a broadband measurement basis. Please refer to... Figure 4 and Figure 5 , Figure 4 This is one of the schematic diagrams illustrating the workflow of the spectral modulation microstructure screening device based on a broadband measurement basis provided by the present invention. Figure 5 This is the second schematic diagram of the workflow of the spectral modulation microstructure screening device based on broadband measurement basis provided by the present invention.

[0109] like Figure 4 As shown, for the visible spectrum reconstruction scenario, the candidate spectral modulation microstructure can be a metasurface. The microstructure transmission spectrum solving unit can obtain the transmission spectra of all metasurface microstructures from the transmission spectrum database. The transmission spectra of the metasurface microstructure can be obtained through simulation of the random structure of the metasurface. The sparse matrix solving unit can be iteratively trained using the visible spectrum dataset, and after training, predict the sparse matrix suitable for the visible spectrum reconstruction scenario. The transmission spectra of all metasurface microstructures provided by the microstructure transmission spectrum solution unit and the sparse matrix provided by the sparse matrix solution unit are used to solve the problem. As input data to the transmission spectrum iterative screening unit, the transmission spectrum iterative screening unit can determine the objective function based on the broadband measurement basis theory. As an optimization objective, the optimal combination of spectral modulation microstructures suitable for visible spectrum reconstruction is selected from all metasurface microstructures through reverse design and iterative screening.

[0110] like Figure 5 As shown, for fluorescence recognition scenarios, candidate spectral modulation microstructures can be dye microstructures. The microstructure transmission spectrum solving unit can obtain the transmission spectra of all dye microstructures from the transmission spectrum database. The transmission spectra of dye microstructures can be obtained through actual measurements of dye units. The sparse matrix solving unit can directly combine the identified fluorescence spectra to obtain a sparse matrix suitable for fluorescence recognition scenarios. The transmission spectra of all dye microstructures provided by the microstructure transmission spectrum solution unit and the sparse matrix provided by the sparse matrix solution unit are used to solve the problem. As input data to the transmission spectrum iterative screening unit, the transmission spectrum iterative screening unit can determine the objective function based on the broadband measurement basis theory. As an optimization objective, the optimal combination of spectral modulation microstructures suitable for fluorescence recognition scenarios is selected through reverse design and iterative screening from all dye microstructures.

[0111] The present invention also provides an electronic device. Figure 6This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640. The processor 610, communication interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logic instructions from the memory 630 to execute a spectral modulation microstructure screening method based on a broadband measurement basis.

[0112] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spectral modulation microstructure screening method based on a broadband measurement basis provided by the above methods.

[0114] The present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the spectral modulation microstructure screening method based on a broadband measurement basis provided by the above methods.

[0115] The present invention also provides a computational spectral imaging and spectral analysis system, comprising an optical lens, a spectral modulation layer, an image sensor layer, a circuit readout layer, and a computing unit arranged sequentially. The spectral modulation layer includes at least one superpixel unit, and each superpixel unit includes at least one broadband modulation microstructure. The broadband modulation microstructure is a target spectral modulation microstructure using broadband modulation, and the target spectral modulation microstructure is obtained by screening using the spectral modulation microstructure screening method based on broadband measurement basis provided by the above methods.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for screening spectral modulation microstructures based on a broadband measurement basis, characterized in that, include: Obtain the transmission spectra of multiple candidate spectral modulation microstructures; Based on the application scenario of spectroscopy, determine the sparse matrix; The sparse matrix is ​​used to characterize spectral prior information related to the spectral application scenario; Based on the sparse matrix and the transmission spectra of the multiple candidate spectral modulation microstructures, reverse design is performed according to the objective function to iteratively select at least one target spectral modulation microstructure from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. The reverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix, and the sensing matrix is ​​determined based on the sparse matrix. The objective function is determined based on the following steps: Determine the measurement matrix; the measurement matrix is ​​constructed based on the transmission spectra of the multiple candidate spectral modulation microstructures; The sensing matrix is ​​determined based on the measurement matrix and the sparse matrix; the sensing matrix includes multiple columns, and each column represents a broadband measurement basis. The objective function is determined based on each of the broadband measurement bases of the sensing matrix; The objective function is determined based on the maximum column coherence among the broadband measurement bases in the sensing matrix.

2. The method for screening spectral modulation microstructures based on a broadband measurement basis according to claim 1, characterized in that, The transmission spectrum based on the sparse matrix and multiple candidate spectral modulation microstructures, after inverse design according to the objective function, iteratively selecting at least one target spectral modulation microstructure from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures, further includes: If the spectral application scenario is a spectral reconstruction scenario, then the incident light of the object to be measured is collected; Determine the detector measurement value of the incident light under each of the target spectral modulation microstructures; The incident spectrum of the incident light is reconstructed based on the detector measurements of the incident light under each of the target spectral modulation microstructures.

3. The method for screening spectral modulation microstructures based on a broadband measurement basis according to claim 2, characterized in that, The process of reconstructing the incident spectrum of the incident light based on detector measurements of the incident light under each of the target spectral modulation microstructures includes: Determine the detector measurement error for each of the target spectral modulation microstructures; Based on the detector measurement value of the incident light under each of the target spectral modulation microstructures, the sensing matrix, and the detector measurement error of each of the target spectral modulation microstructures, a sparse vector is reconstructed according to the compressed sensing theory. Based on the sparse vector, the incident spectrum of the incident light is reconstructed.

4. The method for screening spectral modulation microstructures based on a broadband measurement basis according to claim 1, characterized in that, The transmission spectrum based on the sparse matrix and multiple candidate spectral modulation microstructures, after inverse design according to the objective function, iteratively selecting at least one target spectral modulation microstructure from the multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures, further includes: If the application scenario of the spectrum is a material identification and classification scenario, then the incident light of the object to be tested is collected; Determine the detector measurement value of the incident light under each of the target spectral modulation microstructures; Based on the detector measurements of the incident light under each of the target spectral modulation microstructures, the object under test is identified and classified to determine the material category of the object under test.

5. A spectral modulation microstructure screening device based on a broadband measurement basis, characterized in that, include: The microstructure transmission spectrum solving unit is used to obtain the transmission spectra of multiple candidate spectral modulation microstructures; The sparse matrix solving unit is used to determine the sparse matrix based on spectral application scenarios. The sparse matrix is ​​used to characterize spectral prior information related to the spectral application scenario; The transmission spectrum iterative screening unit is used to perform inverse design based on the sparse matrix and the transmission spectra of multiple candidate spectral modulation microstructures according to the objective function, and iteratively screen at least one target spectral modulation microstructure from multiple candidate spectral modulation microstructures as the optimal combination of spectral modulation microstructures. The reverse design process aims to minimize the objective function, which is determined based on the maximum column coherence of the sensing matrix, and the sensing matrix is ​​determined based on the sparse matrix. The objective function is determined based on the following steps: Determine the measurement matrix; the measurement matrix is ​​constructed based on the transmission spectra of the multiple candidate spectral modulation microstructures; The sensing matrix is ​​determined based on the measurement matrix and the sparse matrix; the sensing matrix includes multiple columns, and each column represents a broadband measurement basis. The objective function is determined based on each of the broadband measurement bases of the sensing matrix; The objective function is determined based on the maximum column coherence among the broadband measurement bases in the sensing matrix.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the spectral modulation microstructure screening method based on broadband measurement basis as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the spectral modulation microstructure screening method based on broadband measurement basis as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the spectral modulation microstructure screening method based on broadband measurement basis as described in any one of claims 1 to 4.

9. A computational spectral imaging and spectral analysis system, characterized in that, The device includes an optical lens, a spectral modulation layer, an image sensor layer, a circuit readout layer, and a computing unit arranged sequentially. The spectral modulation layer includes at least one superpixel unit, and each superpixel unit includes at least one broadband modulation microstructure. The broadband modulation microstructure is a target spectral modulation microstructure using broadband modulation, and the target spectral modulation microstructure is obtained by screening using the spectral modulation microstructure screening method based on broadband measurement basis as described in any one of claims 1 to 4.

Images