An integrated error suppression method for on-chip spectral imaging system with combined software and hardware optimization

Abstract

The present invention specifically discloses a method for suppressing integrated errors of an on-chip spectral imaging system with combined software and hardware optimization, which relates to the technical field of spectral imaging. The error suppression method includes: S1. Design the inner square unit structure of the coded filter so that it corresponds one by one to the detector pixels, and prepare the filter through semiconductor processes; S2. Jointly optimize the filter structure parameters and the spectral reconstruction model through an end-to-end deep learning framework to suppress registration errors; S3. Use the simulated annealing algorithm to optimize the filter arrangement pattern to minimize the spectral differences between adjacent filters. The method for suppressing integrated errors of an on-chip spectral imaging system with combined software and hardware optimization proposed by the present invention significantly reduces the influence of registration errors on the spectral imaging system while ensuring the compact integration of the system, and provides a new solution for high-precision applications of on-chip computational spectral imaging.

Description

Technical Field

[0001] The present invention relates to the field of spectral imaging technology, and in particular to a method for suppressing integration errors of an on-chip spectral imaging system with combined software and hardware optimization. Background Art

[0002] Spectral imaging technology has a wide range of applications in scientific research and daily life, covering multiple fields. For example, spectral imaging technology is used in aspects of life production such as art and cultural relic identification, drug ingredient analysis, fruit and vegetable sorting, natural disaster monitoring and early warning.

[0003] With the rapid development of spectral imaging technology, the on-chip computational spectral imaging system has received extensive attention from researchers at home and abroad due to its high spectral resolution, compact integration and low power consumption characteristics, and has gradually become an important direction for the development of spectral imaging technology. Compared with the traditional spectral imaging system, the on-chip computational spectral imaging technology relies on the integration of a coded filter array and a detector, detects the compressed information of spectral aliasing on the detector, and then realizes it through a reconstruction algorithm, which can achieve efficient acquisition and reconstruction of multi-spectral or hyperspectral information. However, in practical applications, due to manufacturing process limitations, there is inevitably a registration error between the coded filter and the detector pixels, resulting in spectral aliasing, decreased spatial resolution and information loss during the spectral data acquisition process, thereby affecting the spectral imaging quality and the accuracy of data analysis.

[0004] Currently, the research on the on-chip computational spectral imaging system mainly focuses on optical design optimization and computational reconstruction methods to improve the spectral imaging quality of the system. The optical optimization method mainly aims to improve the manufacturing accuracy of the coded filter or design a more stable optical structure to optimize the spectral modulation characteristics; the computational reconstruction method widely applies technologies such as deep learning and optimization algorithms to improve the reconstruction ability of spectral data. However, existing research pays more attention to the improvement of spectral reconstruction accuracy, and the problem of registration error between the coded filter and the detector pixels has not been systematically studied and solved. Due to factors such as manufacturing process, assembly accuracy and device non-uniformity, this error inevitably exists in the on-chip computational spectral imaging system and may lead to aggravated spectral aliasing, decreased spatial resolution and information loss, seriously affecting the imaging quality and data reliability. Therefore, it is urgent to explore a combined software and hardware optimization strategy to compensate for the registration error by combining computational methods while optimizing the filter structure, so as to comprehensively improve the accuracy, stability and imaging quality of the on-chip computational spectral imaging system. Summary of the Invention

[0005] The object of the present invention is to propose an integrated error suppression method for on-chip spectral imaging systems with combined hardware and software optimization. Starting from two aspects of hardware design and software optimization, in terms of hardware, by adopting an optimized structure in the filter design, the matching degree with the detector is improved; in terms of software, a simulated annealing algorithm is introduced to optimize the filter arrangement pattern to ensure the minimization of errors during the spectral acquisition process. It can not only effectively reduce the impact of registration errors on the quality of spectral data, but also improve the comprehensive performance of the on-chip computational spectral imaging system, providing a new solution for high-precision and high-stability spectral imaging applications.

[0006] To achieve the above object, the present invention proposes an integrated error suppression method for on-chip spectral imaging systems with combined hardware and software optimization, including the following steps:

[0007] Step S1: Design the inner square unit structure of the coded filter so that it corresponds one-to-one with the detector pixels, and fabricate the filter through semiconductor processes.

[0008] Step S3: Jointly optimize the filter structure parameters and the spectral reconstruction model through an end-to-end deep learning framework to suppress registration errors.

[0009] Step S2: Adopt a simulated annealing algorithm to optimize the filter arrangement pattern to minimize the spectral difference between adjacent filters.

[0010] Preferably, in step S1, the minimum repeating unit of the coded filter contains M filters, and the preparation process of the filter is as follows:

[0011] Step S11: Select SiO2 as the substrate material, and deposit a silicon film with a height of h si on its surface through thermal evaporation.

[0012] Step S12: Use polymethyl methacrylate (PMMA) as the photoresist and spin-coat it on the silicon film, and use electron beam lithography technology to engrave the metasurface filter pattern.

[0013] Step S13: Use the exposed photoresist pattern as a mask, and use inductively coupled plasma reactive ion etching (ICP-RIE) technology to etch the exposed silicon film.

[0014] Step S14: Use Remover PG solution to remove the residual photoresist to obtain a filter with a preset silicon film height of h si

[0015] Preferably, the minimum repeating unit of the coded filter realizes the spatial information encoding of the target scene, and the spectral transmittance response of the j th filter is T j (λ) ​, the spectral information encoding of the target scene is achieved; wherein, , λ is the wavelength.

[0016] Preferably, in step S2, the structural parameters of the encoding filter are optimized, and the formula is as follows:

[0017] ;

[0018] Wherein, is the target spectral information, is the parameter to be optimized by the spectral reconstruction algorithm, h is the structural parameter of the encoding filter, is the optimal structural parameter of the encoding filter obtained by optimization, is the optimal parameter of the spectral reconstruction algorithm obtained by optimization, is the spectrum measured on the detector, is the decoding function, is the two-norm, is the parameter that minimizes the following expression.

[0019] Preferably, in step S2, the spectral reconstruction model steps are as follows:

[0020] Assume that the unknown spectral curve of the target object is S(λ) , and the spectral response function of each pixel of the detector is η (λ) , and the original intensity data detected by the detector after the j th filter is:

[0021] ;

[0022] Use the spectral reconstruction algorithm to reconstruct the intensity detected by the detector corresponding to each filter into the spectral curve of the target object as ; wherein, is the original intensity data detected by the detector after the j th filter.

[0023] Preferably, the spectral reconstruction algorithm is a neural network algorithm.

[0024] Preferably, in step S3, the optimization objective of the simulated annealing algorithm is to minimize the spectral transmittance difference between adjacent filters, and the optimization process steps are as follows:

[0025] Step S31, Initialize the spectral data, temperature, and maximum number of iterations, and randomly initialize the filter arrangement;

[0026] Step S32, Calculate the spectral difference between adjacent filters of the current arrangement and set the current arrangement as the optimal arrangement;

[0027] Step S33: Enter a loop, copy the current arrangement, randomly swap N pairs of filters, and calculate the spectral difference of the new arrangement.

[0028] Step S34: Determine whether to accept the current arrangement according to the judgment condition. If the spectral difference between adjacent filters in the current arrangement is less than the spectral difference between adjacent filters in the optimal arrangement, then accept and update the current arrangement and the optimal arrangement. If the spectral difference between adjacent filters in the current arrangement is greater than the spectral difference between adjacent filters in the optimal arrangement, then do not accept and retain the current arrangement.

[0029] Step S35: Update the temperature and print the progress according to the set number of steps.

[0030] Step S36: Determine whether the maximum number of iterations is reached. If so, end the loop and return the optimal arrangement. If not, re-enter the loop and repeat steps S33 - S36.

[0031] Therefore, the present invention proposes a method for suppressing the integration error of an on-chip spectral imaging system with combined software and hardware optimization, and its beneficial effects are as follows:

[0032] (1) Reducing the influence of registration error: Through combined software and hardware optimization, the hardware uses an inner-circle square structure to design coded filters, improving the matching degree with detector pixels and reducing spectral aliasing. The software uses the simulated annealing algorithm to optimize the filter arrangement, making the spectral sampling uniform and improving the overall imaging quality.

[0033] (2) Improving the system stability: The hardware optimizes the filter structure to ensure stable and consistent spectral modulation, and the software optimization algorithm compensates for and optimizes spectral information to suppress errors. The software and hardware cooperate to reduce the influence of registration error on the system performance, enabling the system to maintain high-efficiency and stable imaging capabilities in different scenarios.

[0034] (3) Improving the computational efficiency and robustness: Deep learning constructs an end-to-end optimization framework, jointly trains the optical system design parameters and the computational reconstruction model, adaptively optimizes the filter arrangement and the spectral decoding process, reduces error accumulation, avoids the cumbersome process of manual tuning, realizes efficient and accurate spectral reconstruction, and improves the adaptability and robustness of the system in complex scenarios.

[0035] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0036] Figure 1 It is the overall schematic diagram of a method for suppressing the integration error of an on-chip spectral imaging system with combined software and hardware optimization according to the present invention;

[0037] Figure 2Flowchart of the optimization of filter arrangement based on the simulated annealing algorithm in the present invention;

[0038] Figure 3 Comparison chart of the spectral reconstruction accuracy results of the filter arrangement optimized by the simulated annealing algorithm and the directly optimized filter arrangement under different alignment errors in the embodiment of the present invention. Detailed implementation manners

[0039] To make the technical solutions, advantages and objectives of the present invention clearer, the technical solutions of the embodiments of the present invention will be described clearly and completely below. The described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of this application.

[0040] Unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meaning understood by those of ordinary skill in the art to which the present invention belongs.

[0041] Embodiment

[0042] Taking the short-wave infrared band (900 - 1700 nm) as an example, as Figure 1 shown, the present invention proposes a method for suppressing the integration error of an on-chip spectral imaging system with combined hard and soft optimization. This method comprehensively considers optical structure optimization and computational compensation, and through the synergistic effect of hardware design and software algorithms, fundamentally suppresses the influence of registration error on spectral imaging quality. At the hardware level, the present invention optimizes the design structure of the coded filter to make it have higher stability and matching degree during spectral modulation; at the software level, the simulated annealing algorithm is introduced to optimize the filter arrangement to ensure uniform sampling of spectral information and reduce local error propagation. In addition, the present invention realizes the overall optimization from optical coding to signal decoding through an end-to-end deep learning framework, improving the accuracy of spectral data reconstruction and the robustness of the system. This technical solution significantly reduces the influence of registration error on the spectral imaging system while ensuring the compact integration of the system, providing a new solution for high-precision applications of on-chip computational spectral imaging. The specific steps are as follows:

[0043] S1. Design the inner square unit structure of the coded filter to correspond one-to-one with the detector pixels, and fabricate the filter through semiconductor processes.

[0044] The minimum repeating unit of the coded filter realizes the spatial information coding of the target scene, which contains 16 filters, and the spectral transmittance response of the j th filter is T j (λ), the spectral information of the target scene is encoded. , λ is the wavelength.

[0045] Among them, the preparation process of the filter is as follows:

[0046] S11. Select SiO2 as the substrate material, and form a silicon film with a height of h si on its surface by thermal evaporation deposition;

[0047] S12. Use polymethyl methacrylate PMMA as the photoresist and spin-coat it on the silicon film, and use electron beam lithography technology to engrave the metasurface filter pattern;

[0048] S13. Using the exposed photoresist pattern as a mask, use inductively coupled plasma reactive ion etching technology ICP-RIE to etch the exposed silicon film;

[0049] S14. Use Remover PG solution to remove the residual photoresist, and obtain a filter with a preset silicon film height of h si .

[0050] S2. Jointly optimize the filter structure parameters and the spectral reconstruction model through an end-to-end deep learning framework to suppress the registration error.

[0051] Optimize the structure parameters of the encoding filter, and the formula is as follows:

[0052] ;

[0053] Among them, is the target spectral information, is the parameter to be optimized by the spectral reconstruction algorithm, h is the structure parameter of the encoding filter, is the optimal structure parameter of the encoding filter obtained by optimization, is the optimal parameter of the spectral reconstruction algorithm obtained by optimization, is the spectrum measured on the detector, is the decoding function, is the two-norm, is the parameter that makes the following expression obtain the minimum value.

[0054] The steps of the spectral reconstruction model are as follows:

[0055] Assume that the unknown spectral curve of the target object is S(λ) , and the spectral response function of each pixel of the detector is η (λ) , and the original intensity data detected by the detector after the j th filter is:

[0056] ;

[0057] Use the spectral reconstruction algorithm (neural network algorithm) to reconstruct the spectral curve of the target object from the intensities detected by the detectors corresponding to each filter as follows .

[0058] S3. As shown in Figure 2 , use the simulated annealing algorithm to optimize the arrangement of 16 filters, minimize the spectral differences between adjacent filters, ensure the uniformity of spectral sampling, reduce the impact of local error accumulation on spectral reconstruction, and thus improve the spectral imaging quality and the robustness of the system

[0059] The optimization objective of the simulated annealing algorithm is to minimize the spectral transmittance differences between adjacent filters. The optimization process steps are as follows

[0060] S31. Initialize the spectral data, temperature, and maximum number of iterations, and randomly initialize the filter arrangement

[0061] S32. Calculate the spectral differences between adjacent filters of the current arrangement and set the current arrangement as the optimal arrangement

[0062] S33. Enter the loop, copy the current arrangement method, randomly swap N pairs of filters, and calculate the spectral differences of the new arrangement

[0063] S34. Determine whether to accept the current arrangement according to the judgment condition. If the spectral differences between adjacent filters of the current arrangement are less than those of the optimal arrangement, accept and update the current arrangement and the optimal arrangement. If the spectral differences between adjacent filters of the current arrangement are greater than those of the optimal arrangement, do not accept and retain the current arrangement

[0064] S35. Update the temperature and print the progress according to the set steps

[0065] S36. Determine whether the maximum number of iterations is reached. If so, end the loop and return the optimal arrangement. If not, re-enter the loop and repeat S33 - S36

[0066] By comparing the optimal filter structure parameters obtained by the end-to-end algorithm, that is, the method without sorting and the filter array after arranging 16 filters by introducing the simulated annealing algorithm. As shown in Figure 3 , the results show that when the alignment error is 0.5μm, the spectral reconstruction accuracy of the filter arrangement optimized by the simulated annealing algorithm is 50% higher than that of the directly optimized filter arrangement, significantly enhancing the robustness of the system to the registration error and the accuracy of spectral reconstruction

[0067] Therefore, the present invention provides a method for suppressing integration errors in an on-chip spectral imaging system with combined hardware and software optimization. The encoding filter is designed with an inner square structure in hardware, and the arrangement is optimized using the simulated annealing algorithm in software, effectively reducing the influence of registration errors and improving the spectral imaging quality. At the same time, the combination of hardware and software enhances the system stability, and the end-to-end deep learning framework improves the calculation efficiency and system robustness, reducing error propagation, providing a new solution for high-precision applications of on-chip computational spectral imaging.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements do not make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for suppressing integration errors in an on-chip spectral imaging system with combined software and hardware optimization, characterized in that, Including the following steps: Step S1: Design the inner square unit structure of the coded filter so that it corresponds one-to-one with the detector pixels, and prepare the filter through semiconductor processes. The specific operations are as follows: Step S11: Select SiO2 as the substrate material and form a silicon film with a height of h on its surface by thermal evaporation deposition si ; Step S12: Use polymethyl methacrylate (PMMA) as the photoresist and spin-coat it on the silicon film, and use electron beam lithography technology to engrave the metasurface filter pattern; Step S13: Use the exposed photoresist pattern as a mask, and use inductively coupled plasma reactive ion etching technology (ICP-RIE) to etch the exposed silicon film; Step S14: Use the Remover PG solution to remove the residual photoresist, obtaining a filter with a preset silicon film height of h si ; Step S2: Jointly optimize the structural parameters of the coded filter and the spectral reconstruction model through an end-to-end deep learning framework to suppress registration errors; Step S3: Use the simulated annealing algorithm to optimize the filter arrangement pattern to minimize the spectral difference between adjacent filters.

2. A method for suppressing integration errors of an on-chip spectral imaging system with combined software and hardware optimization, as claimed in claim 1, wherein The minimum repeating unit of the coded filter realizes the spatial information coding of the target scene. The spectral transmittance response of the th filter realizes the spectral information coding of the target scene. Among them, , is the wavelength.

3. A method for suppressing integration errors of an on-chip spectral imaging system with combined software and hardware optimization, as claimed in claim 1, wherein In step S2, the structural parameters of the coded filter are optimized, and the formula is as follows: ； Among them, is the target spectral information, is the parameter to be optimized by the spectral reconstruction algorithm, h is the structural parameter of the encoding filter, is the optimal structural parameter of the encoding filter obtained by optimization, is the optimal parameter of the spectral reconstruction algorithm obtained by optimization, is the spectrum measured on the detector, is the decoding function, is the two-norm, is the parameter that minimizes the following expression.

4. A method for suppressing integration errors of an on-chip spectral imaging system with combined software and hardware optimization, as claimed in claim 1, wherein In step S2, the spectral reconstruction model steps are as follows: Suppose the unknown spectral curve of the target object is , and the spectral response function of each pixel of the detector is , and the original intensity data detected by the detector after the -th filter is: ； Use the spectral reconstruction algorithm to reconstruct the spectral curve of the target object from the intensities detected by the detectors corresponding to each filter as ; where is the original intensity data detected by the detector after the -th filter 5. A method for suppressing integration errors in an on-chip spectral imaging system with combined software and hardware optimization, as claimed in claim 4, wherein The spectral reconstruction algorithm is a neural network algorithm.

6. A method for suppressing integration errors of a hybrid software and hardware optimized on-chip spectral imaging system according to claim 1, characterized in that In step S3, the optimization objective of the simulated annealing algorithm is to minimize the spectral transmittance difference between adjacent filters, and the optimization process steps are as follows: Step S31: Initialize the spectral data, temperature, and maximum number of iterations, and randomly initialize the filter arrangement; Step S32: Calculate the spectral difference between adjacent filters of the current arrangement and set the current arrangement as the optimal arrangement; Step S33: Enter the loop, copy the current arrangement method, randomly exchange N pairs of filters, and calculate the spectral difference of the new arrangement; Step S34: Determine whether to accept the current arrangement according to the judgment condition. If the spectral difference between adjacent filters of the current arrangement is less than the spectral difference between adjacent filters of the optimal arrangement, then accept and update the current arrangement and the optimal arrangement. If the spectral difference between adjacent filters of the current arrangement is greater than the spectral difference between adjacent filters of the optimal arrangement, then do not accept and retain the current arrangement; Step S35: Update the temperature and print the progress according to the set number of steps; Step S36: Determine whether the maximum number of iterations is reached. If so, end the loop and return the optimal arrangement; if not, re-enter the loop and repeat steps S33 - S36.

Images