Software and hardness combined optimization on-chip spectral imaging system integration error suppression method

Abstract

The invention specifically discloses a soft and hard combined optimization-based on-chip spectral imaging system integration error suppression method, and relates to the technical field of spectral imaging. The error suppression method comprises the following steps: S1, designing an inner ring square unit structure of a coding optical filter, enabling the inner ring square unit structure to be in one-to-one correspondence with detector pixels, and preparing the optical filter through a semiconductor process; s2, jointly optimizing optical filter structure parameters and a spectrum reconstruction model through an end-to-end deep learning framework, and inhibiting registration errors; and S3, optimizing the arrangement mode of the optical filters by adopting a simulated annealing algorithm, and minimizing the spectral difference of adjacent optical filters. According to the soft and hard combined optimization on-chip spectral imaging system integration error suppression method provided by the invention, the compact integration of the system is ensured, the influence of the registration error on the spectral imaging system is remarkably reduced, and a brand new solution is provided for the high-precision application of on-chip calculation spectral imaging.

Description

Technical Field

[0001] The present invention relates to the technical field of spectral imaging, and in particular to a software-hardware combined optimized on-chip spectral imaging system integrated error suppression method. Background Art

[0002] Spectral imaging technology has a wide range of applications in scientific research and daily life, covering many fields. For example, spectral imaging technology is used in art and cultural relics identification, drug component analysis, fruit and vegetable sorting, natural disaster monitoring and early warning, and other aspects of life and production.

[0003] With the rapid development of spectral imaging technology, on-chip computational spectral imaging systems have attracted extensive attention from researchers at home and abroad due to their high spectral resolution, compact integration and low power consumption, and have gradually become an important direction for the development of spectral imaging technology. Compared with traditional spectral imaging systems, on-chip computational spectral imaging technology relies on the integration of coded filter arrays and detectors. The compressed information of spectral aliasing is detected on the detector, and then the reconstruction algorithm is used to achieve efficient acquisition and reconstruction of multi-spectral or hyperspectral information. However, in practical applications, due to manufacturing process limitations, there is inevitably an alignment error between the coded filter and the detector pixels, which leads to spectral aliasing, decreased spatial resolution and information loss during spectral data acquisition, which in turn affects the quality of spectral imaging and the accuracy of data analysis.

[0004] At present, the research on on-chip computational spectral imaging systems mainly focuses on optical design optimization and computational reconstruction methods to improve the spectral imaging quality of the system. Optical optimization methods are mainly committed to improving the manufacturing accuracy of coding filters or designing more stable optical structures to optimize spectral modulation characteristics; computational reconstruction methods widely use technologies such as deep learning and optimization algorithms to improve the reconstruction ability of spectral data. However, existing research focuses more on improving the accuracy of spectral reconstruction, while the problem of registration error between coding filters and detector pixels has not been systematically studied and solved. Due to factors such as manufacturing process, assembly accuracy and device non-uniformity, this error is inevitable in on-chip computational spectral imaging systems and may lead to spectral mixing, spatial resolution degradation and information loss, seriously affecting imaging quality and data reliability. Therefore, it is urgent to explore a joint soft and hard optimization strategy to optimize the filter structure while combining computational methods to compensate for the registration error, so as to comprehensively improve the accuracy, stability and imaging quality of on-chip computational spectral imaging systems. Summary of the invention

[0005] The purpose of the present invention is to propose an integrated error suppression method for an on-chip spectral imaging system with joint hardware and software optimization. Starting from the two levels of hardware design and software optimization, in terms of hardware, the matching degree with the detector is improved by adopting an optimized structure in the filter design; in terms of software, a simulated annealing algorithm is introduced to optimize the filter arrangement mode to ensure that the error is minimized during the spectral acquisition process; it can not only effectively reduce the impact of the registration error on the quality of the spectral data, but also improve the comprehensive performance of the on-chip computational spectral imaging system, providing a new solution for high-precision and high-stability spectral imaging applications.

[0006] To achieve the above object, the present invention proposes a software-hardware joint optimized on-chip spectral imaging system integrated error suppression method, comprising the following steps: Step S1, designing the inner circle square unit structure of the coding filter so that it corresponds one-to-one with the detector pixels, and preparing the filter by semiconductor process; Step S3, jointly optimizing the filter structure parameters and the spectral reconstruction model through an end-to-end deep learning framework to suppress the registration error; Step S2: Optimize the filter arrangement pattern by using a simulated annealing algorithm to minimize the spectral difference between adjacent filters.

[0007] Preferably, in step S1, the minimum repeating unit of the coding filter comprises M A filter, the preparation process of the filter is as follows: Step S11, select SiO2 as the substrate material, and form a layer with a height of h on its surface by thermal evaporation deposition. si Silicon membrane; Step S12, using polymethyl methacrylate (PMMA) as a photoresist and spin coating it on the silicon film, and using electron beam lithography technology to carve a supersurface filter pattern; Step S13, using the exposed photoresist pattern as a mask, and using inductively coupled plasma reactive ion etching technology ICP-RIE to etch the exposed silicon film; Step S14: Use Remover PG solution to remove the residual photoresist to obtain a preset silicon film height of h. si Filter.

[0008] Preferably, the minimum repeating unit of the coding filter realizes the encoding of the spatial information of the target scene. j The spectral transmittance response of each filter is T j (l) , realizing the encoding of the spectral information of the target scene; among them, , l is the wavelength.

[0009] Preferably, in step S2, the structural parameters of the coding filter are optimized, and the formula is as follows: ; in, is the target spectrum information, is the parameter to be optimized of the spectrum reconstruction algorithm, h is the structural parameter of the coding filter, In order to optimize the optimal structural parameters of the coded filter, To optimize the optimal parameters of the spectrum reconstruction algorithm, is the spectrum measured on the detector, is the decoding function, is the two-norm, To find the parameter that makes the following expression reach the minimum value.

[0010] Preferably, in step S2, the spectrum reconstruction model steps are as follows: Assume that the unknown spectral curve of the target object is S(λ) , the spectral response function of each pixel of the detector is or (l) , located at j The original intensity data detected by the detector after the filter is: ; Use the spectral reconstruction algorithm to convert the intensity detected by the detector corresponding to each filter The reconstructed spectral curve of the target object is ;in, For the j The raw intensity data detected by the detector behind the filter.

[0011] Preferably, the spectral reconstruction algorithm is a neural network algorithm.

[0012] Preferably, in step S3, the optimization goal of the simulated annealing algorithm is to minimize the difference in spectral transmittance between adjacent filters, and the optimization process steps are as follows: Step S31, initializing spectral data, temperature and maximum number of iterations, and randomly initializing filter arrangement; Step S32, calculating the spectral difference between adjacent filters in the current arrangement and setting the current arrangement as the optimal arrangement; Step S33, enter a loop, copy the current arrangement, randomly exchange N pairs of filters and calculate the spectral difference of the new arrangement; Step S34, decide whether to accept the current arrangement according to the judgment condition. If the spectral difference between adjacent filters in the current arrangement is less than the spectral difference between adjacent filters in the optimal arrangement, then accept and update the current arrangement and the optimal arrangement. If the spectral difference between adjacent filters in the current arrangement is greater than the spectral difference between adjacent filters in the optimal arrangement, then do not accept and retain the current arrangement. Step S35, update the temperature and print the progress according to the set number of steps; Step S36, determine whether the maximum number of iterations has been reached, if so, end the loop and return to the optimal arrangement; if not, re-enter the loop and repeat steps S33-S36.

[0013] Therefore, the present invention proposes a software-hard joint optimization on-chip spectral imaging system integrated error suppression method, which has the following beneficial effects: (1) Reduce the impact of registration errors: Joint optimization of software and hardware. The hardware uses an inner circle square structure to design the coding filter to improve the matching degree with the detector pixels and reduce spectral aliasing. The software uses a simulated annealing algorithm to optimize the filter arrangement to make the spectral sampling uniform and improve the overall imaging quality.

[0014] (2) Improve system stability: Hardware optimizes the filter structure to ensure stable and consistent spectral modulation, and software optimizes the algorithm to compensate and optimize spectral information to suppress errors. The software and hardware work together to reduce the impact of registration errors on system performance, allowing the system to maintain efficient and stable imaging capabilities in different scenarios.

[0015] (3) Improving computational efficiency and robustness: Deep learning is used to build an end-to-end optimization framework, jointly train the optical system design parameters and computational reconstruction model, adaptively optimize the filter arrangement and spectral decoding process, reduce error accumulation, avoid the tedious manual tuning process, achieve efficient and accurate spectral reconstruction, and improve the system's adaptability and robustness in complex scenarios.

[0016] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is an overall schematic diagram of an integrated error suppression method for an on-chip spectral imaging system with software and hardware joint optimization according to the present invention; Figure 2 This is a flowchart of filter arrangement optimization based on simulated annealing algorithm in the present invention; Figure 3 This is a comparison chart of the spectral reconstruction accuracy results under different alignment errors of the filter arrangement optimized by the simulated annealing algorithm and the filter arrangement obtained by direct optimization in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the technical solutions, advantages and purposes of the present invention clearer, the technical solutions of the embodiments of the present invention are clearly and completely described below. The described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of this application.

[0019] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0020] Example Taking the short-wave infrared band (900-1700 nm) as an example, Figure 1 As shown, the present invention proposes an integrated error suppression method for an on-chip spectral imaging system with joint software and hardware optimization. The method comprehensively considers optical structure optimization and computational compensation, and fundamentally suppresses the influence of registration errors on the quality of spectral imaging through the synergy of hardware design and software algorithms. At the hardware level, the present invention optimizes the design structure of the coding filter so that it has higher stability and matching degree during the spectral modulation process; at the software level, a simulated annealing algorithm is introduced to optimize the filter arrangement to ensure uniform sampling of spectral information and reduce local error propagation. In addition, the present invention realizes overall optimization from optical encoding to signal decoding through an end-to-end deep learning framework, thereby improving the accuracy of spectral data reconstruction and the robustness of the system. While ensuring the compact integration of the system, this technical solution significantly reduces the influence of registration errors on the spectral imaging system, and provides a new solution for the high-precision application of on-chip computational spectral imaging. The specific steps are as follows: S1. Design the inner square unit structure of the coding filter so that it corresponds one-to-one with the detector pixels, and prepare the filter through semiconductor technology.

[0021] The minimum repeating unit of the coding filter realizes the encoding of the spatial information of the target scene, which contains 16 filters. j The spectral transmittance response of each filter is T j (l) , which realizes the encoding of the spectral information of the target scene. , l is the wavelength.

[0022] The preparation process of the filter is as follows: S11, select SiO2 as the substrate material, and form a layer with a height of h on its surface by thermal evaporation deposition. si Silicon membrane; S12, using polymethyl methacrylate (PMMA) as a photoresist and spin coating it on the silicon film, and using electron beam lithography technology to carve a super surface filter pattern; S13, using the exposed photoresist pattern as a mask, etching the exposed silicon film using inductively coupled plasma reactive ion etching technology ICP-RIE; S14, using Remover PG solution to remove the residual photoresist, and obtain a preset silicon film height of h si Filter.

[0023] S2. The filter structure parameters and the spectral reconstruction model are jointly optimized through an end-to-end deep learning framework to suppress the registration error.

[0024] Optimize the structural parameters of the coding filter, the formula is as follows: ; in, is the target spectrum information, is the parameter to be optimized of the spectrum reconstruction algorithm, h is the structural parameter of the coding filter, In order to optimize the optimal structural parameters of the coded filter, To optimize the optimal parameters of the spectrum reconstruction algorithm, is the spectrum measured on the detector, is the decoding function, is the two-norm, To find the parameter that makes the following expression reach the minimum value.

[0025] The steps of the spectral reconstruction model are as follows: Assume that the unknown spectral curve of the target object is S(λ) , the spectral response function of each pixel of the detector is or (l) , located at j The original intensity data detected by the detector after the filter is: ; Use the spectral reconstruction algorithm (neural network algorithm) to convert the intensity detected by the detector corresponding to each filter The reconstructed spectral curve of the target object is .

[0026] S3, such as Figure 2 As shown in the figure, the simulated annealing algorithm is used to optimize the arrangement of 16 filters, minimize the spectral differences between adjacent filters, ensure the uniformity of spectral sampling, and reduce the impact of local error accumulation on spectral reconstruction, thereby improving the quality of spectral imaging and the robustness of the system.

[0027] The optimization goal of the simulated annealing algorithm is to minimize the difference in spectral transmittance between adjacent filters. The optimization process steps are as follows: S31, initializing spectral data, temperature and maximum number of iterations, and randomly initializing filter arrangement; S32, calculating the spectral difference between adjacent filters in the current arrangement and setting the current arrangement as the optimal arrangement; S33, enter a loop, copy the current arrangement, randomly exchange N pairs of filters and calculate the spectral difference of the new arrangement; S34, deciding whether to accept the current arrangement according to the judgment condition; if the spectral difference between adjacent filters in the current arrangement is less than the spectral difference between adjacent filters in the optimal arrangement, then accepting and updating the current arrangement and the optimal arrangement; if the spectral difference between adjacent filters in the current arrangement is greater than the spectral difference between adjacent filters in the optimal arrangement, then not accepting and retaining the current arrangement; S35, update the temperature and print the progress according to the set number of steps; S36. Determine whether the maximum number of iterations has been reached. If so, end the loop and return to the optimal arrangement; if not, re-enter the loop and repeat S33-S36.

[0028] By comparing the optimal filter structure parameters obtained by the end-to-end algorithm, that is, the filter array after the unsorted method and the simulated annealing algorithm are introduced to arrange the 16 filters. Figure 3 As shown, the results show that the spectral reconstruction accuracy of the filter arrangement optimized using the simulated annealing algorithm is 50% higher than that of the filter arrangement obtained by direct optimization when the alignment error is 0.5μm, which significantly enhances the system's robustness to alignment errors and the accuracy of spectral reconstruction.

[0029] Therefore, the present invention provides an integrated error suppression method for an on-chip spectral imaging system with joint software and hardware optimization. The coding filter is designed with an inner circle square structure in hardware, and the arrangement is optimized using a simulated annealing algorithm in software, which effectively reduces the impact of alignment errors and improves the quality of spectral imaging. At the same time, the combination of software and hardware enhances system stability, and the end-to-end deep learning framework improves computing efficiency and system robustness, reduces error propagation, and provides a new solution for high-precision applications of on-chip computational spectral imaging.

[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A software-hardware joint optimization on-chip spectral imaging system integrated error suppression method, characterized in that: The following steps are involved: Step S1, designing the inner circle square unit structure of the coding filter so that it corresponds one-to-one with the detector pixels, and preparing the filter by semiconductor process; Step S2, jointly optimizing the filter structure parameters and the spectral reconstruction model through an end-to-end deep learning framework to suppress the registration error; Step S3: using a simulated annealing algorithm to optimize the filter arrangement pattern and minimize the spectral difference between adjacent filters.

2. The integrated error suppression method for on-chip spectral imaging system with software and hardware joint optimization according to claim 1 is characterized in that: In step S1, the minimum repeating unit of the coding filter comprises M A filter, the preparation process of the filter is as follows: Step S11, select SiO2 as the substrate material, and form a layer with a height of h on its surface by thermal evaporation deposition. si Silicon membrane; Step S12, using polymethyl methacrylate (PMMA) as a photoresist and spin coating it on the silicon film, and using electron beam lithography technology to carve a supersurface filter pattern; Step S13, using the exposed photoresist pattern as a mask, and using inductively coupled plasma reactive ion etching technology ICP-RIE to etch the exposed silicon film; Step S14: Use Remover PG solution to remove the residual photoresist to obtain a preset silicon film height of h. si Filter.

3. The integrated error suppression method for on-chip spectral imaging system with software and hardware joint optimization according to claim 2 is characterized in that: The minimum repeating unit of the coding filter realizes the encoding of the spatial information of the target scene. j The spectral transmittance response of each filter is T j (λ) , realizing the encoding of the spectral information of the target scene; among them, , λ is the wavelength.

4. The integrated error suppression method for on-chip spectral imaging system with software and hardware joint optimization according to claim 1 is characterized in that: In step S2, the structural parameters of the coding filter are optimized, and the formula is as follows: ； in, is the target spectrum information, is the parameter to be optimized of the spectrum reconstruction algorithm, h is the structural parameter of the coding filter, In order to optimize the optimal structural parameters of the coded filter, To optimize the optimal parameters of the spectral reconstruction algorithm, is the spectrum measured on the detector, is the decoding function, is the two-norm, To find the parameter that makes the following expression reach the minimum value.

5. The integrated error suppression method for on-chip spectral imaging system with software and hardware joint optimization according to claim 1, characterized in that: In step S2, the spectrum reconstruction model steps are as follows: Assume that the unknown spectral curve of the target object is S(λ) , the spectral response function of each pixel of the detector is η(λ) , located at j The original intensity data detected by the detector after the filter is: ； Use the spectral reconstruction algorithm to convert the intensity detected by the detector corresponding to each filter The reconstructed spectral curve of the target object is ;in, For the j The raw intensity data detected by the detector behind the filter.

6. The integrated error suppression method for on-chip spectral imaging system with software and hardware joint optimization according to claim 5 is characterized in that: The spectrum reconstruction algorithm is a neural network algorithm.

7. The integrated error suppression method for on-chip spectral imaging system with software and hardware joint optimization according to claim 1 is characterized in that: In step S3, the optimization goal of the simulated annealing algorithm is to minimize the difference in spectral transmittance between adjacent filters. The optimization process steps are as follows: Step S31, initializing spectral data, temperature and maximum number of iterations, and randomly initializing filter arrangement; Step S32, calculating the spectral difference between adjacent filters in the current arrangement and setting the current arrangement as the optimal arrangement; Step S33, enter a loop, copy the current arrangement, randomly exchange N pairs of filters and calculate the spectral difference of the new arrangement; Step S34, decide whether to accept the current arrangement according to the judgment condition. If the spectral difference between adjacent filters in the current arrangement is less than the spectral difference between adjacent filters in the optimal arrangement, then accept and update the current arrangement and the optimal arrangement. If the spectral difference between adjacent filters in the current arrangement is greater than the spectral difference between adjacent filters in the optimal arrangement, then do not accept and retain the current arrangement. Step S35, update the temperature and print the progress according to the set number of steps; Step S36, determine whether the maximum number of iterations has been reached, if so, end the loop and return to the optimal arrangement; if not, re-enter the loop and repeat steps S33-S36.

Images