Multispectral imaging method and system based on precise microscanning and active spectral modulation

Abstract

This invention discloses a multispectral imaging method and system based on precision micro-scanning and active spectral modulation. The system includes: a host computer module for sending commands to synchronously control spectral switching and micro-displacement of the image sensor, and for receiving acquired multispectral data; a spectral image acquisition module, consisting of an industrial telecentric lens, a liquid crystal tunable filter, an image sensor, and a precision micro-scanning stage, which acquires image sequences with sub-pixel displacement in specific spectral bands according to commands; and a spectral image reconstruction module that uses a super-resolution algorithm driven by both physics and data to process the image sequences in each band and generate a high-fidelity multispectral image cube. This invention, by integrating active spectral modulation and sub-pixel displacement technologies, solves the problem of simultaneously achieving high spectral dimension, high resolution, and high geometric fidelity in multispectral imaging, providing a solution for semiconductor detection, biomedicine, and other fields that combines ultra-high spatial detail with accurate spectral information.

Description

Technical Field

[0001] This invention relates to the fields of precision optical imaging and machine vision, specifically to a multispectral imaging method and system based on precision micro-scanning and active spectral modulation. Background Technology

[0002] Multispectral imaging technology can simultaneously acquire two-dimensional spatial information and one-dimensional spectral information of the target object, forming a three-dimensional data cube that integrates image and spectrum. Compared with traditional wideband black-and-white or RGB color imaging, multispectral imaging, by subdividing the spectrum into multiple narrow bands, can capture spectral fingerprint features that are invisible to the human eye, thereby enabling qualitative and quantitative analysis of material composition. Currently, this technology has been widely used in high-precision fields such as wafer defect detection in semiconductors, microscopic analysis of pathological sections, precision agriculture monitoring, and digital preservation of cultural relics.

[0003] Existing multispectral imaging technologies primarily employ the following methods: filter wheel or liquid crystal tunable filter type, which sequentially acquires images of different wavelengths through time modulation; Bayer array type, which involves coating an array of filters of different wavelengths onto the sensor, sacrificing spatial resolution for spectral acquisition in a single exposure; and pushbroom type, which utilizes prisms or gratings for beam splitting, combined with mechanical motion for line scanning. However, with the increasing demands for detection accuracy, the contradiction between high spatial resolution, high spectral dimensionality, and imaging efficiency in the above-mentioned traditional technologies is becoming increasingly prominent, making it difficult to achieve all three simultaneously. Specifically:

[0004] (1) Limited spatial resolution: For array-type multispectral cameras, the spatial resolution decreases exponentially with the increase of the number of spectral channels. For example, for a 16-channel array-type multispectral camera, the effective physical spatial resolution of a single band is only 1 / 16 of the total number of sensor pixels. This loss of information at the hardware level leads to blurred details and aliasing in the reconstructed image, making it difficult to apply to scenarios with high geometric accuracy requirements, such as the detection of tiny defects in semiconductors.

[0005] (2) Data redundancy and low acquisition efficiency: Traditional time-modulated multispectral systems typically employ a blind acquisition strategy such as "full-band equal-interval scanning" in their working mode (Yuan X, Brady DJ, Katsaggelos A K. Snapshot compressive imaging: Theory, algorithms, and applications[J]. IEEE Signal Processing Magazine, 2021, 38(2): 65-88.). For example, one image is acquired every 5nm from 400nm to 900nm, for a total of 100 images. However, this blind traversal acquisition strategy has serious efficiency defects in practical applications: the feature information of the target object is often concentrated in a few bands or a narrow spectral range. Full-band scanning not only generates a massive amount of spectral redundancy information, but the accumulated wavelength switching and exposure events also greatly waste the acquisition cycle and increase the burden of transmission and storage.

[0006] (3) Limitations of Image Reconstruction Algorithms: To address the problem of insufficient resolution, a large number of reconstruction algorithms based on deep neural networks have emerged in recent years (Liang J, Cao J, Sun G, et al. Swinir: Imagerestoration using swin transformer[C] / / Proceedings of the IEEE / CVFinternational conference on computer vision. 2021: 1833-1844.), such as image super-resolution networks based on SwinTransformer. Although these end-to-end deep learning methods have achieved excellent visual effects on natural scene images, they have fundamental defects when applied to precise multispectral detection: these algorithms are essentially data-driven "black box" models, tending to "invent" high-frequency textures based on statistical regularities in the training set. Due to the lack of explicit modeling of the physical optical characteristics of the imaging system, they cannot correct the lateral chromatic aberration that is prevalent in multispectral imaging. Furthermore, when the spectral reflectance of the object being tested is inconsistent with the distribution of the training data, such as when the reflectance of a semiconductor differs in different wavelength bands, this type of algorithm is prone to producing illusory textures, causing the generated image to be geometrically misaligned with the real physical space, which seriously affects subsequent quantitative analysis and defect localization.

[0007] Therefore, there is an urgent need to develop a new multispectral imaging method and system that can intelligently balance imaging efficiency and image quality, while simultaneously achieving ultra-high spatial resolution and accurate spectral reconstruction. Summary of the Invention

[0008] This invention aims to address the following major problems existing in current high-resolution multispectral imaging technologies: the contradiction between spatial resolution and spectral dimension, the rigid and inflexible spectral data acquisition strategies, and the insufficient fidelity of reconstruction algorithms. To solve these technical problems, this invention proposes a multispectral imaging method and system based on precision micro-scanning and active spectral modulation.

[0009] The present invention is achieved by at least one of the following technical solutions.

[0010] A multispectral imaging method based on precision micro-scanning and active spectral modulation includes the following steps: S1. Micro-scanning is performed by controlling a precision micro-scanning stage through an active spectral modulation strategy to acquire image sequences, including micro-scanning reference image sequences and spectral feature image sequences. S2. Reconstruct a high-resolution structure-guided image based on the micro-scanning reference image sequence; S3. Based on the structure-guided image and spectral feature image sequence, a high-resolution multispectral data cube is finally output after the fusion algorithm.

[0011] Furthermore, the active spectral modulation strategy has the following two operating modes: (1) Reference band high resolution mode: Select the corresponding center wavelength, and control the micro-scanning sequence executed by the precision micro-scanning stage under each backbone wavelength; (2) Spectral scanning mode of interest region: The user sets the spectral region of interest, and the host computer controls the liquid crystal adjustable filter to perform step scanning to acquire a low spatial resolution spectral feature image sequence.

[0012] Furthermore, a high-resolution structure-guided image is reconstructed based on the micro-scanning reference image sequence, including the following steps: S21. Initialization and blind estimation: A multi-scale phase correlation algorithm is used to perform sub-pixel geometric registration on the micro-scanning reference image sequence. At the same time, a pre-trained adversarial generative network is used as input for any acquired image. The generator outputs a blur kernel and obtains an initial high-resolution image through bicubic interpolation and brightness alignment. S22, Inner Loop Physical Inversion Update: Based on the iterative back projection algorithm, the initial high-resolution image is physically updated using physical model constraints during iteration to obtain the physically updated image; S23. Depth Prior Denoising: Input the physically updated image into a depth denoising neural network, perform depth feature extraction and denoising under the control of the set noise level parameter, and output a prior constraint image with clean texture features. The noise level parameter shows a decreasing trend as the iteration round increases. S24. Gradient-gated adaptive fusion: Calculate the local spatial gradient of the physically updated image to construct a spatial structure weight map, and use the spatial structure weight map to perform weighted fusion of the physically updated image and the prior constraint image to generate an intermediate high-resolution image. S25. Blur kernel fine-tuning: Construct a kernel optimization objective function that includes kernel fidelity terms and total variation regularization terms. Using the initial blur kernel as the anchor point, fine-tune and update the current blur kernel using the gradient descent method to match the optical characteristics of the actual imaging system. S26. External loop iteration control: Repeatedly execute steps S22 to S25 until the preset number of iterations is reached or the data residuals converge, and finally output a high-resolution structure-guided image.

[0013] Furthermore, step S3 specifically includes: S31. Geometric Prior Transfer: Using the structure-guided image as a spatial reference, construct a structure-guided tensor or gradient field to perform sub-pixel-level geometric registration on the spectral feature image sequence. S32. Construct a joint energy functional: Establish an energy functional that includes a data fidelity term and a structure consistency regularization term, wherein the structure consistency regularization term forces the gradient distribution of each spectral image to approximate the gradient distribution of the structure-guided image. S33. Optimization Solution: The steepest descent method is used to minimize the total energy functional, accurately transferring the high-frequency texture information of the structure-guided image to each spectral band, thereby significantly improving the spatial resolution of the entire band while maintaining spectral accuracy.

[0014] The device for implementing the multispectral imaging method based on precision micro-scanning and active spectral modulation includes a camera top cover, a camera base, a liquid crystal tunable filter, an image sensor, and a precision micro-scanning stage. The camera top cover and the camera base together form an internal cavity with light-shielding and dust-proof functions. Inside the cavity, a precision micro-scanning stage is installed on the inner side of the camera base; the image sensor is installed on the moving end of the precision micro-scanning stage. The liquid crystal adjustable filter is installed on the light incident end of the camera base, i.e. outside the camera base, and is located in the optical path in front of the image sensor. In order to adapt to different imaging needs, the front end of the liquid crystal adjustable filter is provided with a standard optical interface for configuring a detachable screw-on industrial telecentric lens. After the light of the object under test is filtered by the liquid crystal adjustable filter, it is accurately focused onto the internal image sensor.

[0015] Furthermore, the center of the image sensor is aligned with the optical axis of the industrial telecentric lens, serving as the initial position for image acquisition. The object under test is placed at the object-side working distance of the industrial telecentric lens, ensuring a clear image of the object on the photosensitive surface of the image sensor.

[0016] Furthermore, a liquid crystal tunable filter is installed in the optical path between the telecentric lens and the image sensor to switch spectral channels according to instructions.

[0017] The system for implementing the multispectral imaging method based on precision micro-scanning and active spectral modulation includes: Spectral image acquisition module, used to acquire spectral images; The host computer module is used to control and execute the hierarchical active spectral modulation strategy; The spectral image reconstruction module is used to acquire structure-guided images and accurately transfer the high-frequency texture information of the structure-guided images to each spectral band.

[0018] A computer device according to the present invention includes a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, which, when executed by the processor, causes the processor to implement the method described herein.

[0019] The present invention provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor implements the method described herein.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Breaking the constraints of resolution and efficiency: Through the active modulation strategy of "reference band micro-scanning + region of interest fast scanning", only a few key bands are subjected to time-consuming mechanical micro-scanning, while a large number of spectral bands are captured in a single shot in combination with algorithm super-resolution. This method ensures that the final output image has ultra-high resolution across the entire band while reducing the data acquisition time by several to tens of times.

[0021] (2) High geometric fidelity of reconstructed images: The reconstruction algorithm proposed in this invention abandons the traditional simple interpolation and instead uses adversarial generative networks to blindly estimate the blur kernel, combining the advantages of physical inversion and deep learning. In particular, the introduction of the "gradient-guided fusion" mechanism effectively solves the problem of false textures that are easily generated by deep learning methods, ensuring the geometric edge sharpness at the microscopic imaging level.

[0022] (3) Accurate and reliable spectral information: During the fusion stage, the geometric consistency between the spectral image and the structure-guided image is strictly constrained by the gradient-based variational model. At the same time, the physical authenticity of the spectral values ​​is locked by the data fidelity term, avoiding the spectral aliasing and color distortion problems common in traditional fusion algorithms.

[0023] (4) High applicability: This system supports customizing the “reference band” and “range of interest” according to the characteristics of the object being measured, which has extremely high flexibility and can be widely used in precision industrial testing, biomedical diagnosis and materials analysis. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the overall structure of a multispectral imaging system based on precision micro-scanning and active spectral modulation, as shown in the embodiment. Figure 2 This is an exploded view of components of a multispectral imaging system based on precision micro-scanning and active spectral modulation, as shown in the embodiment. Figure 3 This is a schematic diagram of the working mode of the host computer module of a multispectral imaging method based on precision micro-scanning and active spectral modulation, as shown in the embodiment. Figure 4 This example illustrates a multispectral imaging method based on precision micro-scanning and active spectral modulation, using structure-guided images. C Reconstruction module flowchart; Figure 5 This is a schematic diagram of the full-band fusion reconstruction module of a multispectral imaging method based on precision micro-scanning and active spectral modulation, as shown in the embodiment. Figure 6 This is a schematic diagram of the overall process of a multispectral imaging method based on precision micro-scanning and active spectral modulation, as shown in the embodiment. The diagram shows: 1-Camera top cover, 2-Camera base, 3-Liquid crystal adjustable filter, 4-Image sensor, 5-Precision micro-scanning platform. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] Example 1 like Figure 1 , Figure 2 As shown, this embodiment of a multispectral imaging system based on precision micro-scanning and active spectral modulation includes a camera top cover 1, a camera base 2, a liquid crystal tunable filter 3, an image sensor 4, and a precision micro-scanning stage 5.

[0027] The camera top cover 1 and the camera base 2 are interlocked and connected to form an internal cavity that provides light protection and dust protection. Inside this cavity, such as... Figure 2As shown, the precision micro-scanning stage 5 serves as a motion carrier, with its fixed end firmly connected to the inner mounting surface of the camera base 2; the image sensor 4 is rigidly connected to the moving end of the precision micro-scanning stage 5, ensuring that the center of the photosensitive surface of the image sensor 4 is coaxially aligned with the preset main optical axis of the system.

[0028] The liquid crystal adjustable filter 3 is installed at the light incident end of the camera base 2, i.e., outside the camera base 2, and located in the front optical path of the image sensor 4. To adapt to different imaging requirements, the front end of the liquid crystal adjustable filter 3 is provided with a standard optical interface, such as a C interface. This interface is equipped with a detachable, screw-on industrial telecentric lens. The center of the image sensor is aligned with the optical axis of the industrial telecentric lens as the initial position for image acquisition by the image sensor. The object to be measured is placed at the object-side working distance of the industrial telecentric lens. At this time, the object to be measured can be clearly imaged on the photosensitive surface of the image sensor, so that the light from the object to be measured is filtered by the liquid crystal adjustable filter 3 and accurately focused onto the internal image sensor 4.

[0029] In this embodiment, an industrial dual telecentric lens with a magnification of 1.0x is used to ensure no perspective error during imaging and a field distortion rate of less than 0.1%. The liquid crystal tunable filter 3 is a broadband liquid crystal tunable filter with a working wavelength covering 400nm to 1200nm and a response time of less than 50ms. The liquid crystal tunable filter 3 is installed in the optical path between the telecentric lens and the image sensor 4 to switch the spectral channel according to the command. The image sensor 4 is a visible-near infrared image sensor with a physical pixel size of 3.45um×3.45um and a resolution of 2048×2048. The precision micro-scanning stage 5 is a precision micro-scanning platform driven by piezoelectric ceramics with a closed-loop resolution of 0.01um and a travel range of 12um×12um.

[0030] Example 2 This embodiment provides a system based on a multispectral imaging method using precision micro-scanning and active spectral modulation, comprising: A spectral image acquisition module is used to acquire spectral images; the spectral image acquisition module includes: an industrial telecentric lens, a liquid crystal tunable filter, an image sensor, and a precision micro-scanning stage.

[0031] A host computer module is used to control and execute a hierarchical active spectral modulation strategy. This module is communicatively connected to the liquid crystal tunable filter, the image sensor, and the precision micro-scanning platform, and is configured to execute the hierarchical active spectral modulation strategy, which includes two modes: sequential or alternating execution by the control system. The image sensor is used to respond to the synchronous control commands from the host computer, perform exposure acquisition, and process the reference image sequence. I and the aforementioned spectral feature image sequenceP The data is transmitted to the host computer module.

[0032] The spectral image reconstruction module is used to acquire structure-guided images and accurately transfer the high-frequency texture information of the structure-guided images to each spectral band. This spectral image reconstruction module is configured in the host computer module or an independent data processing unit and is used to perform multispectral super-resolution reconstruction, specifically including the structure-guided images. C Reconstruction module and full-band fusion reconstruction module.

[0033] Example 3 like Figure 6 As shown, this embodiment of a multispectral imaging method based on precision micro-scanning and active spectral modulation includes the following steps: S1. The precision micro-scanning stage 5 is controlled by the active spectral modulation strategy configured by the host computer module to perform micro-scanning and acquire image sequences.

[0034] like Figure 3 As shown, the active spectral modulation strategy includes the control system sequentially or alternately executing the following two operating modes: (1) Reference band high-resolution mode: Select the center wavelengths corresponding to the three primary colors of RGB, such as 460nm, 530nm, and 650nm, as the backbone wavelengths. Under each backbone wavelength, control the precision micro-scanning stage 5 to execute a 3×3 or 4×4 micro-scanning sequence. For example, when executing a 3×3 micro-scanning sequence, the subpixel sampling ratio of the precision micro-scanning stage is 3. The precision micro-scanning stage 5 will drive the image sensor 4 to move in a 3×3 sampling grid in the plane, acquiring a total of 9 images. The physical spacing between adjacent sampling points is 1 / 3 of the physical pixel size of the image sensor.

[0035] Let the micro-scan step size be , The physical pixel size of image sensor 4; The subpixel sampling magnification of the precision micro-scanning stage. The acquired micro-scanning reference image sequence is denoted as... .

[0036] (2) Region of Interest (ROI) Spectral Scanning Mode: The user sets the spectral region of interest, such as the infrared band from 680nm to 750nm. The host computer controls the liquid crystal adjustable filter 3 to perform step scanning in 5nm increments. During this period, the precision micro-scanning stage 5 remains locked in position, the image sensor 4 performs a single exposure, and the acquired spectral feature image sequence is recorded as follows: .

[0037] S2, the structure-guided image reconstruction module reconstructs images based on the micro-scanning reference image sequence. A high-resolution structure-guided image was reconstructed from the column. .

[0038] like Figure 4 As shown, an algorithm driven by a hybrid physical model and depth prior is used to analyze micro-scanned reference image sequences. A high-resolution structure-guided image was reconstructed. ,in The total number of baseline images acquired includes the following steps: S211. Initialization and Blind Estimation: (1) Constructing the imaging model and defining variables: Assume the high-resolution image to be reconstructed is The imaging degradation process is modeled as follows: .

[0039] in Represents the high-resolution image to be solved; Indicates the sequence index of the micro-scan acquired image; Indicates the corresponding number The reference image acquired by image sensor 4 under sub-micro-displacement; Indicates containing the first Spatial downsampling matrix for sub-pixel displacement information; The point spread function, or blur kernel, represents the optical imaging system. This represents additive Gaussian noise.

[0040] (2) Fuzzy kernel blind estimation: Using a pre-trained adversarial generative network as input, any acquired reference image is used. The generator outputs the initial fuzz kernel. , which serves as the anchor point and initial value for subsequent kernel optimization steps.

[0041] (3) Constructing the initial image based on the frequency domain correlation method: In order to construct the iterative initial values ​​of the high-resolution image The sub-pixel displacement between micro-scanned images is calculated using a frequency domain-based phase correlation method. The specific calculation process is as follows: First, from the micro-scanned reference image sequence... The first image is selected as the reference image. All other images are used as images to be registered. Enter the information sequentially; secondly, for... and Hanning windows are applied to suppress edge artifacts. Next, the two processed images are transformed to the frequency domain using a fast Fourier transform to obtain complex spectra, and the normalized cross-power spectrum between the two images is calculated. Then, regarding An inverse Fourier transform is performed to obtain the impulse function graph in the spatial domain. The coordinates of the peak points in this graph correspond to the pixel displacement between the two images. Under ideal conditions, a clear peak is observed at the displacement locations, while the values ​​are close to 0 at other locations. Finally, the calculated displacements are used to center-align all acquired images, the median is taken to remove noise, and the images are magnified to the target resolution using bicubic interpolation to obtain the initial high-resolution image. .

[0042] S212, Internal Circulation Physical Inversion Update: (1) Step-by-step logic: This step constructs a physical constraint sub-loop based on an iterative back-projection algorithm. When the outer loop count... When (i.e., upon first entering the loop), the initial high-resolution image obtained in step S211 is... As the initial input for this step; when the outer loop count... (i.e., subsequent loops) output the image from the previous outer loop (i.e., the image fed back from step S216). Use this as the initial input for this step.

[0043] (2) Internal Iteration Process: Set the current input image as... ,implement The internal iterative update generates a physically updated image. . No. The update formula for the next internal iteration is as follows:

[0044] in Initialize to (Right now or ); Indicates the inner loop number 1 Transient image updated in the next iteration; This is the physical inversion step size; and These represent the deconvolution and upsampling operations corresponding to the transposes of the fuzzy matrix and the downsampling matrix, respectively; after... After one internal iteration, the final physically updated image is output. .

[0045] S213, Depth-Prior Denoising: Physically update the image Input deep denoising neural network: .

[0046] in, This represents a pre-trained deep denoising neural network; It is the output image after passing through a deep denoising neural network; For the first The noise level parameter of the secondary loop; this parameter follows an exponential decay strategy: . It is the initial noise level parameter. The coefficient representing the decay with the number of external circulation cycles. .

[0047] S214, Gradient-gated adaptive fusion: Calculate the local gradient magnitude of the physically updated image and construct a spatially adaptive weight map. : .

[0048] in, Gradient threshold; A coefficient to control the steepness of the transition; It is a spatial gradient operator; This represents the gradient magnitude map of the physically updated image.

[0049] The fusion formula is: .

[0050] in This represents the fused intermediate high-resolution image, which will be used as input for blur kernel fine-tuning.

[0051] In gradient Edge regions exceeding the threshold The algorithm preserves the sharp structure of the physical inversion; in flat regions with small gradients, The algorithm uses deep networks to remove noise.

[0052] S215, Fuzzy kernel fine-tuning: Construct a kernel optimization objective function to correct the estimation error of the initial fuzzy kernel:

[0053] in Indicates the first The optical blur kernel is updated after the second outer loop; The fuzzy kernel variable to be optimized; Represents convolution operation; This represents the fuzzy kernel obtained from the initial blind estimation by the adversarial generative network; Represents the regularization weight of the kernel fidelity term; express The square of the norm.

[0054] S216, External Circulation Control: Steps S212 to S215 are defined as a complete outer loop iteration cycle.

[0055] Let the outer loop counter The image output in step S214 As the initial input for the next round S212 and update the fuzz kernel in step S215. Used for the next round of calculations.

[0056] Repeat the outer loop steps described above until the preset maximum number of loops is reached. Or satisfying the condition... At the end of the second outer loop Output high-resolution structure-guided images .

[0057] S3, The full-band fusion reconstruction module receives the structure-guided image. The structural guide image output by the reconstruction module and the spectral feature image sequence output in the spectral scanning mode of the region of interest. After fusion algorithm, the final output is a high-resolution multispectral data cube.

[0058] like Figure 5 As shown, the full-band fusion reconstruction module is designed for low-resolution spectral feature image sequences. ,in The total number of spectral bands is used to guide the image using high-resolution structure. Super-resolution is performed as a priori. For each band image Solve for the corresponding high-resolution image Specifically, it includes the following steps: S311, Geometric Prior Transfer: Computational Structure-Guided Image gradient field To address the issue of brightness inversion between wavelengths—for example, some materials appear bright under red light but dark under infrared light—a structural consistency regularization term is defined: .

[0059] in, Indicates the pixel coordinate position in the image; Represents the spatial gradient operator; These are local linear mapping coefficients used to fit structure-guided images. With the spectral image to be reconstructed The differences in local gradient intensity and direction between them allow gradient directions to be opposite while maintaining structural consistency; express The square of the norm.

[0060] S312. Constructing the Joint Energy Functional: Establishing the Total Energy Functional Specifically, this includes data fidelity terms, structural consistency regularization terms, and total variation regularization terms: .

[0061] in, It is the spatial downsampling matrix of the current spectral image. It is the blur matrix of the current spectral image. It is the weight parameter of the structural consistency regularization term; It is the weight parameter of the total variation regularization term; This is the total variation regularization term.

[0062] S313. Optimization Solution: The steepest descent method is used to minimize the above energy functional, and its update formula is: .

[0063] in, It is the first Image estimates for each iteration; This is the iteration step size; and These are the transpose operators of the blur matrix and the spatial downsampling matrix of the current spectral image, respectively; It is a divergence operator; Represents the spatial gradient operator; It is a very small constant to avoid numerical instability caused by a denominator of zero.

[0064] The structure-guided image is obtained through the full-band fusion reconstruction module. Rich high-frequency texture information, by Carried and effectively transmitted to low-resolution spectral images. This allows the final generated multispectral data cube to achieve spatial resolution comparable to that of the structure-guided image in each band, while maintaining the physical accuracy of spectral intensity.

[0065] Those skilled in the art will understand that, although this embodiment uses While 680nm to 750nm can be selected as the backbone band, depending on the characteristics of the object being measured, such as chlorophyll fluorescence detection, this does not depart from the scope of protection of this invention.

[0066] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A multispectral imaging method based on precision micro-scanning and active spectral modulation, characterized in that, Includes the following steps: S1. Micro-scanning is performed by controlling a precision micro-scanning stage through an active spectral modulation strategy to acquire image sequences, including micro-scanning reference image sequences and spectral feature image sequences. S2. Reconstruct a high-resolution structure-guided image based on the micro-scanning reference image sequence; S3. Based on the structure-guided image and spectral feature image sequence, a high-resolution multispectral data cube is finally output after the fusion algorithm.

2. The multispectral imaging method based on precision micro-scanning and active spectral modulation according to claim 1, characterized in that, The active spectral modulation strategy has the following two operating modes: (1) Reference band high resolution mode: Select the corresponding center wavelength, and control the micro-scanning sequence executed by the precision micro-scanning stage under each backbone wavelength; (2) Spectral scanning mode of interest region: The user sets the spectral region of interest, and the host computer controls the liquid crystal adjustable filter to perform step scanning to acquire a low spatial resolution spectral feature image sequence.

3. The multispectral imaging method based on precision micro-scanning and active spectral modulation according to claim 1, characterized in that, A high-resolution structure-guided image is reconstructed based on a micro-scan reference image sequence, including the following steps: S21. Initialization and blind estimation: A multi-scale phase correlation algorithm is used to perform sub-pixel geometric registration on the micro-scanning reference image sequence. At the same time, a pre-trained adversarial generative network is used as input for any acquired image. The generator outputs a blur kernel and obtains an initial high-resolution image through bicubic interpolation and brightness alignment. S22, Inner Loop Physical Inversion Update: Based on the iterative back projection algorithm, the initial high-resolution image is physically updated using physical model constraints during iteration to obtain the physically updated image; S23. Depth Prior Denoising: Input the physically updated image into a depth denoising neural network, perform depth feature extraction and denoising under the control of the set noise level parameter, and output a prior constraint image with clean texture features. The noise level parameter shows a decreasing trend as the iteration round increases. S24. Gradient-gated adaptive fusion: Calculate the local spatial gradient of the physically updated image to construct a spatial structure weight map, and use the spatial structure weight map to perform weighted fusion of the physically updated image and the prior constraint image to generate an intermediate high-resolution image. S25. Blur kernel fine-tuning: Construct a kernel optimization objective function that includes kernel fidelity terms and total variation regularization terms. Using the initial blur kernel as the anchor point, fine-tune and update the current blur kernel using the gradient descent method to match the optical characteristics of the actual imaging system. S26. External loop iteration control: Repeatedly execute steps S22 to S25 until the preset number of iterations is reached or the data residuals converge, and finally output a high-resolution structure-guided image.

4. The multispectral imaging method based on precision micro-scanning and active spectral modulation according to claim 1, characterized in that, Step S3 specifically includes: S31. Geometric Prior Transfer: Using the structure-guided image as a spatial reference, construct a structure-guided tensor or gradient field to perform sub-pixel-level geometric registration on the spectral feature image sequence. S32. Construct a joint energy functional: Establish an energy functional that includes a data fidelity term and a structure consistency regularization term, wherein the structure consistency regularization term forces the gradient distribution of each spectral image to approximate the gradient distribution of the structure-guided image. S33. Optimization Solution: The steepest descent method is used to minimize the total energy functional, accurately transferring the high-frequency texture information of the structure-guided image to each spectral band, thereby significantly improving the spatial resolution of the entire band while maintaining spectral accuracy.

5. An apparatus for implementing the multispectral imaging method based on precision micro-scanning and active spectral modulation as described in claim 1, characterized in that, Includes camera top cover, camera base, liquid crystal adjustable filter, image sensor, and precision micro-scanning stage; The camera top cover and the camera base together form an internal cavity with light-shielding and dust-proof functions. Inside the cavity, a precision micro-scanning stage is installed on the inner side of the camera base; the image sensor is installed on the moving end of the precision micro-scanning stage. The liquid crystal adjustable filter is installed on the light incident end of the camera base, i.e. outside the camera base, and is located in the optical path in front of the image sensor. In order to adapt to different imaging needs, the front end of the liquid crystal adjustable filter is provided with a standard optical interface for configuring a detachable screw-on industrial telecentric lens. After the light of the object under test is filtered by the liquid crystal adjustable filter, it is accurately focused onto the internal image sensor.

6. The multispectral imaging method based on precision micro-scanning and active spectral modulation according to claim 5, characterized in that, The center of the image sensor is aligned with the optical axis of the industrial telecentric lens, serving as the initial position for image acquisition by the image sensor; the object under test is placed at the object-side working distance of the industrial telecentric lens, and the object under test can be clearly imaged on the photosensitive surface of the image sensor.

7. A multispectral imaging method based on precision micro-scanning and active spectral modulation according to claim 5, characterized in that, A liquid crystal adjustable filter is installed in the optical path between the telecentric lens and the image sensor to switch spectral channels according to instructions.

8. A system for implementing the multispectral imaging method based on precision micro-scanning and active spectral modulation as described in claim 1, characterized in that, include: Spectral image acquisition module, used to acquire spectral images; The host computer module is used to control and execute the hierarchical active spectral modulation strategy; The spectral image reconstruction module is used to acquire structure-guided images and accurately transfer the high-frequency texture information of the structure-guided images to each spectral band.

9. A computer device comprising a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, characterized in that: When the computer program is executed by the processor, it causes the processor to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor implements the method as described in any one of claims 1 to 8.

Images