Highly distributed bandwidth multiplexing imaging method and system with multi-dimensional coding computation enhancement

Abstract

This invention discloses a high-distributed bandwidth product imaging method and system based on multi-dimensional encoding and decoding computation enhancement, comprising the following steps: Step 1: Constructing an optical imaging model to acquire large-aperture multi-focus visible low-light images and small-aperture multi-exposure near-infrared images; Step 2: Constructing a dataset based on the images obtained in Step 1; Step 3: Constructing a distributed bandwidth product enhancement deep network model; Step 4: Minimizing the loss function and training the distributed bandwidth product enhancement deep network model from Step 3 to obtain a pre-trained distributed bandwidth product enhancement deep network model; Step 5: Inputting the acquired images into the pre-trained distributed bandwidth product enhancement deep network model obtained in Step 4 to obtain the reconstructed images; This invention establishes a bridge between the imaging link and the algorithm link, enabling deep optimization of the bandwidth product; it avoids the design of complex optical components, breaks the trade-off between depth of field and light flux, and achieves ultra-low-light imaging with large depth of field.

Description

Technical Field

[0001] This invention relates to the field of optical imaging technology, and more specifically to a high distributed bandwidth product imaging method and system based on multidimensional encoding and decoding computation enhancement. Background Technology

[0002] High spatial bandwidth product imaging is crucial for applications such as industrial inspection, security monitoring, and medical diagnostics. In diffraction-limited optical systems, the spatial bandwidth product is determined by the system's field of view (FOV) and resolution (θ).

[0003]

[0004] Due to physical constraints and optical aberrations, the bandwidth product of existing imaging systems is greatly limited. This leads to performance limitations and application lags in many extreme imaging conditions, such as low-light scenes.

[0005] Enhancing information capacity is a challenging task due to its ill-posed nature. To date, numerous solutions have been proposed and proven effective, with hardware-based design methods and algorithm-based optimization methods being the most investigated and studied by scholars due to their unique advantages. Hardware-based design methods aim to design special components to enhance information recording capabilities, thereby increasing bandwidth product. Benefiting from the rapid development of optical manufacturing and the semiconductor industry, large physical apertures and high-sensitivity digital sensors have increased data throughput over the past few decades, contributing to improved imaging quality. For example, optical microscopes and telescopes typically pursue high NA values ​​to achieve high-resolution imaging. Nevertheless, the difficulty of optical design and manufacturing increases exponentially with the amount of information, resulting in expensive and bulky system designs.

[0006] Algorithm-based optimization methods focus on improving the detail recovery capabilities of image post-processing. To date, numerous image processing methods have been proposed to achieve information expansion, such as single-image reconstruction and multi-image fusion. Single-image restoration aims to enhance the image by suppressing noise using specific priors, such as Gaussian filtering and bilateral filtering algorithms. Recently, deep learning has been widely applied in image restoration tasks, thanks to its ability to learn more general priors by extracting natural image statistics from large-scale data. Multi-image fusion focuses on enhancing usable information to improve image quality by fusing effective information from each frame. It involves capturing a series of low-light images with different noise distributions using a sensor, and adaptively integrating the effective content using appropriate fusion algorithms to achieve enhanced imaging.

[0007] Single-image reconstruction and multi-image fusion have the following problems: they are highly dependent on the given original image information, and are prone to failure when the original information is insufficient, especially under extreme imaging conditions (such as extremely dark conditions); the original input has serious redundancy, resulting in low imaging efficiency.

[0008] In traditional optical design, more complex lens groups are typically designed to improve image quality and eliminate aberrations. Furthermore, the process of light information passing through lenses and entering the sensor for quantization and imaging introduces various types of noise, especially in low-light environments, further degrading image quality. To address this issue, traditional methods usually involve designing sensors with higher sensitivity or larger apertures. However, this approach has the following drawbacks: while increasing the aperture size can increase the light throughput to the sensor, it results in a smaller depth of field, causing image blurring; the design of combined lenses, complex aspherical lenses, and high-sensitivity sensors requires highly specialized optical design knowledge and experience, and lens design and registration are extremely complex and expensive. Summary of the Invention

[0009] This invention addresses the problems existing in the prior art by providing a high-distributed bandwidth product imaging method and system based on multi-dimensional encoding and decoding computation enhancement.

[0010] The technical solution adopted in this invention is: a high-distributed bandwidth product imaging method based on multi-dimensional encoding and decoding computation enhancement, comprising the following steps:

[0011] Step 1: Construct an optical imaging model to acquire large-aperture multi-focus visible low-light images and small-aperture multi-exposure near-infrared images;

[0012] Step 2: Obtain high-quality ground truth images of visible light with large depth of field, and construct a dataset with the images obtained in Step 1;

[0013] Step 3: Construct a distributed bandwidth product-enhanced deep network model;

[0014] The distributed bandwidth product enhanced deep network model includes a visible light multi-focus fusion sub-model for extracting features from large aperture multi-focus visible low-light images, a near-infrared multi-exposure fusion sub-model for extracting features from small aperture multi-exposure near-infrared images, and a detail reconstruction sub-model for fusing the extracted features from large aperture multi-focus visible low-light images and small aperture multi-exposure near-infrared images.

[0015] Step 4: Minimize the loss function and train the distributed bandwidth product augmented deep network model from Step 3 to obtain a pre-trained distributed bandwidth product augmented deep network model.

[0016] Step 5: Input the acquired image into the pre-trained distributed bandwidth product augmented deep network model obtained in Step 4 to obtain the reconstructed image.

[0017] Furthermore, the optical imaging model in step 1 is a multi-coded optical imaging model, including a multi-focus imaging model that codes the imaging focus, a multi-aperture imaging model that codes the physical aperture, a multi-spectral imaging model that codes the imaging spectrum, and a multi-exposure near-infrared imaging model that codes the long and short exposures.

[0018] Furthermore, before constructing the multi-coded optical imaging model, the distributed bandwidth product (DBP) of the imaging system is first calculated:

[0019]

[0020] In the formula: FOV is the imaging field of view, θ' is the limiting resolution of the imaging system affected by aberrations and noise, D is the aperture size, l is the object depth, λ is the imaging wavelength, h is the sensor size, l' is the image distance, t is the exposure time, f is the focal length, M(l,l') is the optical magnification, pixel is the pixel size, η(λ) is the spectral quantum efficiency, L(λ) is the light source brightness, and A s Let denot be the pixel area, Dc be the dark current, and σ be the pixel area. read To read out noise, σ ADC To quantize the noise, g is the voltage gain, and ε is the characteristic width of the point spread function.

[0021] Furthermore, the visible light multi-focus fusion sub-model sequentially includes n convolutional downsampling modules and m convolutional upsampling modules; the convolutional downsampling modules are used to extract spatial focal length structural feature information, and the convolutional upsampling modules are used to fuse depth-of-field information; the near-infrared multi-exposure fusion sub-model has the same structure as the visible light multi-focus fusion sub-model; the detail reconstruction sub-model sequentially includes a convolutional upsampling modules and a feature reconstruction layer.

[0022] Furthermore, the detail reconstruction sub-model also includes two feature extraction modules; the features extracted by each level of the convolution upsampling module in the visible light multi-focus fusion sub-model are all input into the feature extraction module for feature extraction, and then input into the detail reconstruction module; the features extracted by each level of the convolution upsampling module in the near-infrared multi-exposure fusion sub-model are all input into another feature extraction module for feature extraction, and then input into the detail reconstruction module.

[0023] Furthermore, the convolutional downsampling module includes one convolutional layer, two activation layers, one convolutional downsampling layer, and two normalization layers; the convolutional upsampling module includes two convolutional layers, two activation layers, one upsampling layer, and two normalization layers; the feature reconstruction layer includes one upsampling layer, three convolutional layers, two normalization layers, and three activation layers; and the feature extraction module includes one convolutional upsampling layer, one convolutional downsampling layer, two normalization layers, and two activation layers.

[0024] Furthermore, the image output by the distributed bandwidth product enhanced deep network model is the output R of the high bandwidth product:

[0025] R = weight × nir + vis

[0026] In the formula: weight is the output weight map of the detail reconstruction sub-model, vis is the output image of the visible light multi-focus fusion sub-model, and nir is the output image of the near-infrared multi-exposure fusion sub-model.

[0027] Furthermore, the loss function L in step 4 is as follows:

[0028] L = L1 + L2

[0029] In the formula: L1 is the loss function of the visible light multi-focus fusion sub-model and the near-infrared multi-exposure fusion sub-model, and L2 is the loss function of the detail reconstruction sub-network;

[0030]

[0031] L2 = Pix + PL + GL

[0032] In the formula: C is the dimension of the image, H is the height of the image, W is the width of the image, real is the ground truth image, and vis is the output image of the visible light multi-focus fusion sub-model. Here, is the gradient operator, F is the F-norm, Pix is ​​the pixel loss function, PL is the perceptual loss function, and GL is the gradient loss function.

[0033] Furthermore, the pixel loss function is as follows:

[0034]

[0035] In the formula: R is the output image of the detail reconstruction sub-model;

[0036] The perceptual loss function is as follows:

[0037]

[0038] In the formula: j is the convolutional layer number, Here, x represents the pre-trained network model and the reconstructed image. The size of the generated convolutional layer of the network model is C. j ×H j ×W j Feature map;

[0039] The gradient loss function is as follows:

[0040]

[0041] An imaging system based on a high distributed bandwidth product imaging method enhanced by multidimensional encoding and decoding computation, comprising an optical imaging module and a distributed bandwidth product enhanced depth network model;

[0042] The optical imaging module is a multi-coded optical imaging model, including a multi-focus imaging model that encodes the imaging focus, a multi-aperture imaging model that encodes the physical aperture, a multi-spectral imaging model that encodes the imaging spectrum, and a multi-exposure near-infrared imaging model that encodes long and short exposures; it is used to acquire large-aperture multi-focus visible light images and small-aperture multi-exposure near-infrared images, and uses them as input to the distributed bandwidth product enhanced deep network model;

[0043] The distributed bandwidth product enhanced deep network model includes a visible light multi-focus fusion sub-model for extracting features from large-aperture multi-focus visible low-light images, a near-infrared multi-exposure fusion sub-model for extracting features from small-aperture multi-exposure near-infrared images, and a detail reconstruction sub-model for fusing the extracted features from large-aperture multi-focus visible low-light images and small-aperture multi-exposure near-infrared images; and a model for extracting spatial-spectral information from the input image to obtain the reconstructed image.

[0044] The beneficial effects of this invention are:

[0045] (1) In this invention, an optical imaging model is constructed, a bridge is established between the imaging link and the algorithm link, and the bandwidth product for specific applications is optimized in depth;

[0046] (2) This invention achieves high-quality low-light imaging with high distributed bandwidth product by deeply integrating multi-sensory optical encoders and learnable decoders in multi-code-decode imaging.

[0047] (3) The distributed bandwidth product enhanced deep network model of the present invention can integrate spatial spectral information from encoded optical images, solve the image degradation caused by the limited degrees of freedom of the imaging system, and achieve a significant improvement in the distributed bandwidth product. Attached Figure Description

[0048] Figure 1 This is a schematic diagram illustrating the principle of the method of the present invention.

[0049] Figure 2 This is a schematic diagram of the distributed bandwidth product enhanced deep network model structure of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0051] like Figure 1 As shown, a high-distributed bandwidth product imaging method based on multi-dimensional encoding and decoding computation enhancement includes the following steps:

[0052] Step 1: Construct an optical imaging model to acquire large-aperture multi-focus visible low-light images and small-aperture multi-exposure near-infrared images;

[0053] Before constructing the multi-coded optical imaging model, the distributed bandwidth product (DBP) of the imaging system is first calculated:

[0054]

[0055] In the formula: FOV is the imaging field of view, θ' is the limiting resolution of the imaging system affected by aberrations and noise, D is the aperture size, l is the object depth, λ is the imaging wavelength, h is the sensor size, l' is the image distance, t is the exposure time, f is the focal length, M(l,l') is the optical magnification, pixel is the pixel size, η(λ) is the spectral quantum efficiency, L(λ) is the light source brightness, and A s Let denot be the pixel area, Dc be the dark current, and σ be the pixel area. read To read out noise, σ ADC To quantize the noise, g is the voltage gain, and ε is the characteristic width of the point spread function.

[0056]

[0057] The distributed bandwidth product proposed in this invention differs from the traditional definition of spatial bandwidth product. It can characterize the ability of an imaging system with aberrations and noise to record information at different spatial depths. Based on the analysis of the distributed bandwidth product of the imaging system, it was found that the light information recorded under different imaging parameters (focal length, image distance, aperture, spectrum, exposure, etc.) has a high degree of complementarity, which helps to achieve high-quality computational imaging. 1) A large aperture can increase the light flux and alleviate the noise effect in low light, but the depth of field is small, causing image blurring; a small aperture can extend the depth of field, but the amount of light entering the camera decreases sharply, and the imaging signal-to-noise ratio decreases; 2) By adjusting the focal length or image distance of the imaging system, different depth of field information can be obtained; 3) Based on the invisibility of near-infrared light to the human eye, near-infrared supplementary lighting can be used to equip a near-infrared camera with a small aperture to record large depth of field detail information; 4) Based on the fact that long and short exposures can alleviate the noise effect in dark areas and the loss of information in bright areas, the exposure time of a small-aperture near-infrared system is encoded.

[0058] Based on the above analysis, an optical imaging model is constructed. This model is a multi-coded optical imaging model, including a multi-focus imaging model that codes the imaging focus, a multi-aperture imaging model that codes the physical aperture, a multi-spectral imaging model that codes the imaging spectrum, and a multi-exposure near-infrared imaging model that codes long and short exposures. By hybrid coding the imaging focus, physical aperture, imaging spectrum, and sensor exposure, two parallel optical path imaging models are formed: a large-aperture multi-focus visible light path and a small-aperture multi-exposure near-infrared light path.

[0059] The encoding method of this model can be flexibly adjusted according to the application. For example, for scenes that require a large depth of field, more focus points can be set; for scenes that require a high dynamic range, more exposures can be set.

[0060] Step 2: Construct a dataset based on the images obtained in Step 1. The dataset also includes ground truth images with large depth of field and high signal-to-noise ratio, which are used to train the model in Step 3.

[0061] Step 3: Construct a distributed bandwidth product-enhanced deep network model, such as Figure 2 As shown.

[0062] The distributed bandwidth product enhanced deep network model includes a visible light multi-focus fusion sub-model (VIS-Net) for extracting features from large aperture multi-focus visible low-light images, a near-infrared multi-exposure fusion sub-model (NIR-Net) for extracting features from small aperture multi-exposure near-infrared images, and a detail reconstruction sub-model for fusing the extracted features from large aperture multi-focus visible low-light images and small aperture multi-exposure near-infrared images.

[0063] The visible light multi-focus fusion sub-model sequentially comprises n convolutional downsampling modules and m convolutional upsampling modules. The convolutional downsampling modules are used to extract spatial focal length structural features, and the convolutional upsampling modules are used to fuse depth information. In this invention, n is 5 and m is 5. Each convolutional downsampling module includes one convolutional layer, two activation layers, one convolutional downsampling layer, and two normalization layers. Each convolutional upsampling module includes two convolutional layers, two activation layers, one upsampling layer, and two normalization layers. The activation layers enhance the nonlinearity of the network model and optimize the gradient descent process; the normalization layers accelerate network convergence.

[0064] The near-infrared multi-exposure fusion sub-model structure is the same as the visible light multi-focus fusion sub-model structure, which completes the extraction and fusion of effective detail information from multi-exposure near-infrared images.

[0065] The detail reconstruction sub-model sequentially comprises *a* convolutional upsampling modules and a feature reconstruction layer. In this invention, *a* is 4, and the feature reconstruction layer includes one upsampling layer, three convolutional layers, two normalization layers, and three activation layers. The upsampled feature information with rich contextual details from VIS-Net and NIR-Net is concatenated and fed into the detail reconstruction sub-model to enhance the network's detail recovery capability. To mitigate the differences between multi-level features, a feature extraction module is inserted along short connectivity paths, such as... Figure 2As shown, it includes two feature extraction modules: features extracted by each level of convolutional upsampling module in the visible light multi-focus fusion sub-model are input into the feature extraction module for feature extraction, and then input into the detail reconstruction module; features extracted by each level of convolutional upsampling module in the near-infrared multi-exposure fusion sub-model are input into another feature extraction module for feature extraction, and then input into the detail reconstruction module.

[0066] Each feature extraction module includes one convolutional downsampling layer, one upsampling layer, one convolutional layer, two normalization layers, and two activation layers. The convolutional upsampling layer is used to extract depth spectral and spatial features; the normalization layers are used to accelerate convergence; and the activation layers are used to increase the nonlinearity of the network model and enhance its resistance to overfitting.

[0067] The distributed bandwidth product augmentation deep network model outputs an image as the high bandwidth product output R. The output weight map of the detail reconstruction sub-model is multiplied by the NIR-Net output image nir, and then added to the noise-free and detailed image vis generated by VIS-Net to produce the high bandwidth product output. This method is beneficial for preserving spatial texture details and realistic structural information.

[0068] R = weight × nir + vis

[0069] In the formula: weight is the output weight map of the detail reconstruction sub-model, vis is the output image of the visible light multi-focus fusion sub-model, and nir is the output image of the near-infrared multi-exposure fusion sub-model.

[0070] Step 4: Minimize the loss function and train the distributed bandwidth product augmented deep network model from Step 3 to obtain a pre-trained distributed bandwidth product augmented deep network model.

[0071] The loss function L is as follows:

[0072] L = L1 + L2

[0073] In the formula: L1 is the loss function of the visible light multi-focus fusion sub-model and the near-infrared multi-exposure fusion sub-model, and L2 is the loss function of the detail reconstruction sub-network;

[0074]

[0075] L2 = Pix + PL + GL

[0076] In the formula: C is the dimension of the image, H is the height of the image, W is the width of the image, real is the ground truth image, and vis is the output image of the visible light multi-focus fusion sub-model. Here, is the gradient operator, F is the F-norm, Pix is ​​the pixel loss function, PL is the perceptual loss function, and GL is the gradient loss function.

[0077] The first half of L1 is the loss function of VIS-Net, the second half is the loss function of NIR-Net, and L2 is the loss function of the detail reconstruction subnetwork.

[0078] The pixel loss function is as follows:

[0079]

[0080] In the formula: R is the output image of the detail reconstruction sub-model;

[0081] The perceptual loss function is as follows:

[0082]

[0083] In the formula: j is the convolutional layer number, Here, x represents the pre-trained network model and the reconstructed image. The size of the generated convolutional layer of the network model is C. j ×H j ×W j Feature map;

[0084] The gradient loss function is as follows:

[0085]

[0086] Step 5: Input the acquired image into the pre-trained distributed bandwidth product augmented deep network model obtained in Step 4 to obtain the reconstructed image.

[0087] A distributed bandwidth product-enhanced deep network model is proposed to solve the inverse problem of imaging. It extracts spatial spectral information from the input image Ii and integrates the extracted information to generate an image that is close to the ideal image I'.

[0088]

[0089] In the formula: F θ Represents a distributed bandwidth product augmented deep network model, I1, I2, ..., I K The input image.

[0090] Imaging using the method of this invention shows that even with a large aperture, images captured in extremely low-light scenes suffer severe loss of sharpness due to the limited number of photons and noise, making it impossible to discern details such as the doll's head in the selected image. Furthermore, image blurring caused by the large aperture also degrades image quality. The image after coarse restoration using VIS-Net shows that although noise is removed, details are still lost. The image after NIR-Net fusion demonstrates that key information is effectively integrated after NIR-Net fusion, overcoming the effects of overexposure and underexposure; it integrates the advantages of both long and short exposures, restoring detailed scene information.

[0091] The final image obtained using the method of this invention shows that by further fusing the reconstructed structural information and texture details in the detail reconstruction sub-network, the resulting image has sharpness comparable to the ground truth, and accurate colors due to the preservation of structural information from the visible image. Details and colors are accurately restored. Magnified details are displayed within rectangular frames; they are very close to the ground truth image.

[0092] One hundred test samples were selected for testing, and the quantitative results are shown in Table 1. The table shows that the PSNR and SSIM of the reconstructed image are significantly higher than those of the input visible light image. The combined qualitative and quantitative results demonstrate that this method can effectively achieve high distributed bandwidth integrative imaging under extremely low illumination conditions (below 0.01 lux).

[0093] Table 1. Average quantitative evaluation results of the test samples

[0094]

[0095] This invention establishes a quantization imaging framework, bridging the imaging and algorithmic links to optimize the bandwidth product for feature applications. For example... Figure 1 In section A, the optical imaging model is mathematically represented, and sensor noise is quantified to characterize its impact on the resolution degradation of the imaging system. The distributed bandwidth product characterizes the ability of the imaging system, affected by aberrations and noise, to perceive effective information from different spatial locations. A multi-encoder / decoder computational imaging method that deeply integrates a multi-sensor optical encoder and a learnable decoder is provided, enabling high-quality low-light imaging, such as... Figure 1 As shown in B.

[0096] In optical encoders, imaging parameters (such as focal length, image distance, aperture, spectrum, and exposure) can be mathematically quantified, enabling flexible encoding, which is crucial for the implementation of distributed optical coding. The coding model, as shown in d, encodes the image distance or focal length based on geometric optical imaging principles, designing a multi-focus imaging model to capture light information at different focal lengths. Based on the difference between human vision and camera perception, the imaging aperture is encoded, designing a multi-aperture collaborative multispectral imaging model. A large aperture visible light path records scene structure information, while a small aperture near-infrared light path captures scene details. Based on the differences between long and short exposure imaging, a multi-exposure near-infrared imaging model is designed to mitigate noise in dark areas and overcome information loss in bright areas. Analysis of the system's distributed bandwidth product reveals that the encoded light information obtained by the imaging system has a high degree of information complementarity.

[0097] For optical encoders, a special decoder, namely a distributed bandwidth product enhanced deep network model, was designed, such as... Figure 1As shown in Figure e, the spatial-spectral information from the encoded optical image can be integrated to solve the image degradation caused by the limited degrees of freedom of the imaging system and achieve a significant improvement in the distributed bandwidth product. Figure 1 The illustrations are merely examples; the encoding and decoding methods can differ for different applications. For instance, for scenes requiring large depth of field, the number of imaging focal planes can be increased, and for extreme imaging conditions, the distributed bandwidth product enhancement model can be more complex. Combining a multi-dimensional optical coding model and a distributed bandwidth product enhancement deep network model provides a highly flexible new theory and approach for system construction, enabling high-quality enhanced imaging. By mathematically quantifying the complete imaging chain, the distributed bandwidth product enhancement theory is proposed, establishing a bridge between the imaging chain and the algorithm chain, and improving the coupling between the optical system and the post-processing algorithm. Our system can be deployed in the mass production of any visible-infrared camera model to achieve high-quality ultra-low-light imaging.

Claims

1. A method of high distributed bandwidth synthesis imaging based on multi-dimensional coding computation enhancement, characterized in that, Includes the following steps: Step 1: Construct an optical imaging model to acquire large-aperture multi-focus visible low-light images and small-aperture multi-exposure near-infrared images; Step 2: Obtain high-quality ground truth images of visible light with large depth of field, and construct a dataset with the images obtained in Step 1; Step 3: Construct a distributed bandwidth product-enhanced deep network model; The distributed bandwidth product enhanced deep network model includes a visible light multi-focus fusion sub-model for extracting features from large aperture multi-focus visible low-light images, a near-infrared multi-exposure fusion sub-model for extracting features from small aperture multi-exposure near-infrared images, and a detail reconstruction sub-model for fusing the extracted features from large aperture multi-focus visible low-light images and small aperture multi-exposure near-infrared images. Step 4: Minimize the loss function and train the distributed bandwidth product augmented deep network model from Step 3 to obtain a pre-trained distributed bandwidth product augmented deep network model. Step 5: Input the acquired image into the pre-trained distributed bandwidth product augmented deep network model obtained in Step 4 to obtain the reconstructed image.

2. A high distributed bandwidth integration imaging method based on multi-dimensional coding calculation enhancement according to claim 1, characterized in that, The optical imaging model in step 1 is a multi-coded optical imaging model, including a multi-focus imaging model that codes the imaging focus, a multi-aperture imaging model that codes the physical aperture, a multi-spectral imaging model that codes the imaging spectrum, and a multi-exposure near-infrared imaging model that codes long and short exposures.

3. The high distributed bandwidth product imaging method based on multidimensional encoding and decoding computation enhancement according to claim 2, characterized in that, The multi-coding optical imaging model is constructed by first calculating the distributed bandwidth product of the imaging system DBP : In the formula: FOV For the imaging field of view, To achieve the limit resolution of imaging systems affected by aberrations and noise, D For aperture size, l For object depth, λ For imaging wavelength, h For sensor size, Image distance, t For the exposure time, f Focal length For optical magnification, pix In pixels For spectral quantum efficiency, For the brightness of the light source, A s Where is the pixel area, and Dc is the dark current. σ read To read out noise, To quantize noise, g For voltage gain, ε The characteristic width of the point spread function.

4. The high distributed bandwidth product imaging method based on multidimensional encoding and decoding computation enhancement according to claim 1, characterized in that, The visible light multifocusing fusion sub-model includes, in sequence, the following: n Each convolutional downsampling module and m The convolutional upsampling module is used to extract spatial focal length structural feature information, and the convolutional upsampling module is used to fuse depth information. The near-infrared multi-exposure fusion sub-model structure is the same as the visible light multi-focus fusion sub-model structure; the detail reconstruction sub-model includes, in sequence, the following: a Each convolutional upsampling module and feature reconstruction layer.

5. The high distributed bandwidth product imaging method based on multidimensional encoding and decoding computation enhancement according to claim 4, characterized in that, The detail reconstruction sub-model also includes two feature extraction modules; in the visible light multi-focus fusion sub-model, the features extracted by each level of the convolution upsampling module are all input into the feature extraction module for feature extraction, and then input into the detail reconstruction module; in the near-infrared multi-exposure fusion sub-model, the features extracted by each level of the convolution upsampling module are all input into another feature extraction module for feature extraction, and then input into the detail reconstruction module.

6. The high distributed bandwidth product imaging method based on multidimensional encoding and decoding computation enhancement according to claim 5, characterized in that, The convolutional downsampling module includes one convolutional layer, two activation layers, one convolutional downsampling layer, and two normalization layers; the convolutional upsampling module includes two convolutional layers, two activation layers, one upsampling layer, and two normalization layers; the feature reconstruction layer includes one upsampling layer, three convolutional layers, two normalization layers, and three activation layers; and the feature extraction module includes one convolutional upsampling layer, one convolutional downsampling layer, two normalization layers, and two activation layers.

7. The high distributed bandwidth product imaging method based on multidimensional encoding and decoding computation enhancement according to claim 6, characterized in that, The image output by the distributed bandwidth product enhanced deep network model is the output of a high bandwidth product. R : In the formula: weight To reconstruct the output weight map of the sub-model for details, vis Output image for visible light multi-focus fusion sub-model. nir Output image for near-infrared multi-exposure fusion sub-model.

8. The high distributed bandwidth product imaging method based on multidimensional encoding and decoding computation enhancement according to claim 1, characterized in that, loss function in step 4 L as follows: In the formula: L 1 represents the loss function of the visible light multi-focus fusion sub-model and the near-infrared multi-exposure fusion sub-model. L 2 represents the loss function for reconstructing the detailed subnetwork; In the formula: C For the dimensions of the image, H The height of the image. W The width of the image. real For the true value image, vis Output image for visible light multi-focus fusion sub-model. For gradient operators, F It is the F-norm. Pix For pixel loss function, PL For the perceptual loss function, GL Let be the gradient loss function. nir Output image for near-infrared multi-exposure fusion sub-model.

9. A high-distributed bandwidth product imaging method based on multi-dimensional encoding and decoding computation enhancement according to claim 8, characterized in that, The pixel loss function is as follows: In the formula: R Output images for detailed reconstruction of sub-models; The perceptual loss function is as follows: In the formula: j The convolutional layer number. φ For pre-trained network models, x To reconstruct the image, φ j ( x ) is the network model of the first j The size generated by each convolutional layer is C j × H j × W j Feature map; The gradient loss function is as follows: 。 10. The imaging system of the high distributed bandwidth product imaging method based on multidimensional encoding and decoding computation enhancement as described in any one of claims 1 to 9, characterized in that, This includes an optical imaging module and a distributed bandwidth product-enhanced deep network model; The optical imaging module is a multi-coded optical imaging model, including a multi-focus imaging model that encodes the imaging focus, a multi-aperture imaging model that encodes the physical aperture, a multi-spectral imaging model that encodes the imaging spectrum, and a multi-exposure near-infrared imaging model that encodes long and short exposures; it is used to acquire large-aperture multi-focus visible light images and small-aperture multi-exposure near-infrared images, and uses them as input to the distributed bandwidth product enhanced deep network model; The distributed bandwidth product enhanced deep network model includes a visible light multi-focus fusion sub-model for extracting features from large-aperture multi-focus visible low-light images, a near-infrared multi-exposure fusion sub-model for extracting features from small-aperture multi-exposure near-infrared images, and a detail reconstruction sub-model for fusing the extracted features from large-aperture multi-focus visible low-light images and small-aperture multi-exposure near-infrared images; and a model for extracting spatial-spectral information from the input image to obtain the reconstructed image.

Images