Reflection-type optical system based on computational imaging drive processing simplification

Abstract

The invention relates to a reflection-type optical system based on computational imaging drive processing simplification, which comprises a reflection-type optical system, a detector and an image reconstruction system, and is characterized in that the reflection-type optical system comprises a first free-form surface reflector, a second even-order aspheric reflector and a third free-form surface reflector; the detector coincides with the focal plane of the reflective optical system, the image reconstruction system comprises an NPU image data processor, and the NPU image data processor runs an image restoration algorithm model; incident light sequentially passes through the first free-form surface reflecting mirror, the second even-order aspherical reflecting mirror and the third free-form surface reflecting mirror to be focused and then enters the detector, a detection result is transmitted to the image reconstruction system through the detector to be processed, and an image close to diffraction limit image quality is output. Compared with the prior art, the processing requirement of the optical system can be reduced by at least one order of magnitude, and the optical system has the imaging quality close to the diffraction limit image quality.

Description

Technical Field

[0001] The present invention relates to the field of reflective optical system structures, and in particular to a reflective optical system with simplified processing driven by computational imaging. Background Art

[0002] The reflective optical system structure is the mainstream configuration of space optical payloads, such as the reflective optical system disclosed in the invention with publication number CN115453732A, which includes a primary reflector, a secondary reflector, a first folding mirror, a second folding mirror, a third reflector, and a focal plane detector assembly; the optical axes of the above-mentioned folding mirrors, reflectors and focal plane detector assembly are located in the same plane; the aperture stop of the entire optical system coincides with the primary reflector; the light from the scene is reflected by the primary reflector and the secondary reflector in turn, and reaches the first folding mirror, the first folding mirror deflects the light to the second folding mirror, and then the second folding mirror deflects the light to the third reflector, and finally it is reflected and converged by the three reflectors to reach the focal plane detector assembly.

[0003] In order to achieve high-quality imaging, the surface shape of the reflective optical system has changed from the traditional spherical surface to the aspherical surface and free-form surface, and nanometer-level processing accuracy is required. Usually, the root mean square (RMS) value of the processing surface error is better than 1 / 50λ. In order to achieve such high processing accuracy, the current processing methods include: single-point diamond turning based on compensation correction, precision shaping based on small grinding head, ultra-precision shaping based on ion beam, and ultra-precision shaping based on magnetorheological fluid. Although these processing methods can achieve nanometer-level processing accuracy, they take a lot of time and are expensive, and cannot meet the needs of rapid manufacturing and mass modular production in commercial aerospace. Summary of the invention

[0004] The purpose of the present invention is to provide a reflective optical system with simplified processing based on computational imaging in order to overcome the defects of the above-mentioned prior art that the reflective optical system requires nanometer-level processing precision, consumes a lot of time and is expensive.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A reflective optical system based on computational imaging driven processing simplification, comprising a reflective optical system, a detector and an image reconstruction system, wherein the reflective optical system comprises a first free-form surface reflector, a second even-order aspheric reflector and a third free-form surface reflector, the detector coincides with the focal plane of the reflective optical system, and the image reconstruction system comprises an NPU image data processor, and the NPU image data processor runs an image restoration algorithm model;

[0007] The incident light is focused by the first free-form surface reflector, the second even-order aspheric reflector and the third free-form surface reflector in sequence and then incident on the detector. The detection result is transmitted to the image reconstruction system for processing through the detector, and an image with a quality close to the diffraction limit is output.

[0008] Furthermore, the F number of the reflective optical system is within the range of 1 to 14, and the entrance pupil diameter is within the range of 15 mm to 600 mm.

[0009] Furthermore, the processed surface shapes of the first free-form surface reflector, the second even-order aspheric reflector and the third free-form surface reflector all satisfy that the root mean square is better than λ / 2, wherein λ is the wavelength of the light wave.

[0010] Furthermore, the detector is a visible light COMS detector.

[0011] Furthermore, the image restoration algorithm model is a deep learning model based on physical priors, the physical priors are Wiener filtering algorithm models based on the point spread function of a reflective optical system, and the deep learning model is a dynamic convolution deblurring model based on a field of view attention mechanism.

[0012] Furthermore, the Wiener filtering algorithm model based on the point spread function of the reflective optical system performs filtering processing according to the input image and the corresponding point spread function PSFs, and adds a regularization parameter to finally obtain a deconvolution image.

[0013] Furthermore, the input of the dynamic convolution deblurring model based on the field of view attention mechanism includes a field of view coordinate matrix and a deconvolution image;

[0014] The dynamic convolutional deblurring model based on the field of view attention mechanism includes an interconnected FOV encoder and a deformable decoder.

[0015] Further, the FOV encoder includes a FOV module, a first encoding module, a second encoding module, a third encoding module and a fourth encoding module connected in sequence, the first encoding module includes convolutional layers Conv and ResBlock connected in sequence, and the second encoding module, the third encoding module and the fourth encoding module each include convolutional layers Conv and ResBlock with a stride of 2 connected in sequence;

[0016] The deformable decoder comprises a first decoding module, a second decoding module, a third decoding module, a fourth decoding module, a KPN module, a first connection module, a second connection module, a third connection module and a fourth connection module;

[0017] The first decoding module includes a deformable ResBlock, the second decoding module and the third decoding module each include a deconvolution layer Deconv, a convolution layer Conv and a deformable ResBlock connected in sequence, and the fourth decoding module includes a deconvolution layer Deconv, a convolution layer Conv, a deformable ResBlock and a convolution layer Conv connected in sequence; the first connection module includes an offset adjustment layer Offset Transfer, and the second connection module, the third connection module and the fourth connection module each include a bilinear upsampling layer Bilinear Upsampling and an offset adjustment layer OffsetTransfer connected in sequence;

[0018] The fourth encoding module, the first decoding module, the second decoding module, the third decoding module, the fourth decoding module and the KPN module are connected in sequence, the input of the first connection module is connected to the ResBlock of the fourth encoding module, and the output is used as the input of the first decoding module;

[0019] The input of the bilinear upsampling layer Bilinear Upsampling in the second connection module is the output of the first connection module, the input of the offset adjustment layer Offset Transfer also includes the output of the convolution layer Conv of the second decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the second decoding module; and the input of the convolution layer Conv of the second decoding module also includes the output of the ResBlock of the third encoding module;

[0020] The input of the bilinear upsampling layer Bilinear Upsampling in the third connection module is the output of the offset adjustment layer Offset Transfer of the second connection module, the input of the offset adjustment layer Offset Transfer also includes the output of the convolution layer Conv of the third decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the third decoding module; and the input of the convolution layer Conv of the third decoding module also includes the output of the ResBlock of the second encoding module;

[0021] The input of the bilinear upsampling layer Bilinear Upsampling in the fourth connection module is the output of the offset adjustment layer Offset Transfer of the third connection module and the output of the first convolution layer Conv of the fourth decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the fourth decoding module; and the input of the convolution layer Conv of the fourth decoding module also includes the output of the ResBlock of the first encoding module;

[0022] The input of the KPN module is also connected to the input deconvolution image, and after corresponding superposition processing with the input feature map, it is summarized and convolution operation is performed to obtain the final output.

[0023] Furthermore, the FOV module first extracts features of the input field of view coordinate matrix FOV through a convolution layer, and adds a sigmoid activation function to obtain a FOV feature map; and processes the input deconvolution image through a convolution layer to obtain a deconvolution feature map; then multiplies the FOV feature map with the deconvolution feature map and superimposes the deconvolution feature map with the deconvolution feature map to obtain an output;

[0024] The deformable ResBlock extracts features according to the offset Offset output by the corresponding connection module and the input convolution layer Conv signal, and then performs a convolution operation and superimposes the convolution layer Conv signal with the input to obtain an output.

[0025] Furthermore, the NPU image data processor is an AI chip with the ability to deploy neural networks at the edge, which is used to deploy image restoration algorithm models and meet the computing power required for video-level real-time reasoning.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] (1) The present invention proposes to shift the focus of the reflective optical system from the system processing accuracy to the computational imaging technology, set up an NPU image data processor in the image reconstruction system, and deploy an image restoration algorithm model to process the collected image data in real time, thereby realizing efficient image restoration, recognition and analysis, thereby supporting the system to accurately image and reconstruct high-quality images under complex optical conditions; thereby, the optical component processing requirements can be effectively reduced, and the processing accuracy requirements can be reduced by more than an order of magnitude compared to the traditional system, which can greatly improve the optical component processing efficiency, reduce the optical system cost, and improve the imaging quality.

[0028] (2) The present invention introduces computational imaging to perform aberration correction and uses a suitable image processing algorithm to effectively restore the degraded image of the optical system to an excellent state.

[0029] (3) The physical prior deep learning model of the present invention organically combines the Wiener filtering algorithm based on the system point spread function and the dynamic convolution deblurring algorithm based on the field of view attention mechanism, which can achieve high-quality restoration of degraded images with drastic spatial degradation and the presence of large and small aberrations, and the restored image performance is close to the diffraction limit. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1A schematic diagram of the structure of a reflective optical system based on computational imaging driven processing simplification provided in an embodiment of the present invention;

[0031] Figure 2 An off-axis three-reflection light path diagram provided in an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of an optical element processing error boundary provided in an embodiment of the present invention;

[0033] Figure 4 A schematic diagram of an image restoration algorithm model provided in an embodiment of the present invention;

[0034] Figure 5 A schematic diagram of a FOV_KPN neural network model provided in an embodiment of the present invention;

[0035] Figure 6 A comparison diagram before and after image restoration provided in an embodiment of the present invention;

[0036] In the figure, 1. reflective optical system, 101. first free-form surface reflector, 102. second even-order aspheric reflector, 103. third free-form surface reflector, 2. detector, 3. image reconstruction system, 301. image restoration algorithm model, 302. NPU image data processor. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0038] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0040] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, or are the directions or positional relationships in which the inventive product is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0041] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0042] In addition, the terms "horizontal", "vertical" and the like do not mean that the components are required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0043] Example 1

[0044] In view of the high technical requirements and cost consumption of the reflective surface type of the reflective optical system structure in diamond turning processing, computational imaging can be used to correct the aberration technology and shift the focus of problem processing from optical hardware to image processing algorithms. In this regard, this embodiment proposes a reflective optical system based on computational imaging driven processing simplification. By introducing computational imaging for aberration correction, the processing accuracy requirements are reduced while the imaging quality can be guaranteed.

[0045] like Figure 1 As shown, it specifically includes a reflective optical system 1, a detector 2 and an image reconstruction system 3, the reflective optical system 1 includes a first free-form surface reflector 101, a second even-order aspheric reflector 102 and a third free-form surface reflector 103, the detector 2 coincides with the focal plane of the reflective optical system 1, and the image reconstruction system 3 includes an NPU image data processor 302, and the NPU image data processor 302 runs an image restoration algorithm model 301;

[0046] The incident light is focused by the first free-form surface reflector 101, the second even-order aspheric reflector 102 and the third free-form surface reflector 103 in sequence and then incident on the detector 2. The detection result is transmitted to the image reconstruction system 3 for processing through the detector 2, and an image with a quality close to the diffraction limit is output.

[0047] In this embodiment, the reflective optical system adopts an off-axis three-mirror system, such as Figure 2 When the processing error data is not added, the field of view of the system is 8.0°×6.4°, the F number is 2.8, the wavelength range is from 400nm to 1050nm, the maximum RMS spot diameter of the system's full field of view is 0.988μm, and the MTF of the system exceeds 0.78 at the Nyquist frequency of 72lp / mm.

[0048] In a reflective optical system, the smaller the F number of the reflective optical system, the larger the entrance pupil diameter, the stronger the light-gathering ability, the larger the effective entrance pupil, and the smaller the corresponding field of view; according to the detector model and parameters, the actual parameters are selected by compromising the field of view and light-gathering ability, then the F number of the reflective optical system should be limited to 1 to 14, and the entrance pupil diameter should be limited to 15 mm to 600 mm.

[0049] In the reflective optical system structure, the processing accuracy of the reflective surface determines the imaging quality. The smaller the processing surface error data, the better the imaging quality, and vice versa. Generally, to obtain an optical system with imaging quality close to the diffraction limit, the processing error needs to be less than λ / 50; and the present invention can use computational imaging technology to reduce the requirements for the processing quality of the reflective surface. According to actual preliminary experiments and image restoration experience, the image restoration can be relatively good when the MTF of the imaging system is greater than 0.3. Therefore, the corresponding processing error level that can ensure that the system MTF is greater than 0.3 is calculated. The final calculated processing error boundary is as follows: Figure 3 As shown, in the system with the image preprocessing deconvolution module added, the processing errors of the first free-form surface reflector 101, the second even-order aspheric reflector 102 and the third free-form surface reflector 103 of the system should satisfy the requirements that the primary mirror is less than 0.675λ, the secondary mirror is less than 0.924λ, and the third mirror is less than 0.556λ respectively. Therefore, as long as the processing errors of the processing surface shapes of the optical system satisfy the RMS better than λ / 2, they can meet the requirements.

[0050] In the detector image acquisition system 2, the actual detector selected is a visible light COMS detector, whose pixel size is 6.9 μm and the total number of pixels is 1224*10244.

[0051] After the optical system is set up to obtain the PSF, the image is imported into the image reconstruction system for image restoration. Figure 4 shown.

[0052] The image reconstruction system 3 includes an NPU image data processor 302, which is an AI chip with the ability to deploy neural network edge, and has the computing power required to deploy the deep learning-based image restoration algorithm model 301 and meet the video-level 25Hz real-time reasoning. The AI ​​chip realizes efficient image restoration, recognition and analysis by real-time processing of the collected image data, thereby supporting the system's accurate imaging and high-quality image reconstruction under complex optical conditions.

[0053] The image restoration algorithm model 301 is a deep learning model based on physical priors, wherein the physical priors are Wiener filtering algorithm models based on the point spread function of a reflective optical system, and the deep learning model is a dynamic convolution deblurring model based on a field of view attention mechanism.

[0054] The network structure is as follows Figure 4 As shown, the Wiener filtering algorithm model based on the point spread function of the reflective optical system performs filtering processing according to the input image and the corresponding point spread function PSFs, and adds a regularization parameter to finally obtain a deconvolution image;

[0055] like Figure 5 As shown, a dynamic convolution deblurring model based on the field of view attention mechanism, namely, the FOV_KPN neural network, has a deconvolution image and a field of view coordinate matrix as input, and a network structure including a FOV encoder and a deformable decoder connected to each other. The FOV encoder includes a FOV module, a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence. The first encoding module includes convolutional layers Conv and ResBlock connected in sequence, and the second encoding module, the third encoding module, and the fourth encoding module all include convolutional layers Conv and ResBlock connected in sequence with a stride of 2.

[0056] The deformable decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module, a KPN module, a first connection module, a second connection module, a third connection module and a fourth connection module;

[0057] The first decoding module includes a deformable ResBlock, the second decoding module and the third decoding module each include a deconvolution layer Deconv, a convolution layer Conv and a deformable ResBlock connected in sequence, and the fourth decoding module includes a deconvolution layer Deconv, a convolution layer Conv, a deformable ResBlock and a convolution layer Conv connected in sequence; the first connection module includes an offset adjustment layer Offset Transfer, and the second connection module, the third connection module and the fourth connection module each include a bilinear upsampling layer Bilinear Upsampling and an offset adjustment layer Offset Transfer connected in sequence;

[0058] The fourth encoding module, the first decoding module, the second decoding module, the third decoding module, the fourth decoding module and the KPN module are connected in sequence, the input of the first connection module is connected to the ResBlock of the fourth encoding module, and the output is used as the input of the first decoding module;

[0059] The input of the bilinear upsampling layer Bilinear Upsampling in the second connection module is the output of the first connection module, the input of the offset adjustment layer Offset Transfer also includes the output of the convolution layer Conv of the second decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the second decoding module; and the input of the convolution layer Conv of the second decoding module also includes the output of the ResBlock of the third encoding module;

[0060] The input of the bilinear upsampling layer Bilinear Upsampling in the third connection module is the output of the offset adjustment layer Offset Transfer of the second connection module, the input of the offset adjustment layer Offset Transfer also includes the output of the convolution layer Conv of the third decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the third decoding module; and the input of the convolution layer Conv of the third decoding module also includes the output of the ResBlock of the second encoding module;

[0061] The input of the bilinear upsampling layer Bilinear Upsampling in the fourth connection module is the output of the offset adjustment layer Offset Transfer of the third connection module and the output of the first convolution layer Conv of the fourth decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the fourth decoding module; and the input of the convolution layer Conv of the fourth decoding module also includes the output of the ResBlock of the first encoding module;

[0062] The input of the KPN module is also connected to the input deconvolution image, and after corresponding superposition processing with the input feature map, it is summarized and convolution operation is performed to obtain the final output.

[0063] Among them, the FOV module first extracts features from the input field of view coordinate matrix FOV through the convolution layer, and adds the sigmoid activation function to obtain the FOV feature map; and processes the input deconvolution image through the convolution layer to obtain the deconvolution feature map; then the FOV feature map is multiplied by the deconvolution feature map and superimposed with the deconvolution feature map to obtain the output;

[0064] The deformable ResBlock extracts features based on the offset Offset output by the corresponding connection module and the input convolution layer Conv signal, and then performs a convolution operation and superimposes it with the input convolution layer Conv signal to obtain the output.

[0065] The physical prior deep learning model organically combines the Wiener filtering algorithm based on the system point spread function and the dynamic convolution deblurring algorithm based on the field of view attention mechanism. It can achieve high-quality restoration of degraded images with drastic spatial degradation and large and small aberrations. The restored image performance is close to the diffraction limit.

[0066] Based on the measured PSF of the optical system, this embodiment trains the network of the image restoration algorithm model 301 and performs actual shooting measurements on the trained system. Figure 6 shown. Figure 6 (1) shows the MTF comparison of the image before and after restoration using the blade edge method. Figure 6 (2) shows the comparison between the real shot image, the restored real shot image and the image of the Canon lens with good existing imaging quality. It can be seen that this method can restore the image to a level with MTF greater than 0.5, showing relatively good performance in the real shooting process.

[0067] In summary, the all-aluminum reflective optical system with simplified processing driven by computational imaging provided in this embodiment introduces computational imaging for aberration correction, and the processing accuracy requirement is reduced by more than an order of magnitude compared with the traditional system. It has low manufacturing cost, short development cycle and good imaging quality.

[0068] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A reflective optical system based on computational imaging driven processing simplification, characterized in that: The invention comprises a reflective optical system (1), a detector (2) and an image reconstruction system (3), wherein the reflective optical system (1) comprises a first free-form surface reflector (101), a second even-order aspheric reflector (102) and a third free-form surface reflector (103), the detector (2) coincides with the focal plane of the reflective optical system (1), and the image reconstruction system (3) comprises an NPU image data processor (302), and the NPU image data processor (302) runs an image restoration algorithm model (301); The incident light is focused by the first free-form surface reflector (101), the second even-order aspheric reflector (102) and the third free-form surface reflector (103) in sequence and then incident on the detector (2). The detection result is transmitted to the image reconstruction system (3) for processing through the detector (2), and an image with a quality close to the diffraction limit is output.

2. A reflective optical system based on computational imaging driven processing simplification according to claim 1, characterized in that: The F number of the reflective optical system (1) is within the range of 1 to 14, and the entrance pupil diameter is within the range of 15 mm to 600 mm.

3. The reflective optical system based on computational imaging driven processing simplification according to claim 1, characterized in that: The processed surface shapes of the first free-form surface reflector (101), the second even-order aspheric reflector (102) and the third free-form surface reflector (103) all satisfy that the root mean square is better than λ / 2, where λ is the wavelength of the light wave.

4. The reflective optical system based on computational imaging driven processing simplification according to claim 1, characterized in that: The detector (2) is a visible light COMS detector.

5. The reflective optical system based on computational imaging driven processing simplification according to claim 1, characterized in that: The image restoration algorithm model (301) is a deep learning model based on a physical driving model, the physical driving model is a Wiener filtering algorithm model based on a point spread function of a reflective optical system, and the deep learning model is a dynamic convolution deblurring model based on a field of view attention mechanism.

6. The reflective optical system based on computational imaging driven processing simplification according to claim 5, characterized in that: The Wiener filtering algorithm model based on the point spread function of the reflective optical system performs filtering processing according to the input image and the corresponding point spread function PSFs, and adds a regularization parameter to finally obtain a deconvolution image.

7. The reflective optical system based on computational imaging driven processing simplification according to claim 6, characterized in that: The input of the dynamic convolution deblurring model based on the field of view attention mechanism includes a field of view coordinate matrix and a deconvolution image; The dynamic convolutional deblurring model based on the field of view attention mechanism includes an interconnected FOV encoder and a deformable decoder.

8. The reflective optical system based on computational imaging driven processing simplification according to claim 7, characterized in that: The FOV encoder includes a FOV module, a first encoding module, a second encoding module, a third encoding module and a fourth encoding module connected in sequence, the first encoding module includes a convolutional layer Conv and a ResBlock connected in sequence, and the second encoding module, the third encoding module and the fourth encoding module each include a convolutional layer Conv and a ResBlock with a stride of 2 connected in sequence; The deformable decoder comprises a first decoding module, a second decoding module, a third decoding module, a fourth decoding module, a KPN module, a first connection module, a second connection module, a third connection module and a fourth connection module; The first decoding module includes a deformable ResBlock, the second decoding module and the third decoding module each include a deconvolution layer Deconv, a convolution layer Conv and a deformable ResBlock connected in sequence, and the fourth decoding module includes a deconvolution layer Deconv, a convolution layer Conv, a deformable ResBlock and a convolution layer Conv connected in sequence; the first connection module includes an offset adjustment layer Offset Transfer, and the second connection module, the third connection module and the fourth connection module each include a bilinear upsampling layer Bilinear Upsampling and an offset adjustment layer Offset Transfer connected in sequence; The fourth encoding module, the first decoding module, the second decoding module, the third decoding module, the fourth decoding module and the KPN module are connected in sequence, the input of the first connection module is connected to the ResBlock of the fourth encoding module, and the output is used as the input of the first decoding module; The input of the bilinear upsampling layer Bilinear Upsampling in the second connection module is the output of the first connection module, the input of the offset adjustment layer Offset Transfer also includes the output of the convolution layer Conv of the second decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the second decoding module; and the input of the convolution layer Conv of the second decoding module also includes the output of the ResBlock of the third encoding module; The input of the bilinear upsampling layer Bilinear Upsampling in the third connection module is the output of the offset adjustment layer Offset Transfer of the second connection module, the input of the offset adjustment layer Offset Transfer also includes the output of the convolution layer Conv of the third decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the third decoding module; and the input of the convolution layer Conv of the third decoding module also includes the output of the ResBlock of the second encoding module; The input of the bilinear upsampling layer Bilinear Upsampling in the fourth connection module is the output of the offset adjustment layer Offset Transfer of the third connection module and the output of the first convolution layer Conv of the fourth decoding module, and the output of the offset adjustment layer Offset Transfer is connected to the deformable ResBlock of the fourth decoding module; and the input of the convolution layer Conv of the fourth decoding module also includes the output of the ResBlock of the first encoding module; The input of the KPN module is also connected to the input deconvolution image, and after corresponding superposition processing with the input feature map, it is summarized and convolution operation is performed to obtain the final output.

9. The reflective optical system based on computational imaging driven processing simplification according to claim 8, characterized in that: The FOV module first extracts features of the input field of view coordinate matrix FOV through the convolution layer, and adds a sigmoid activation function to obtain a FOV feature map; and processes the input deconvolution image through the convolution layer to obtain a deconvolution feature map; then multiplies the FOV feature map with the deconvolution feature map and superimposes the deconvolution feature map with the deconvolution feature map to obtain an output; The deformable ResBlock extracts features according to the offset Offset output by the corresponding connection module and the input convolution layer Conv signal, and then performs a convolution operation and superimposes the convolution layer Conv signal with the input to obtain an output.

10. The reflective optical system based on computational imaging driven processing simplification according to claim 1, characterized in that: The NPU image data processor (302) is an AI chip with neural network edge deployment capability, which is used to deploy the image restoration algorithm model (301) and meet the computing power required for video-level real-time reasoning.

Images