Design method of end-to-end refractive-diffractive hybrid imaging system based on ray tracing

By combining a full-ray tracing model with an aberration correction neural network, the limitations of accuracy and application of refractive and diffractive optical elements in end-to-end optical imaging systems are solved, realizing a high-precision and high-real-time hybrid refractive and diffractive imaging design.

CN118962975AActive Publication Date: 2024-11-15ZHEJIANG UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411120305.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-15
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

In existing end-to-end optical imaging system designs, the model conversion accuracy of refractive and diffractive optical elements suffers from loss, manufacturing limitations, and restricted application range, making it difficult to achieve high-precision imaging, especially in complex hybrid refractive and diffractive optical systems.

Method used

A full ray tracing model is adopted, incorporating refractive and diffractive optical elements into a unified framework. An aberration correction neural network is designed, and combined with Snell's theorem and the local grating equation, a unified ray tracing model for a hybrid refractive and diffractive optical system is constructed. This model is then optimized end-to-end using a U-shaped image restoration neural network.

Benefits of technology

It achieves high-precision and high-real-time performance in hybrid refractive index imaging, and is applicable to various types of hybrid refractive index optical systems, improving the accuracy and flexibility of imaging systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118962975A_ABST
    Figure CN118962975A_ABST
Patent Text Reader

Abstract

The invention discloses a ray tracing-based end-to-end refractive-diffractive hybrid imaging system design method, which comprises the following steps of: finding an intersection point of a ray and an optical element by using a Newton method, realizing ray tracing of a refractive optical element by using a Snell law, and realizing ray tracing of a diffractive optical element by using a local grating equation. Differentiatable ray tracing of all optical paths is realized; a simulated aberration picture is calculated by using a ray tracing result, an image restoration neural network sensitive to local texture of the image is constructed, the aberration picture is input, and the picture with partial aberration removed is output; and constructing a loss function by taking the quality of the finally output picture as a judgment standard, and simultaneously carrying out end-to-end gradient descent optimization on optical parameters and neural network parameters to realize design of the refractive-diffractive hybrid optical element and deep learning collaborative imaging system. According to the invention, a new ray tracing frame is constructed for the diffractive hybrid optical system, end-to-end optimization is realized, the degree of adaptation between the optical system and an optimization algorithm is increased, and high-quality imaging is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of optical design and deep learning, and in particular to a design method of an end-to-end refractive-diffractive hybrid imaging system based on ray tracing. Background Art

[0002] End-to-end optical design has recently attracted much attention. This integrated approach aims to find the most efficient combination of optical components and backend algorithms for a specific task by simultaneously optimizing optical systems and neural networks via gradient backpropagation. In this framework, optical elements such as refractive lenses, diffractive optical elements, or metasurfaces are part of the overall imaging network. They collaborate with the backend processing network to perform the imaging task and reconstruct the final output, which includes an image or other visual representation. This approach shows great promise in scientific imaging, computer vision, and the evolution of optical systems beyond traditional lens design methods.

[0003] Diffractive optical elements control the phase of light by optimizing the design of the phase plane. Compared with traditional lenses, they have opposite chromatic aberration and thermal aberration characteristics and have powerful aberration correction capabilities. The diffraction grating invented at the end of the 18th century was the earliest diffractive optical element and was widely used in optical instruments. In 1820, Fresnel proposed the Fresnel lens and successfully developed it in 1822. In the mid-1980s, the MIT Lincoln Laboratory in the United States took the lead in proposing the concept of "binary optics". It is based on the diffraction theory of broadcasting, and a relief structure with two or more step depths is etched on the surface of traditional optical elements to form a type of diffractive optical element with pure phase, coaxial reproduction, and extremely high diffraction efficiency. In 1988, Swanson Veuknmp and others used the dispersion characteristics of diffractive optical elements to correct the axial chromatic aberration and spherical aberration of single lenses and developed multi-order phase lenses. Since then, the application research of diffractive optical elements in the field of optical imaging has begun. Refractive-diffractive hybrid lenses combine the advantages of refractive and diffractive optical elements, showing great promise for the next generation of optical imaging systems, especially in the field of infrared optics. However, current differentiable image formation models do not fully support the end-to-end design of more complex refractive-diffractive hybrid optical systems. Traditional wave optics models usually oversimplify diffractive elements into thin phase plates, ignoring optical aberrations. Recent researchers use ray models for refractive lenses and wave diffraction models for diffractive optical elements at the end of the optical system. This model has three main problems. The first is that the conversion between rays and waves will lead to a loss of accuracy. The second is that the diffractive elements in this model can only be processed on a flat plate, and the tracking of diffractive elements processed on an aspherical surface cannot be achieved. The third is that it only supports placing the diffractive element at the very end of the optical system, which limits its scope of application. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes an end-to-end refractive-diffractive hybrid imaging system design method based on ray tracing. The method adopts a full ray tracing model, incorporates refractive optical elements and diffractive optical elements into a unified framework, and designs an aberration correction neural network based on aberration characteristics. The method can realize the end-to-end collaborative design of high-precision refractive-diffractive hybrid optical imaging systems, and can be applied to various forms of refractive-diffractive hybrid optical systems.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A method for designing an end-to-end hybrid imaging system based on ray tracing comprises the following steps:

[0007] S1: Establish a virtual refractive-diffractive hybrid optical structure and the initial structure of the detector plane as a virtual optical system; and construct a data set consisting of real clear pictures;

[0008] S2: Establish a ray tracing model for the refractive-diffractive hybrid optical system. For the refractive surface, use Snell's theorem to calculate the light deflection direction. For the diffractive surface, use the local grating equation to calculate the light deflection direction.

[0009] S3: setting a clear image in the data set as a virtual object plane at a preset object distance, sampling the object plane light to obtain the initial light, and propagating it into the virtual optical system, using the ray tracing model established in S2, tracing the light from the incident point to the detector plane, and obtaining a simulated aberration-containing degraded image;

[0010] S4: Establish an image restoration neural network, the input of which is a simulated degraded image with aberration, and the output is a picture with some aberrations removed; the image restoration neural network as a whole is a U-shaped structure, including a downsampling part and an upsampling part, and a full-field attention module connecting the downsampling part and the upsampling part; the downsampling part and the upsampling part each include a 3×3 convolution module and four full-field attention modules, and the corresponding full-field attention modules of the downsampling part and the upsampling part are jump-connected to promote the transmission of effective information; the full-field attention module includes two parts, the first part is five layers, which are LN layer, 1×1 convolution layer, 3×3 convolution layer, activation function ReLU and full-dimensional dynamic convolution in sequence; the second part is four layers, which are LN layer, 1×1 convolution layer, activation function ReLU and full-dimensional dynamic convolution layer in sequence; the input and output of the first part of the full-field attention module are added element-wise and used as the input of the second part; the input of the second part and the output of the second part are added element-wise and used as the output of the entire full-field attention module;

[0011] The full-dimensional dynamic convolution layer adopts a parallel strategy inside any convolution kernel to learn the attention value inside the convolution kernel from four dimensions: position, input channel, filter, and convolution kernel, thereby obtaining the full-dimensional convolution kernel attention value;

[0012] S5: inputting the simulated aberration-containing picture obtained in step S3 into the image restoration neural network, constructing a loss function, calculating the error between the picture output by the image restoration neural network with some aberrations removed and the clear picture actually taken, and optimizing the parameters of the refractive-diffractive hybrid optical structure and the neural network parameters simultaneously by using back-propagation gradient descent;

[0013] S6: Using all the real clear pictures in the data set, repeat S3 to S6 until clear imaging is achieved, obtain the optimized refractive-diffractive hybrid optical structure and image restoration neural network, and complete the design of the end-to-end refractive-diffractive hybrid imaging system based on ray tracing.

[0014] Furthermore, the initial structure of the refractive-diffractive hybrid optical structure is two flat plates, along the direction from the incident light to the outgoing light, the first light-transmitting surface of the first flat plate is set as a diffractive surface, and the remaining light-transmitting surfaces of the two flat plates are set as even-refractive aspheric surfaces;

[0015] The expressions for all refractive and diffractive surfaces are:

[0016]

[0017] Where c is the curvature, k is the cone coefficient, α 2x is the high-order coefficient of the aspheric surface, r is the radial distance, x is the aspheric order, and X is the maximum aspheric order;

[0018] The additional phase introduced by the diffraction surface is:

[0019]

[0020] Among them, β y is the phase coefficient of the diffraction surface, R is the maximum radius of the diffraction element, y is the phase order, and Y is the maximum phase order.

[0021] Furthermore, in S2, the direction vector d of the outgoing light of the refractive surface t The vector form of the expression is

[0022]

[0023] Among them, n1 is the normal vector of the incident point of the light on the refractive surface, d i is the direction vector of the incident light, n i is the refractive index of the material on the incident side, n tis the refractive index of the material on the output side;

[0024] The vector form of the outgoing light from the diffraction surface is expressed as:

[0025]

[0026] Among them, n2 is the normal vector of the incident point of the aspherical surface of the diffraction surface base, λ is the wavelength of the incident light, m ​​is the selected diffraction order, φ(r) is the additional phase introduced by the diffraction surface, and f is the direction vector of the local grating.

[0027] Furthermore, in S3, the Newton iteration method is used to calculate the intersection of the light ray and the virtual optical system to obtain the incident point; the light deflection angles are calculated for the refractive surface and the diffraction surface respectively, and the light ray is traced to the detector plane through the refractive surface and the diffraction surface; for the light ray on the detector plane, it is weighted according to the intensity of its virtual image plane sampling point to obtain a simulated aberration-containing degraded image P1.

[0028] Furthermore, the loss function in S5 is expressed as:

[0029] L=||P2(θ,n)-P0|| 2 +εL opt

[0030] Among them, the first term in the formula represents the gap between the output image of the image restoration neural network and the real image, P2 represents the picture output by the image restoration neural network, θ represents the parameters of the virtual optical system, n represents the parameters of the image restoration neural network; ε is a set proportional coefficient, L opt Represents the limitations of the virtual optical system itself.

[0031] The beneficial effects of the present invention are as follows:

[0032] The end-to-end refractive-diffractive hybrid imaging system design method proposed in the present invention constructs a unified ray tracing model of the refractive-diffractive hybrid optical system through Snell's law and a local grating equation model, and proposes an aberration recovery network that can enhance local texture features, thereby realizing the end-to-end optical design of any refractive-diffractive hybrid system; the refractive-diffractive hybrid imaging system obtained by the end-to-end design of the present invention can achieve high-precision and high-real-time imaging, and the design method can be adapted to the design of various forms of refractive-diffractive hybrid optical systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The present invention is a flowchart of a method for designing an end-to-end refractive-diffractive hybrid imaging system based on ray tracing in an embodiment of the present invention.

[0034] Figure 2 Graph showing the operating results of the ray tracing method in an embodiment of the present invention.

[0035] Figure 3 4 is a structural diagram of an image restoration neural network in an embodiment of the present invention.

[0036] Figure 4 It is the direct imaging result of the end-to-end design in the embodiment of the present invention, where (a) is the aberration-free clear picture of the input system, (b) is the direct imaging result through the optical system, and (c) is the restoration result of the neural network. DETAILED DESCRIPTION

[0037] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more clear. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] like Figure 1 As shown, the end-to-end refractive-diffractive hybrid imaging system design method based on ray tracing of this embodiment includes the following steps:

[0039] Step 1: Establish a virtual refractive-diffractive hybrid optical structure and the initial structure of the detector plane as a virtual optical system; and construct a data set consisting of real clear pictures.

[0040] As one of the implementation methods, in order to reflect the optimization capability of the method of the present invention, the initial structure of the refractive-diffractive hybrid optical structure is two flat plates. Along the direction from the incident light to the outgoing light, the first light-transmitting surface of the first flat plate is set as a binary diffraction surface, and the remaining light-transmitting surfaces of the two flat plates are set as even-order refractive aspheric surfaces. The diffraction surface can be processed on any substrate. In this embodiment, the substrate takes an even-order aspheric surface as an example.

[0041] The expressions for diffractive base aspheric surfaces and refractive aspheric surfaces are both:

[0042]

[0043] Where c is the curvature, k is the cone coefficient, α 2x is the high-order coefficient of the aspheric surface, r is the radial distance, x is the aspheric order, and X is the maximum aspheric order.

[0044] The additional phase introduced by the diffraction surface is:

[0045]

[0046] Among them, β y is the phase coefficient of the diffraction surface, R is the maximum radius of the diffraction element, y is the phase order, and Y is the maximum phase order.

[0047] Step 2: Establish a ray tracing model for the refractive-diffractive hybrid optical system. For the refractive surface, use Snell's theorem to calculate the light deflection direction. For the diffractive surface, use the local grating equation to calculate the light deflection direction, thus forming a unified ray tracing model.

[0048] The direction vector d of the outgoing light from the refractive surface t The vector form of the expression is

[0049]

[0050] Among them, n1 is the normal vector of the incident point of the light on the refractive surface, d i is the direction vector of the incident light, n i is the refractive index of the material on the incident side, n t is the refractive index of the material on the output side.

[0051] The vector form of the local grating equation of the diffraction surface is expressed as:

[0052]

[0053] Among them, n2 is the normal vector of the incident point of the aspherical surface of the diffraction surface base described in S1, λ is the wavelength of the incident light, m ​​is the selected diffraction order, φ(r) is the additional phase introduced by the diffraction surface described in S1, and f is the direction vector of the local grating.

[0054] Through vector calculation, the expression of the vector form of the outgoing light of the diffraction surface can be obtained as follows:

[0055]

[0056]

[0057] Among them, n2 is the normal vector of the incident point of the aspherical surface of the diffraction surface base, λ is the wavelength of the incident light, m ​​is the selected diffraction order, φ(r) is the additional phase introduced by the diffraction surface, and f is the direction vector of the local grating. This allows the tracing of refractive elements and diffractive elements to be incorporated into a unified framework.

[0058] Step 3: Set a clear image in the data set as a virtual object plane at a preset object distance, sample the object plane light to obtain the initial light, and propagate it into the virtual optical system. Use the ray tracing model established in S2 to trace the light from the incident point to the detector plane to obtain a simulated aberration-degraded image.

[0059] In this embodiment, the object distance is set to 8000 meters. According to the size of the detector, the light is traced backward and intersected with the object surface to obtain the size of the virtual object surface. Then, for each pixel point of the virtual image surface, 64 rays are sampled and forward propagated. Figure 2 As shown, Newton's iteration method is used to calculate the intersection of the light and the optical system to obtain the incident point, and the light deflection angles are calculated for the refractive surface and the diffraction surface respectively. The light passes through the refractive surface and the diffraction surface and is traced to the detector plane. The light on the detector plane is weighted according to the intensity of its virtual image plane sampling point to obtain a simulated aberration-containing degraded image P1.

[0060] Step 4: Establish an image restoration neural network, such as Figure 3 As shown in the figure, the input of the image restoration neural network is a simulated aberration-containing degraded image, and the output is a picture with some aberrations removed; the image restoration neural network is a U-shaped structure as a whole, including a downsampling part and an upsampling part, and a full-field attention module FDA-Block connecting the downsampling part and the upsampling part; the downsampling part and the upsampling part each include a 3×3 convolution module and four full-field attention modules FDA-Block, and the corresponding full-field attention modules of the downsampling part and the upsampling part are jump-connected to promote the transmission of effective information; the full-field attention module includes two parts, the first part has five layers, which are LN layer, 1×1 convolution layer, 3×3 convolution layer, activation function ReLU and full-dimensional dynamic convolution in sequence; the second part has four layers, which are LN layer, 1×1 convolution layer, activation function ReLU and full-dimensional dynamic convolution layer in sequence; the input and output of the first part of the full-field attention module FDA-Block are added element-wise and used as the input of the second part; the input of the second part and the output of the second part are added element-wise and used as the output of the entire full-field attention module.

[0061] In this embodiment, in the input and output of the network, based on the degradation characteristics of the aberration as the field of view changes, the field of view prior is introduced into the input. Specifically, the input one-channel blurred picture is converted into a three-channel input: one channel blurred picture, one channel field of view change in the X direction, and one channel field of view change in the Y direction. The input of the three channels is 640×512 in size, and the final output is a single-channel 640×512 clear picture. This input can be divided into regions for learning different fields of view, and different restoration measures can be taken to effectively improve the restoration quality of the picture, especially the field of view near the edge. In addition, the residual connection of the overall structure of the network is also used, that is, the network does not learn the output picture, but the difference between the output picture and the input picture.

[0062] The full-dimensional dynamic convolution layer adopts a parallel strategy inside any convolution kernel to learn the attention value inside the convolution kernel from four dimensions: position, input channel, filter, and convolution kernel, thereby obtaining the full-dimensional convolution kernel attention value. This can better integrate the input field of view prior, and make the weights of coarse texture and fine texture different, strengthen local texture learning, and play a role in enhancing attention.

[0063] Step 5: Input the simulated aberration-containing image obtained in step 3 into the image restoration neural network, construct a loss function, calculate the error between the image output by the image restoration neural network with some aberrations removed and the real clear image, and use back-propagation gradient descent to simultaneously optimize the parameters of the refractive-diffractive hybrid optical structure and the neural network parameters.

[0064] The output image of the neural network is recorded as P2, which is related to the parameter θ of the optical system and the parameter n of the neural network, so it can be expressed as P2(θ,n). The expression of the constructed loss function is as follows:

[0065] L=||P2(θ,n)-P0|| 2 +εL opt

[0066] The first term represents the difference between the neural network output image and the real image, ε is a set proportional coefficient, and L opt Represents the limiting requirements of the optical system itself, such as aperture size, maximum lens thickness, minimum air gap, etc.

[0067] Step 6: Using all the real clear pictures in the data set, repeat steps 3 to 6 until clear imaging is achieved, obtain the optimized refractive-diffractive hybrid optical structure and image restoration neural network, and complete the design of the end-to-end refractive-diffractive hybrid imaging system based on ray tracing.

[0068] A specific embodiment of the method of the present invention is given below to illustrate the technical effect of the present invention.

[0069] In this embodiment, 1000 real-shot infrared data sets of 640*512 size are used, and the clear pictures in the data set are recorded as P0. In addition, a virtual lens is established, and the optimization starting point is a flat plate. Along the direction from the incident light to the outgoing light, the first light-transmitting surface of the first flat plate is a binary diffraction surface, and the remaining surfaces are spherical surfaces. The number of pixels on the detector plane is 640×512, and the pixel size is 15 microns. A virtual image plane is set at an object distance of 8000 meters, and sampling is performed, with 64 light rays sampled for each pixel. A unified ray tracing model for a refractive-diffractive hybrid optical system is established to trace all incident light rays to the detector plane, thereby obtaining a simulated picture with aberrations. Figure 3As shown in the figure, an image restoration neural network is constructed and trained. Inside the full-field attention module FDA-Block, the convolution kernel size of 1×1 convolution is 1, the padding is 0, and the stride is 1; the convolution kernel size of 3×3 convolution is 3, the padding is 1, and the stride is 1; the convolution kernel size of full-dimensional dynamic convolution is 3, the padding is 1, and the stride is 1. Inside the full-field attention module FDA-Block, the size of the image remains unchanged. After each downsampling, the number of channels of the image becomes twice as much as before, and after each upsampling, the number of channels of the image is divided by 2. At the last 3×3 convolution, the number of channels of the image becomes 1, and the output image is obtained.

[0070] The simulated three-channel 640×512 image with aberration and field of view prior is input into the image restoration neural network, and the network restoration result is a single-channel 640×512 image. During training, the learning rate is set to 0.001, and the back-propagation Adam algorithm is used to achieve gradient descent, while optimizing the parameters of the refractive-diffractive hybrid optical system and the neural network parameters. Iterate the optimization 100 times to obtain the final optimization result.

[0071] like Figure 4 As shown in the figure, it is the result of this embodiment, where (a) is a clear picture without aberration of the input system, (b) is the direct imaging result through the optical system, and the structural similarity is 0.65, and (c) is the restoration result of the neural network. Figure 4 It can be seen that the structural similarity between the final imaging result and the original image is above 0.9, which proves that the present invention can realize the high-precision design of end-to-end refractive-diffractive hybrid optical system. For infrared images of size 640×512, the imaging system designed by this method can achieve a single-image calculation time of less than 30ms and can run 30 frames per second, reflecting its high real-time characteristics.

[0072] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention shall be included in the protection scope of the invention.

Claims

1. A design method for an end-to-end hybrid refractive imaging system based on ray tracing. It is characterized in that The following steps are involved: S1: Establish a virtual refractive-diffractive hybrid optical structure and an initial structure of the detector plane as a virtual optical system; And build a dataset consisting of real clear pictures; S2: Establish a ray tracing model for the refractive-diffractive hybrid optical system. For the refractive surface, use Snell's theorem to calculate the light deflection direction. For the diffractive surface, use the local grating equation to calculate the light deflection direction. S3: setting a clear image in the data set as a virtual object plane at a preset object distance, sampling the object plane light to obtain the initial light, and propagating it into the virtual optical system, using the ray tracing model established in S2, tracing the light from the incident point to the detector plane, and obtaining a simulated aberration-containing degraded image; S4: Establish an image restoration neural network, the input of which is a simulated degraded image with aberration, and the output is a picture with some aberrations removed; the image restoration neural network as a whole is a U-shaped structure, including a downsampling part and an upsampling part, and a full-field attention module connecting the downsampling part and the upsampling part; the downsampling part and the upsampling part each include a 3×3 convolution module and four full-field attention modules, and the corresponding full-field attention modules of the downsampling part and the upsampling part are jump-connected to promote the transmission of effective information; the full-field attention module includes two parts, the first part is five layers, which are LN layer, 1×1 convolution layer, 3×3 convolution layer, activation function ReLU and full-dimensional dynamic convolution in sequence; the second part is four layers, which are LN layer, 1×1 convolution layer, activation function ReLU and full-dimensional dynamic convolution layer in sequence; the input and output of the first part of the full-field attention module are added element-wise and used as the input of the second part; the input of the second part and the output of the second part are added element-wise and used as the output of the entire full-field attention module; The full-dimensional dynamic convolution layer adopts a parallel strategy inside any convolution kernel to learn the attention value inside the convolution kernel from four dimensions: position, input channel, filter, and convolution kernel, thereby obtaining the full-dimensional convolution kernel attention value; S5: inputting the simulated aberration-containing picture obtained in step S3 into the image restoration neural network, constructing a loss function, calculating the error between the picture output by the image restoration neural network with some aberrations removed and the clear picture actually taken, and optimizing the parameters of the refractive-diffractive hybrid optical structure and the neural network parameters simultaneously by using back-propagation gradient descent; S6: Using all the real clear pictures in the data set, repeat S3 to S6 until clear imaging is achieved, obtain the optimized refractive-diffractive hybrid optical structure and image restoration neural network, and complete the design of the end-to-end refractive-diffractive hybrid imaging system based on ray tracing.

2. According to the ray tracing-based end-to-end hybrid imaging system design method of claim 1, It is characterized in that The initial structure of the refractive-diffractive hybrid optical structure is two flat plates. Along the direction from incident to outgoing light, the first light-transmitting surface of the first flat plate is set as a diffractive surface, and the remaining light-transmitting surfaces of the two flat plates are set as even-refractive aspheric surfaces. The expressions for all refractive and diffractive surfaces are: Where c is the curvature, k is the cone coefficient, α 2x is the high-order coefficient of the aspheric surface, r is the radial distance, x is the aspheric order, and X is the maximum aspheric order; The additional phase introduced by the diffraction surface is: Among them, β y is the phase coefficient of the diffraction surface, R is the maximum radius of the diffraction element, y is the phase order, and Y is the maximum phase order.

3. The end-to-end hybrid imaging system design method based on ray tracing according to claim 1, It is characterized in that In S2, the direction vector d of the outgoing light from the refractive surface t The vector form of the expression is Among them, n 1 is the normal vector of the incident point of the refracting surface, d i is the direction vector of the incident light, n i is the refractive index of the material on the incident side, n t is the refractive index of the material on the output side; The vector form of the outgoing light from the diffraction surface is expressed as: Among them, n 2 is the normal vector of the incident point of the aspherical surface of the diffraction surface base, λ is the wavelength of the incident light, m ​​is the selected diffraction order, φ(r) is the additional phase introduced by the diffraction surface, and f is the direction vector of the local grating.

4. The method for designing an end-to-end hybrid imaging system based on ray tracing according to claim 1, It is characterized in that In S3, the Newton iteration method is used to calculate the intersection of the light and the virtual optical system to obtain the incident point; the light deflection angles are calculated for the refractive surface and the diffractive surface respectively, and the light is traced to the detector plane through the refractive surface and the diffractive surface; for the light on the detector plane, the intensity of its virtual image plane sampling point is weighted to obtain the simulated aberration-containing degraded image P 1 .

5. The method for designing an end-to-end hybrid imaging system based on ray tracing according to claim 1, It is characterized in that The expression of the loss function in S5 is: L=||P 2 (θ,n)-P 0 || 2 +εL opt The first term in the formula represents the gap between the output image of the image restoration neural network and the real image, P 2 represents the image output by the image restoration neural network, θ represents the parameters of the virtual optical system, n represents the parameters of the image restoration neural network; ε is a set proportional coefficient, L opt Represents the limitations of the virtual optical system itself.

Citation Information

Patent Citations

  • Method and device for designing end-to-end imaging equipment.

    CN113077540A

  • End-to-end design method for Fresnel single lens calculation imaging system

    CN114859550A

  • Single-lens large-depth-of-field calculation imaging system and method based on deep learning

    CN114897752A

  • Ultra-light refractive-diffractive hybrid optical imaging system and method combined with neural network

    CN117008324A

  • Hybrid refractive / diffractive achromatic camera lens and camera using such

    WO1995018393A1