Multimodal camera optical imaging system and performance evaluation method thereof

Abstract

The application discloses a multi-mode camera optical imaging system and a performance evaluation method thereof, belongs to the technical field of optical imaging, and solves the problem that a general lens cannot transmit a high-fidelity object plane light field to an internal image sensor of a multi-mode camera. The system comprises an object plane correction lens, an imaging objective lens assembly and an image plane correction lens arranged in sequence, and the optical centers of the object plane correction lens, the imaging objective lens assembly and the image plane correction lens are located on the same optical axis; the object plane correction lens and the image plane correction lens are both thin positive lenses, the focal length of the object plane correction lens is equal to the object distance, and the focal length of the image plane correction lens is equal to the image distance; and the object plane correction lens and the image plane correction lens are close to the planes where they are located.

Description

Technical Field

[0001] This invention relates to the field of optical imaging technology, specifically to a multimodal camera optical imaging system and its performance evaluation method. Background Technology

[0002] A multimodal camera is a digital camera that can detect and record the temporal, spatial, spectral, polarization, and energy information of an object's light field. Multimodal cameras generally use solid-state image sensors, with a light modulation layer in front of the pixel, and photoelectric conversion behind the modulation layer; each pixel is a miniature optical system.

[0003] During optical imaging, the plane elements of the objective aperture... In parallel operation, during general-purpose objective lens imaging, the contribution of one pixel of the objective aperture to each pixel is uneven. For example... Figure 1 As shown, where, For object distance, Image distance, from the objective aperture The light rays passing through the point Points of contribution and The contributions of individual points differ. Like a surface... Point distance The points are close together, and the light rays are incident on the pixels almost perpendicularly. However, Point distance The point is relatively far away, and the light rays are incident on the pixel at an angle. Similarly, the aperture element has different abilities to collect light waves from different points on the object surface. The above problems indicate that general-purpose lenses are not suitable for multi-modal cameras.

[0004] Therefore, a multimodal camera's optical imaging system must faithfully transmit the object's light field information to the image sensor to fully utilize its performance; general-purpose lenses may not be sufficient. Thus, developing a dedicated optical imaging system tailored to the characteristics of multimodal camera image sensors is essential. Summary of the Invention

[0005] This invention solves the problem that general-purpose lenses cannot transmit the object plane light field to the internal image sensor of a multimodal camera with high fidelity.

[0006] The multimodal camera optical imaging system of the present invention includes an object plane correction mirror, an imaging objective lens assembly and an image plane correction mirror arranged in sequence, wherein the optical centers of the object plane correction mirror, the imaging objective lens assembly and the image plane correction mirror are located on the same optical axis. Both the object plane correction mirror and the image plane correction mirror are thin positive lenses, and the focal length of the object plane correction mirror is equal to the object distance, while the focal length of the image plane correction mirror is equal to the image distance. The object plane correction mirror and the image plane correction mirror are close to their respective surfaces.

[0007] Furthermore, in one embodiment of the present invention, the imaging objective lens assembly includes an aperture stop.

[0008] The multimodal camera optical imaging performance evaluation system of the present invention is implemented based on the multimodal camera optical imaging system described above. The imaging objective lens assembly includes a front aperture lens, an aperture stop, and a rear aperture lens arranged in sequence. Both the front and rear lenses of the aperture stop are positive lenses, and the focal length of the front lens is equal to the object distance, while the focal length of the rear lens is equal to the image distance.

[0009] The multimodal camera optical imaging system performance evaluation method of the present invention is used to evaluate the actual image and geometric optical ideal image function of the multimodal camera optical imaging performance evaluation system described above from the frequency domain and the spatial domain, respectively.

[0010] Furthermore, in one embodiment of the present invention, the frequency domain is based on the coherent transfer function of a multimodal camera optical imaging system. What is reflected; ; Among them, the coordinates of the light wave mode nodes With the mode in the image space frequency The relations are respectively , , the aperture function Substituting the independent variable yields , Image distance, The wavelength of light Let be the coordinate system of the aperture plane.

[0011] Furthermore, in one embodiment of the present invention, the spatial domain is based on the point spread function of the multimodal camera optical imaging system. What is reflected; ; in, To Perform the inverse Fourier transform. For the aperture function, in the image plane coordinate system With the object plane coordinate system opposite directions and These represent the node coordinates of the optical wave mode and the spatial frequency of the mode in the image plane, respectively. Let be the coordinate system of the aperture plane.

[0012] Furthermore, in one embodiment of the present invention, the geometrical optical ideal image function is specifically: ; in, For surface function, For geometric optics ideal image function, For image plane total function, image plane coordinate system With the object plane coordinate system opposite directions To Perform the inverse Fourier transform. Let f be the aperture function, and let g be the coordinates of the light wave mode node. With the mode in the image space frequency The relations are respectively , , For imaging magnification, Let be the point spread function. Let be the coordinate system of the aperture plane.

[0013] Furthermore, in one embodiment of the present invention, the actual image specifically refers to: ; in, It is a matter function. For geometric optics ideal image function, For the total function of the image plane, For image plane coordinate system, For imaging magnification, For coherent transfer function, the node coordinates of the optical wave mode. With the mode in the image space frequency The relations are respectively , , the aperture function Substituting the independent variable yields , Image distance, The wavelength of light Let the aperture plane coordinate system be used. This is a Fourier transform.

[0014] This invention solves the problem that general-purpose lenses cannot transmit the object plane light field to the internal image sensor of a multimodal camera with high fidelity. Specific beneficial effects include: 1. The multimodal camera optical imaging system of the present invention can transmit the object plane light field to the image sensor with high fidelity through the object plane correction mirror, the imaging objective lens assembly and the image plane correction mirror. 2. The performance evaluation method for a multimodal camera optical imaging system described in this invention, by re-disassembling the multimodal camera optical imaging system, can rigorously calculate the performance of the optical imaging system based on its structural parameters. 3. The performance evaluation method for the multimodal camera optical imaging system described in this invention can accurately calculate the image function based on the object surface light function. Attached Figure Description

[0015] The above and / or additional methods and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a diagram illustrating the uneven contribution of the universal objective aperture element to each pixel, as described in the background technology. Figure 2 This is a schematic diagram of the principle of evaluating the quality of the optical imaging system as described in Implementation Method 3; Figure 3 The multimodal camera optical imaging system described in Embodiment 1 comprises three unit diagrams; Figure 4 As described in Embodiment 1, the plane wave component on the object plane still forms a plane wave pattern when it propagates to the image plane. Figure 5 This is a design feature diagram of the optical imaging system described in Embodiment 2; Figure 6 The imaging process described in Implementation Method 3 is decomposed into Fourier transform and inverse Fourier transform diagrams; Figure 7 This is a diagram illustrating how the correction mirror in Implementation Method 1 alters the angle at which light enters the image sensor. Figure 8 It is the object surface point light source imaging diagram described in Embodiment 3; Figure 9 This is a light wave pattern diagram of the imaging system described in Embodiment 3; Figure 10 This is the aperture function and optical wave mode transmission diagram described in Implementation Method 3; Figure 11 This is a flowchart of image calculation based on coherent transfer function as described in Implementation Method 3; Figure 12 This is a diagram of the imaging process described in Implementation Method 4. Detailed Implementation

[0016] Various embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. The embodiments described with reference to the drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0017] Implementation Method 1: To address the issue of inconsistent contributions from aperture elements to individual pixels in existing multimodal camera optical imaging systems, this implementation method proposes a multimodal camera optical imaging system, such as... Figure 3 As shown, the system adds correction lenses to both the object plane and the image plane, specifically including an object plane correction lens. Thin positive lens, close to the object plane, focal length equal to object distance; imaging objective lens assembly. It includes an objective lens and an aperture stop; an image plane correction mirror. A thin positive lens, close to the image plane, with a focal length equal to the image distance.

[0018] Based on the multimodal camera optical imaging system described in this embodiment, with plane waves as the basic unit of the object plane light field, the plane wave remains a plane wave on the image plane after passing through the multimodal camera optical imaging system, such as... Figure 4 As shown, the objective lens The equivalent prism near the point indicates that the multimodal camera optical imaging system has the ability to transmit the state of the object's light field to the image sensor with high fidelity.

[0019] To better illustrate the multimodal camera optical imaging system described in this embodiment, the following examples will be used for detailed explanation: The image sensor of a multimodal camera is significantly affected by the angle of incident light. The optical imaging system of this multimodal camera controls the angle of light, such as... Figure 7 As shown, off-axis pixels and on-axis pixels receive the same light, which is especially beneficial for edge pixels of image sensors. For example, in an optical imaging system with an aperture diameter of... Image distance is Whether it's pixels at the edge of the image plane or pixels at the center, ensure that each pixel receives light at the correct angle. This achieves accuracy and uniformity in information detection.

[0020] Implementation Method Two: To address the issue of numerous approximation steps in optical image calculation, this implementation method re-decomposes the multimodal camera optical imaging system described in Implementation Method One, as follows: Figure 5 As shown, an equivalent imaging objective lens assembly using two positive lenses and one aperture stop is employed. Front lens of aperture The focal length is equal to the object distance, and the lens behind the aperture is... The focal length is equal to the image distance, and is used for performance evaluation of multimodal camera optical imaging systems.

[0021] Implementation Method 3: Current principles for evaluating the quality of optical imaging systems are as follows Figure 2 As shown, the object surface is represented by points or stripes, and the light rays are grouped according to the object surface coordinates. The model has many approximation links, and the evaluation focuses on image sharpness and distortion. However, it cannot be used to evaluate the multimodal camera optical imaging performance evaluation system described in Implementation Method 2.

[0022] Therefore, this embodiment proposes a performance evaluation method for multimodal camera optical imaging systems, which is used to rigorously calculate the performance of the optical imaging system based on its structural parameters. Specifically: Implementation method two involves re-disassembling the multimodal camera optical imaging system, breaking down the optical imaging process into two steps, such as... Figure 6 The figures shown are the Fourier transform and the inverse Fourier transform, respectively.

[0023] The method described in this embodiment can accurately derive the point spread function and coherence transfer function of a multimodal camera optical imaging system. The point spread function evaluates the performance of the multimodal camera optical imaging system in the spatial domain, while the coherence transfer function evaluates the performance of the multimodal camera optical imaging system in the frequency domain. The following examples illustrate this in detail: In this embodiment, as Figure 8 As shown, the object surface The point is a monochromatic point light source with a wavelength of . The amplitude of the object surface can be obtained. for: ; Aperture plane amplitude for: ; Image plane amplitude for:

[0024]

[0025] ; in, To evaluate the point spread function of a multimodal camera optical imaging system, For the aperture function, To Perform inverse Fourier transform to obtain the node coordinates of the light wave mode. The frequency relationships between the modes in the image plane space are as follows: , , Image distance, for function, For Fourier transform, For image magnification, image plane coordinate system With the object plane coordinate system opposite directions It is an impulse response.

[0026] Therefore, the point spread function of a multimodal camera optical imaging system The spatial domain reflects the system's performance, and the process of calculating the point spread function is transformed into Fourier transform and inverse Fourier transform, making the calculation process simple and the results accurate.

[0027] In this embodiment, the aperture is decomposed into surface elements, such as... Figure 9As shown. The surface element scale is the wavelength of light divided by the field of view; the larger the field of view, the smaller the surface element. The spatial distribution of light waves passing through the same surface element constitutes a light wave pattern (or simply light wave pattern) of the imaging system. A light wave pattern includes several states: object plane wave, object-side converging spherical wave, image-side diverging spherical wave (concentric with the converging spherical wave), and image plane wave.

[0028] The surface element of the aperture is a node of the light wave mode. If the coordinates of the light wave mode node are... This model is called , Pattern, for Figure 9 The multimodal camera optical imaging system shown is , The mode has a frequency of in the image plane space. , , , The spatial frequency of the mode on the object surface is , ,and , .

[0029] like Figure 10 As shown in Table 1, the effects of each mode on the lens imaging process are summarized.

[0030] Table 1. Effects of Objective Aperture on Each Mode

[0031] If the objective lens has aberrations, these aberrations affect the phase and amplitude of each mode on the image plane, thus reducing image quality. By calculating the amplitude and phase of each mode involved in imaging on the image plane and then superimposing the amplitudes of each mode on the image plane, the total image plane function can be obtained.

[0032] The phase of the transmittance function of a mode node determines the phase delay of that mode during imaging. Therefore, the aperture function reflects the transmission of each mode by the optical imaging system. This is based on the relationship between the coordinates of the optical mode node and the spatial frequency of the mode in the image plane. , , the aperture function By substituting the independent variable, we can obtain .based on Redefine a function ,make ,use Describes the ability of an optical imaging system to transmit light waves of various frequencies across the image plane. This refers to the coherence transfer function of an optical imaging system. The coherence transfer function is not only used to evaluate multimodal camera optical imaging systems, but also to calculate image amplitude. The calculation process is as follows: Figure 11 As shown. To highlight the optical process, in Figure 11 The process did not use Instead, it utilizes ,in , Let be the coordinate system of the aperture plane.

[0033] Therefore, the coherence transfer function of a multimodal camera optical imaging system The frequency domain reflects the system's performance. The method described in this embodiment can directly obtain the coherent transfer function of the multimodal camera optical imaging system.

[0034] Implementation Method Four: Based on the multimodal camera optical imaging performance evaluation system described in Implementation Method Two, this implementation method can accurately calculate the optical image. The imaging process involves multiple planes, such as... Figure 12 As shown.

[0035] Let the surface function be , It is the spatial angular frequency. Using the imaginary unit, we obtain the amplitudes of each plane: Surface amplitude: ; Surface amplitude: ; Amplitude in front of: ; Amplitude behind the surface: ; Surface amplitude: ; Surface amplitude: ; Surface amplitude: ; : ; in, It is a geometrically ideal image; The point spread function; Convolution calculation The equation can be obtained by performing a Fourier transform. ; in, for Transmittance function, for Transmittance function, for Transmittance function, for Transmittance function, For the total function of the image plane, It is a matter function. It is the ideal image function of geometric optics.

[0036] The multimodal camera optical imaging system and its performance evaluation method proposed in this invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A multimodal camera optical imaging performance evaluation system, wherein the system is based on a multimodal camera optical imaging system, characterized in that, The multimodal camera optical imaging system includes an object plane correction mirror, an imaging objective lens assembly, and an image plane correction mirror arranged in sequence, wherein the optical centers of the object plane correction mirror, the imaging objective lens assembly, and the image plane correction mirror are located on the same optical axis. Both the object plane correction mirror and the image plane correction mirror are thin positive lenses, and the focal length of the object plane correction mirror is equal to the object distance, while the focal length of the image plane correction mirror is equal to the image distance. The object plane correction mirror and the image plane correction mirror are close to their respective surfaces; The imaging objective lens assembly includes a front aperture lens, an aperture stop, and a rear aperture lens arranged in sequence. Both the front and rear lenses of the aperture stop are positive lenses, and the focal length of the front lens is equal to the object distance, while the focal length of the rear lens is equal to the image distance.

2. The multimodal camera optical imaging performance evaluation system according to claim 1, characterized in that, This is used to evaluate the actual image and geometrically ideal image functions of the multimodal camera optical imaging performance evaluation system from the frequency domain and spatial domain, respectively.

3. The multimodal camera optical imaging performance evaluation system according to claim 2, characterized in that, The frequency domain is based on the coherent transfer function of a multimodal camera optical imaging system. What is reflected; ； Among them, the coordinates of the light wave mode nodes With the mode in the image space frequency The relations are respectively , , the aperture function Substituting the independent variable yields , Image distance, The wavelength of light Let be the coordinate system of the aperture plane.

4. The multimodal camera optical imaging performance evaluation system according to claim 2, characterized in that, The aforementioned spatial domain is based on the point spread function of a multimodal camera optical imaging system. What is reflected; ； in, To Perform the inverse Fourier transform. For the aperture function, in the image plane coordinate system With the object plane coordinate system opposite directions and These represent the node coordinates of the optical wave mode and the spatial frequency of the mode in the image plane, respectively. Let be the coordinate system of the aperture plane.

5. The multimodal camera optical imaging performance evaluation system according to claim 2, characterized in that, The aforementioned ideal image function of geometric optics is specifically as follows: ； in, For surface function, For geometric optics ideal image function, For image plane total function, image plane coordinate system With the object plane coordinate system opposite directions To Perform the inverse Fourier transform. Let f be the aperture function, and let g be the coordinates of the light wave mode node. With the mode in the image space frequency The relations are respectively , , For imaging magnification, Let be the point spread function. Let be the coordinate system of the aperture plane.

6. The multimodal camera optical imaging performance evaluation system according to claim 2, characterized in that, The actual image mentioned above is specifically: ； in, It is a matter function. For geometric optics ideal image function, For the total function of the image plane, For image plane coordinate system, For imaging magnification, For coherent transfer function, the node coordinates of the optical wave mode. With the mode in the image space frequency The relations are respectively , , the aperture function Substituting the independent variable yields , Image distance, The wavelength of light Let the aperture plane coordinate system be used. This is a Fourier transform.

Images