Imaging simulation method, device and electronic equipment based on point spread function

Abstract

This application belongs to the field of image processing and specifically discloses an imaging simulation method, device, and electronic device based on a point spread function. The method includes: establishing a light source model corresponding to an image captured by an optical system, obtaining the point spread function of the point light source of the light source model after being acted upon by the optical system; obtaining error point spread functions under different optical system models; performing weighted averaging of each error point spread function according to preset weights to obtain a comprehensive point spread function; forming an array diagram of the comprehensive point spread function according to light distribution, convolving the array diagram with the captured image to obtain a degraded image that simulates the blurring effect caused by aberrations; applying a distortion model of the optical system to the captured image to obtain a deformation effect image that simulates the distortion, and implementing imaging simulation of the optical system based on the deformation effect image and the degraded image. This application can improve imaging accuracy and efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to an imaging simulation method, device, and electronic device based on a point spread function. Background Art

[0002] Image quality is a crucial metric in various imaging systems, such as optical microscopes, astronomical telescopes, digital cameras, and medical imaging equipment. Image quality directly impacts the accuracy and effectiveness of observation, analysis, and diagnosis performed using these systems. Traditional methods for evaluating image quality include modulation transfer function (MTF) and signal-to-noise ratio (SNR). However, these methods often require complex experimental equipment and measurement methods, and in some cases, they cannot accurately reflect the overall performance of the imaging system.

[0003] At present, some research results have proposed methods that can already evaluate imaging quality, but most of these methods focus on specific types of imaging systems, such as optical microscopes or astronomical telescopes, and mostly focus on theoretical analysis, lacking specific simulation steps and implementation methods. In addition, in complex optical imaging systems, traditional experimental methods require complex experimental equipment and measurement methods, and existing simulation platforms cannot fully and accurately reflect the performance of the imaging system. Therefore, the accuracy of the simulation results and the simulation efficiency of current imaging simulation methods still have certain defects.

[0004] Therefore, how to improve the accuracy and efficiency of imaging quality is an urgent problem to be solved. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of this application is to provide an imaging simulation method, device and electronic equipment based on a point spread function, aiming to solve the current problems of poor accuracy and efficiency of imaging simulation.

[0006] In a first aspect, the present application provides an imaging simulation method based on a point spread function, comprising:

[0007] Establishing a light source model corresponding to the image captured by the optical system, and obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system;

[0008] updating the geometric model of the optical system according to the lens material and processing errors of the optical system to obtain optical system models with different errors, and acquiring error point spread functions under different optical system models;

[0009] Perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function;

[0010] forming an array graph of the integrated point spread function according to light distribution, and convolving the array graph with the captured image to obtain a degraded image that simulates a blurring effect caused by aberration;

[0011] The distortion model of the optical system is applied to the captured image to obtain a deformation effect image simulating the distortion, and the imaging simulation of the optical system is realized based on the deformation effect image and the degraded image.

[0012] By comprehensively considering factors such as the optical system's materials, processing errors, aberrations, and distortion, the present embodiment can more accurately simulate the imaging effects of an actual optical system, thereby improving the accuracy and efficiency of imaging simulation. Furthermore, by weighted averaging the point spread function under multiple errors, it can reflect the combined impact of different errors, avoiding the potential bias caused by considering only a single error. By applying convolution operations and distortion models, it is possible to generate images that more closely resemble the imaging effects of a real optical system.

[0013] Optionally, updating the geometric model includes:

[0014] Updating the geometric model according to the curvature radius, thickness, refractive index distribution of each lens of the optical system, and microscopic irregularities and positional deviations of the lens surface;

[0015] The parameters of each lens are randomly perturbed within the tolerance range to obtain optical system models with different errors.

[0016] Optionally, obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system includes:

[0017] Dividing the captured image into blocks to obtain a plurality of blocks;

[0018] The center point of each field of view block is used as the sampling point of the current field of view range;

[0019] Performing ray tracing calculation on the optical system using the sampling points to obtain a diffuse spot after the point light source passes through the optical system;

[0020] The point spread function is determined based on the diffuse speckle.

[0021] Optionally, performing ray tracing calculation on the optical system using sampling points to obtain a diffuse spot of the point light source after passing through the optical system includes:

[0022] Determine the light path of the sampling point light through the optical system and determine the coordinates of the intersection point formed on the imaging plane;

[0023] Perform pixel attribution calculation on the intersection coordinates to determine the image plane discrete pixels of each intersection coordinate;

[0024] The number of received intersections of the discrete pixels on the image plane is counted, the number of intersections is mapped into image pixel values, the image pixel values ​​are normalized, and the diffuse spots are obtained according to the processed pixel image.

[0025] Optionally, performing weighted averaging on each error point spread function according to a preset weight to obtain a comprehensive point spread function includes:

[0026] Assigning different weights to each of the error point spread functions according to a preset weighting factor, and weighting different error point spread functions of the same sampling point to obtain a comprehensive spread function after comprehensively considering the processing error;

[0027] The weighting factor is determined based on the frequency or severity of the error.

[0028] Optionally, convolving the array image with the captured image to obtain a degraded image that simulates a blurring effect caused by aberration includes:

[0029] Convolve each integrated point spread function with the original image to obtain a set of degraded images under a single field of view;

[0030] The degraded image set is fused using an interpolation method to obtain degraded images under different fields of view of the optical system.

[0031] Optionally, applying the distortion model of the optical system to the captured image to obtain a deformation effect image with simulated distortion includes:

[0032] A distortion model of the optical system is established, pixel values ​​of the captured image are obtained using an interpolation method, and a deformation effect caused by the distortion is simulated according to the pixel values ​​to obtain a deformation effect image.

[0033] In a second aspect, the present application further provides an imaging simulation device based on a point spread function, comprising:

[0034] A function determination module is used to establish a light source model corresponding to the image captured by the optical system, and obtain a point spread function of a point light source of the light source model after passing through the optical system;

[0035] A model updating module is used to update the geometric model of the optical system according to the lens material and processing error of the optical system, obtain optical system models with different errors, and obtain error point spread functions under different optical system models;

[0036] The weighted average module is used to perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function;

[0037] a convolution module, configured to form an array diagram from the integrated point spread function according to light distribution, and convolve the array diagram with the captured image to obtain a degraded image that simulates a blurring effect caused by aberration;

[0038] The distortion simulation module is used to apply the distortion model of the optical system to the captured image to obtain a deformation effect image of simulated distortion, and realize imaging simulation of the optical system based on the deformation effect image and the degraded image.

[0039] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0041] In a fifth aspect, the present application provides a computer program product, which, when executed on a processor, enables the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0042] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0043] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the existing technologies:

[0044] (1) Through in-depth analysis of the light source model and point spread function, this application can more accurately evaluate the imaging performance of the optical system under different conditions, and can improve the accuracy and efficiency of imaging simulation by clarifying the impact of various factors (such as lens design, material properties, processing errors, etc.) on the final image quality. In addition, by weighting the updated model to obtain a comprehensive point spread function, it can more accurately simulate the aberrations in the optical system and further improve the accuracy of imaging quality prediction.

[0045] (2) By establishing a comprehensive point spread function and performing convolution operations, this application can generate more realistic degraded images, predict image blur and distortion that may occur under actual shooting conditions, comprehensively analyze and improve image quality, and enhance the reliability of image processing and analysis.

[0046] (3) By establishing a distortion model for the optical system and applying interpolation to calculate pixel values, this application can effectively correct image distortion, improve image accuracy and clarity, and make it closer to the actual scene. After the image distortion is corrected, subsequent image analysis and processing can be performed more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 1 is a flow chart of an imaging simulation method based on a point spread function provided in an embodiment of the present application;

[0048] Figure 2 Schematic diagram of the structure of the double Gauss lens provided in an embodiment of the present application;

[0049] Figure 3 This is a point spread function simulation result diagram provided by an embodiment of the present application;

[0050] Figure 4 Schematic diagram of the structure of an imaging simulation device based on a point spread function provided in an embodiment of the present application;

[0051] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0053] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0054] The terms "first" and "second" in this specification and claims are used to distinguish different objects rather than to describe a specific order of objects. For example, "first response message" and "second response message" are used to distinguish different response messages rather than to describe a specific order of response messages.

[0055] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0056] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more, for example, multiple processing units means two or more processing units, etc.; multiple elements means two or more elements, etc.

[0057] Next, the technical solutions provided in the embodiments of this application are introduced.

[0058] Reference Figure 1 , the present application provides an imaging simulation method based on a point spread function, comprising:

[0059] S101. Establishing a light source model corresponding to the image captured by the optical system, and obtaining a point spread function of the point light source of the light source model after passing through the optical system;

[0060] S102. Update the geometric model of the optical system according to the lens material and processing error of the optical system to obtain optical system models with different errors, and obtain the error point spread function under different optical system models;

[0061] S103. Perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function;

[0062] S104. The integrated point spread function forms an array diagram according to the light distribution, and the array diagram is convolved with the captured image to obtain a degraded image with a blurred effect caused by simulated aberration;

[0063] S105. Apply the distortion model of the optical system to the captured image to obtain a deformation effect image that simulates the distortion, and implement imaging simulation of the optical system based on the deformation effect image and the degraded image.

[0064] First, a light source model is established and a point spread function (PSF) is obtained through the above S101.

[0065] A light source model is created for the scene captured by the optical system. This model is usually based on a collection of object points or point light sources to simulate how the optical system processes light emitted from different positions.

[0066] Specifically, according to the size and resolution requirements of the actual shooting scene, as well as the wavelength distribution, light source models of three colors, red R, green G, and blue B, are established for the point spread function.

[0067] In general, the captured scene is regarded as an image, and the pixel unit of the image is the basic light-emitting unit. For a color image, it contains three light-emitting units: R, G, and B. It is necessary to define the light source model of the corresponding color according to their respective wavelength distributions. When the working distance is long, each pixel can be regarded as a point light source, that is, when defining the starting information of the light, the starting position of all light rays is consistent, and the direction of the light follows a specific angular distribution, such as the Lambertian distribution. The wavelength of the light is defined by the spectral distribution of the shooting scene, thereby completing the establishment of the corresponding light source model. When the area of ​​the pixel cannot be ignored, a small number of pixel units can be selected through experimental means to measure their angular distribution, and the area of ​​the light source can be defined according to the size of the pixel unit to establish a light source model that is closer to the actual scene.

[0068] Furthermore, the point spread function (PSF) is obtained. The point spread function is the light intensity distribution formed on the image plane when an optical system images a single point light source. Through ray tracing or optical simulation, the point spread function (PSF) of the point light source in the light source model after passing through the optical system is calculated.

[0069] The geometric model of the optical system is updated in step S102. Considering the lens material and processing errors, the initial optical system geometric model is adjusted according to the material properties (such as refractive index) and processing errors (such as deviations in lens curvature and thickness) of the actual optical system to generate an optical system model with different errors.

[0070] Then, for each updated optical system model, the corresponding point spread function is recalculated to simulate the imaging quality changes caused by manufacturing errors or material inhomogeneities.

[0071] The weighted average error point spread function is performed in step S103 : all calculated error point spread functions are weighted averaged according to preset weights to obtain a comprehensive point spread function.

[0072] Furthermore, the blur effect of aberration is simulated in S104 , where the integrated point spread function is formed into an array diagram according to the light distribution, and the array diagram is convolved with the captured image to simulate the blur effect caused by the aberration of the optical system and generate a degraded image.

[0073] It should be noted that the convolution process in this embodiment is essentially to apply the PSF to the entire image to simulate the blur during imaging by the optical system.

[0074] Finally, the captured image is deformed according to the distortion characteristics of the optical system to generate a distorted image that simulates the distortion. Distortion usually manifests as stretching, compression, or bending of the image edges.

[0075] By combining the deformation effect image and the degraded image, the imaging simulation of the optical system is completed, simulating the image effects that may be produced by a real optical system.

[0076] By comprehensively considering factors such as the optical system's materials, processing errors, aberrations, and distortion, the present embodiment can more accurately simulate the imaging effects of an actual optical system, thereby improving the accuracy and efficiency of imaging simulation. Furthermore, by weighted averaging the point spread function under multiple errors, it can reflect the combined impact of different errors, avoiding the potential bias caused by considering only a single error. By applying convolution operations and distortion models, it is possible to generate images that more closely resemble the imaging effects of a real optical system.

[0077] Optionally, updating the geometric model includes:

[0078] Updating the geometric model according to the curvature radius, thickness, refractive index distribution of each lens of the optical system, and microscopic irregularities and positional deviations of the lens surface;

[0079] The parameters of each lens are randomly perturbed within the tolerance range to obtain optical system models with different errors.

[0080] Optionally, obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system includes:

[0081] Dividing the captured image into blocks to obtain a plurality of blocks;

[0082] The center point of each field of view block is used as the sampling point of the current field of view range;

[0083] Performing ray tracing calculation on the optical system using the sampling points to obtain a diffuse spot after the point light source passes through the optical system;

[0084] The point spread function is determined based on the diffuse speckle.

[0085] Specifically, in this embodiment, the field of view of the captured scene image is divided into blocks, and the center of each field of view block is selected as the sampling point of the current field of view range. Ray tracing calculation is performed on the optical system to obtain the diffuse spot after the point light source passes through the optical system, that is, the PSF;

[0086] For optical imaging systems, when a point light source passes through the optical system, it no longer focuses on a single point on the image plane due to various aberrations and diffraction effects, but instead appears as a diffuse spot with a certain area distribution. The PSF provides the intensity distribution of a single point light source on the imaging plane through the optical system. It describes the distribution of the system's spatial response when imaging a point source or small object. It includes all possible aberrations and defocus information in the system and is crucial for understanding and improving image quality.

[0087] This application provides a PSF calculation method based on ray tracing. This method is based on the traditional geometric ray tracing algorithm. By tracing a large number of light rays through the actual optical system and recording their intersection points on the imaging plane, the PSF of the entire system is constructed. The specific process is as follows:

[0088] Determine the light path of the sampling point light through the optical system and determine the coordinates of the intersection point formed on the imaging plane;

[0089] Perform pixel attribution calculation on the intersection coordinates to determine the image plane discrete pixels of each intersection coordinate;

[0090] The number of received intersections of the discrete pixels on the image plane is counted, the number of intersections is mapped into image pixel values, the image pixel values ​​are normalized, and the diffuse spots are obtained according to the processed pixel image.

[0091] Specifically, the light emitted from the object point (sampling point) is refracted and reflected by the optical system to form a unique intersection point on the imaging plane. The coordinates of the intersection point can be expressed as (x i ,y i ), where i corresponds to the number of the ray with different azimuth angles starting from the object point. Since the image plane is composed of discrete pixels, the pixel attribution calculation is required for the image plane ray coordinates obtained by tracing.

[0092] After calculating all ray paths originating from the object point, we quantitatively count the distribution of their intersection points on the discrete pixels of the image plane. By calculating and accumulating the number of intersection points received by each pixel, we obtain the number of rays per pixel and map these values ​​to corresponding pixel values. For intuitive comparison, proper normalization is required to ensure that the sum of the values ​​in the entire pixel matrix is ​​1.

[0093] This process can be formalized using the following formula:

[0094]

[0095] In the above formula, psf p,q represents the normalized pixel value at pixel (p,q) in a speckle distribution map composed of discrete pixels. a and b represent the two-dimensional coordinates of the pixel center. i is the object-space sampling ray number, m is the number of object-space sampling rays, and n is the number of sampling rays detected on the image plane. The round function rounds to the nearest integer.

[0096] Optionally, performing weighted averaging on each error point spread function according to a preset weight to obtain a comprehensive point spread function includes:

[0097] Assigning different weights to each of the error point spread functions according to a preset weighting factor, and weighting different error point spread functions of the same sampling point to obtain a comprehensive spread function after comprehensively considering the processing error;

[0098] The weighting factor is determined based on the frequency or severity of the error.

[0099] Specifically, this embodiment modifies the optical system model based on the lens material and processing tolerances, repeatedly calculates different PSFs at the same sampling point, and weightedly adds the different PSFs according to user-defined weights to simulate the blurring effect caused by aberrations, thereby analyzing the imaging differences caused by processing errors.

[0100] Based on the material properties and manufacturing tolerances of the camera lens, the geometric model of the optical system is adjusted. These adjustments may include parameters such as the radius of curvature, thickness, and refractive index distribution of each lens in the system. Furthermore, non-ideal factors that may arise during the manufacturing process must be considered, such as microscopic irregularities on the lens surface and positional deviations. To more accurately simulate these errors, the key parameters of each lens are randomly perturbed within their tolerance range, generating a series of lens models with different errors. For each adjusted optical system model, the PSF at the same sampling point is recalculated using a ray tracing algorithm. The specific steps are described in the PSF acquisition process above.

[0101] To simulate the blurring effect caused by aberrations in actual imaging, a weighted average of the PSFs obtained from multiple calculations is performed. The specific steps are as follows: Each PSF is assigned a different weight based on a pre-set weighting factor (these factors can be set based on the frequency or severity of the error). The weighted sum of the different PSFs at the same sampling point is then performed to obtain a PSF that comprehensively accounts for processing errors.

[0102] Optionally, convolving the array image with the captured image to obtain a degraded image that simulates a blurring effect caused by aberration includes:

[0103] Convolve each integrated point spread function with the original image to obtain a set of degraded images under a single field of view;

[0104] The degraded image set is fused using an interpolation method to obtain degraded images under different fields of view of the optical system.

[0105] Specifically, in this embodiment, the PSFs of all sampled view points after weighted processing are formed into a PSF array diagram according to the main ray distribution, and the PSF array is convolved with the original image to simulate the blur effect caused by the aberration.

[0106] The PSF describes the dispersion of a point light source through an optical system and is the optical system's response to the imaging of that point light source. After obtaining the PSF of an optical system for a specific field of view using traditional ray tracing methods, convolving it with the scene image yields a degraded image affected by the aberrations in that field of view. Applying distortion processing to the degraded image allows the deformation effects caused by the aberrations to be incorporated into the final imaging simulation.

[0107] For actual optical systems, their PSF is related to the field of view. An optical system may obtain a very small diffuse spot at its central field of view, while at the edge of the field of view, due to the cumulative effect of various aberrations, the diffuse spot may be larger, resulting in a decrease in image quality. Strictly speaking, in order to obtain accurate image degradation results, it is necessary to calculate the corresponding PSF for each pixel position of the original image on the object side, but this method will lead to a decrease in computational efficiency. Since the PSF distribution of adjacent pixels is similar, sampling and interpolation methods can be used to reduce the amount of calculation. Define the physical size of the original image and uniformly sample X*Y grid points on the original image, that is, divide the entire field of view into X*Y field of view blocks, and each grid point corresponds to the center of a field of view block. The PSF of the corresponding grid point is obtained by the method described in the previous section, and the original image is convolved with the PSF of all grid points to obtain a set of degraded images under a single field of view. The image set is fused by interpolation to obtain the overall degraded image under different fields of view of the optical system. Here, a two-dimensional SINC function is used for interpolation to consider the degraded image I affected by aberrations. A The calculation can be expressed mathematically as:

[0108]

[0109] Where S a,b The PSF is the value of the two-dimensional SINC function with the sampling point indexed (a, b) as the center and the width of the corresponding field of view block as the unit length. a,b is the PSF corresponding to the sampling point indexed (a, b), I s is the original image.

[0110] Optionally, applying the distortion model of the optical system to the captured image to obtain a deformation effect image with simulated distortion includes:

[0111] A distortion model of the optical system is established, pixel values ​​of the captured image are obtained using an interpolation method, and a deformation effect caused by the distortion is simulated according to the pixel values ​​to obtain a deformation effect image.

[0112] Establish a distortion model for the imaging optical system and use interpolation methods to calculate the pixel values ​​of the target image to simulate the deformation effect caused by distortion;

[0113] Specifically, the degraded image obtained by convolving the original image with the PSF only reflects the image blur caused by the aberrations of the optical system. In the actual imaging process, in addition to image blur caused by aberrations, image distortion caused by distortion is also a common factor affecting image quality. Therefore, to more comprehensively simulate the imaging performance of the optical system, this embodiment requires obtaining a distortion model of the optical system and applying this model to an image that has not yet considered the effects of distortion. This step typically involves an image remapping process, namely, adjusting the position of each pixel in the image according to the distortion model to simulate the image deformation effect caused by distortion.

[0114] A grid of points is set in object space, the same number as the image pixel arrangement. After the optical system's distortion, the distribution of the grid points changes. A coordinate mapping relationship between the two is established, i.e., a distortion model is built. This model is applied to the original image to produce a simulated distortion image. Because most of the distorted grid points will not fall exactly on pixels in the original image, interpolation calculations are required when assigning values ​​to the pixels in the distorted image. In this embodiment, the nearest neighbor interpolation method is used to assign values ​​to non-integer pixels on the image plane.

[0115] The present application is described in detail below with reference to specific embodiments. This embodiment takes a double Gaussian lens in the Zemax lens database as an example and is described in detail as follows:

[0116] like Figure 2 As shown, the double Gauss lens of the implementation example of this application includes 2 double convex lenses ①⑥, 2 plano-convex lenses ②⑤ and 2 plano-concave lenses ③④. The lens data and specific structural distances in the example are shown in Table 1. The simulation uses a scene image with a resolution of 640×480, the image physical size is 200mm×150mm, and the field of view height of the center of the scene image is (0,0). The distance between the image and the system is set to 300mm, the image plane is 94.071mm away from the rear surface of the last lens of the system, the system wavelength is set to three visible light wavelengths of R (656.3nm), G (587.6nm), and B (486.1nm), and the G light is set to the main wavelength. The entrance pupil diameter is 33.33mm, the distance between the scene image and lens 1 is 300mm, and the aperture stop of the system is set 14.253mm behind lens 3. The simulation results are compared and analyzed with the simulation results of Zemax to verify the accuracy of the imaging quality simulation calculation results.

[0117] Table 1

[0118]

[0119] The first step is to establish a light source model for the point spread function based on the actual scene size and resolution requirements. The simulation uses a scene image with a resolution of 640×480, a physical image size of 200mm×150mm, and a working distance of 300mm. A corresponding light source model is established using pixel units as the light source. Here, a point light source model is established, and the angular distribution of light is set to fill the entrance pupil. The entrance pupil of the lens shown in Table 1 is located 58.93976mm behind the front surface of lens ①. The starting position of the light is represented by (x0, y0, z0), and the starting direction of the light is represented by (k, l, m).

[0120] In the second step, the field of view of the captured scene image is divided into 5×5 blocks. The center of each field of view block is selected as the sampling point of the current field of view range, that is, the starting position (x0, y0, z0) of the light in the light source model in the previous step. The number of sampling rays is selected as 128×128, and ray tracing calculations are performed on the optical system.

[0121] Light propagates along a straight line in an isotropic homogeneous medium. Any point (x, y, z) on the light can be expressed by the known parameters as:

[0122] x=x0+rk,y=y0+rl,z=z0+rm

[0123] When the light reaches the lens, the Newton iteration method is used to solve the intersection direction of the light and the lens surface, and the law of refraction and reflection is used to solve the propagation direction of the light after passing through the lens surface. Taking the rotationally symmetric surface as an example, its surface equation is:

[0124]

[0125] Where c is the radius of curvature of the rotationally symmetric surface, ρ 2 =x 2 +y 2 is the square of the distance between any point on the surface and the optical axis. When the sum of the polynomials in the formula is not zero, the surface type is aspherical. i is a high-order aspheric coefficient; when the sum of all terms is zero, it represents a rotationally symmetric surface, and the surface shape is determined by the coefficient k.

[0126] After obtaining the diffuse spot after the light source passes through the optical system, the resolution of a single PSF is set, which is selected as 64×64 here. The diffuse spot is discretized and normalized to obtain the corresponding PSF.

[0127] The third step is to adjust the curvature radius, thickness, and refractive index distribution of each lens in the double-Gauss lens based on the material properties and processing tolerances of the camera lens. Here, the curvature radius tolerance is set to ±0.01mm, the thickness tolerance is ±0.05mm, and the refractive index tolerance is ±0.01. The key parameters of each lens shown in Table 1 are randomly perturbed within their tolerance ranges. Positional deviations are also introduced, assuming a deviation range of ±0.02mm along the optical axis to simulate displacement during assembly. Ultimately, 10 sets of optical system models with different errors are generated: two sets of models with perturbed curvature radius, two sets of models with perturbed thickness, two sets of models with perturbed refractive index, and four sets of models with perturbed lens position.

[0128] Multiple PSF calculations were performed for the 10 different optical system models generated by the aforementioned machining errors. Weighting factors were then assigned to the PSFs obtained from these calculations for weighted averaging. These weighting factors were set based on the frequency or severity of the machining errors. The weighting factors for curvature radius perturbation, thickness perturbation, refractive index perturbation, and position perturbation were set to 0.3, 0.2, 0.2, and 0.3, respectively. Each PSF was multiplied by the corresponding weighting factor and added together to obtain the PSF that comprehensively accounts for the machining errors. This PSF served as the convolution kernel for subsequent image processing.

[0129] The fourth step is to form a PSF array diagram based on the distribution of the main light according to the comprehensive PSF of 5×5 sampling field points, as shown in the following figure: Figure 3 As shown, each PSF is convolved with the original image to obtain a set of degraded images under a single field of view. By means of interpolation, a two-dimensional SINC function is used here to fuse the image set to obtain the overall degraded image I under different fields of view of the optical system. A Expressed as:

[0130]

[0131] The fifth step is to set a grid of 640×480 points in the object space of the double Gaussian lens. Each grid point corresponds to a pixel value of the degraded image, and the spacing between the grid points is and The distribution of grid points on the image plane is obtained by ray tracing. By means of interpolation, the nearest neighbor interpolation method is sampled here, and the pixel values ​​of the corresponding grid points are assigned to the pixel units of the final imaging effect diagram to simulate the received image effect at the detector end of the final detection system, that is, the imaging effect of the camera, and obtain an imaging quality simulation diagram including the deformation effect.

[0132] Comparing the image degradation effects calculated by this application with the results of Zemax image simulation, the image degradation effects calculated by this application are basically consistent with Zemax in both overall imaging and local magnification. To quantitatively evaluate the reliability of the degraded image calculation results, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are also used to evaluate the similarity between the degraded image and the reference image:

[0133] Table 2 Quantitative comparison of Zemax image simulation results and image degradation results of this application

[0134] Solving Object Double Gaussian lens PSNR / SSIM Zemax 15.7862 / 0.6930 This application 15.9057 / 0.6913 Relative value 26.8894 / 0.9131

[0135] As can be seen from Table 2, the PSNR of the results obtained based on the calculation method described in this application and the Zemax image simulation results are greater than 26, and the SSIM is greater than 0.91. It can be considered that the simulation results of the two are basically consistent, indicating that the calculation results of the imaging quality simulation method of the rotationally symmetric optical system based on ray tracing described in this application are reliable.

[0136] Using the above-mentioned imaging quality simulation method, the obtained imaging effect simulation diagram is very close to the accuracy of existing simulation software. Compared with existing simulation software, it not only has comparable simulation accuracy and efficiency, but also has higher flexibility in describing the light source model. It has good universality and can be widely used in the performance evaluation and optimization design of various imaging systems. In addition, it can be combined with experimental measurement methods to apply the simulation structure to the analysis of imaging differences caused by lens materials, processing tolerances, etc.

[0137] Reference Figure 4 , the present application also provides an imaging simulation device based on a point spread function, comprising:

[0138] A function determination module 410 is used to establish a light source model corresponding to the image captured by the optical system, and obtain a point spread function of a point light source in the light source model after passing through the optical system;

[0139] A model updating module 420 is configured to update the geometric model of the optical system according to the lens material and processing errors of the optical system, obtain optical system models with different errors, and acquire error point spread functions under different optical system models;

[0140] The weighted averaging module 430 is used to perform weighted averaging on each error point spread function according to a preset weight to obtain a comprehensive point spread function;

[0141] a convolution module 440 for forming an array diagram from the integrated point spread function according to light distribution, and convolving the array diagram with the captured image to obtain a degraded image that simulates a blurring effect caused by aberration;

[0142] The distortion simulation module 450 is configured to apply the distortion model of the optical system to the captured image to obtain a deformation effect image of simulated distortion, and implement imaging simulation of the optical system based on the deformation effect image and the degraded image.

[0143] It is understandable that the detailed functional implementation of each of the above units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0144] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method and will not be repeated here.

[0145] Reference Figure 5 Based on the methods in the above embodiments, an embodiment of the present application provides an electronic device, which may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call logic instructions in the memory 530 to execute the methods in the above embodiments.

[0146] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0147] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0148] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0149] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0150] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0151] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0152] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

[0153] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. An imaging simulation method based on point spread function, characterized in that: include: Establishing a light source model corresponding to the image captured by the optical system, and obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system; updating the geometric model of the optical system according to the lens material and processing errors of the optical system to obtain optical system models with different errors, and acquiring error point spread functions under different optical system models; Perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function; forming an array graph of the integrated point spread function according to light distribution, and convolving the array graph with the captured image to obtain a degraded image that simulates a blurring effect caused by aberration; The distortion model of the optical system is applied to the captured image to obtain a deformation effect image simulating the distortion, and the imaging simulation of the optical system is realized based on the deformation effect image and the degraded image.

2. The imaging simulation method based on point spread function according to claim 1, characterized in that: The updating of the geometric model includes: Updating the geometric model according to the curvature radius, thickness, refractive index distribution of each lens of the optical system, and microscopic irregularities and positional deviations of the lens surface; The parameters of each lens are randomly perturbed within the tolerance range to obtain optical system models with different errors.

3. The imaging simulation method based on point spread function according to claim 1, characterized in that: The step of obtaining a point spread function of a point light source of a light source model after being acted upon by the optical system includes: Dividing the captured image into blocks to obtain a plurality of blocks; The center point of each field of view block is used as the sampling point of the current field of view range; Performing ray tracing calculation on the optical system using the sampling points to obtain a diffuse spot after the point light source passes through the optical system; The point spread function is determined based on the diffuse speckle.

4. The imaging simulation method based on point spread function according to claim 3, characterized in that: The method of performing ray tracing calculation on the optical system using sampling points to obtain a diffuse spot of the point light source after passing through the optical system includes: Determine the light path of the sampling point light through the optical system and determine the coordinates of the intersection point formed on the imaging plane; Perform pixel attribution calculation on the intersection coordinates to determine the image plane discrete pixels of each intersection coordinate; The number of received intersections of the discrete pixels on the image plane is counted, the number of intersections is mapped into image pixel values, the image pixel values ​​are normalized, and the diffuse spots are obtained according to the processed pixel image.

5. The imaging simulation method based on point spread function according to claim 1, characterized in that: Perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function, including: Assigning different weights to each of the error point spread functions according to a preset weighting factor, and weighting different error point spread functions of the same sampling point to obtain a comprehensive spread function after comprehensively considering the processing error; The weighting factor is determined based on the frequency or severity of the error.

6. The imaging simulation method based on point spread function according to claim 1, characterized in that: The convolving the array graph with the captured image to obtain a degraded image with a blurred effect caused by simulated aberrations comprises: Convolve each integrated point spread function with the original image to obtain a set of degraded images under a single field of view; The degraded image set is fused using an interpolation method to obtain degraded images under different fields of view of the optical system.

7. The imaging simulation method based on point spread function according to claim 1, characterized in that: Applying the distortion model of the optical system to the captured image to obtain a deformation effect image simulating the distortion includes: A distortion model of the optical system is established, pixel values ​​of the captured image are obtained using an interpolation method, and a deformation effect caused by the distortion is simulated according to the pixel values ​​to obtain a deformation effect image.

8. An imaging simulation device based on a point spread function, characterized in that: include: A function determination module is used to establish a light source model corresponding to the image captured by the optical system, and obtain a point spread function of a point light source of the light source model after passing through the optical system; A model updating module is used to update the geometric model of the optical system according to the lens material and processing error of the optical system, obtain optical system models with different errors, and obtain error point spread functions under different optical system models; The weighted average module is used to perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function; a convolution module, configured to form an array diagram from the integrated point spread function according to light distribution, and convolve the array diagram with the captured image to obtain a degraded image that simulates a blurring effect caused by aberration; The distortion simulation module is used to apply the distortion model of the optical system to the captured image to obtain a deformation effect image of simulated distortion, and realize imaging simulation of the optical system based on the deformation effect image and the degraded image.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images