Imaging simulation method and device based on point spread function and electronic equipment

Abstract

The invention belongs to the field of image processing, and particularly discloses an imaging simulation method and device based on a point spread function and electronic equipment, and the method comprises the steps: building a light source model corresponding to an image shot by an optical system, and obtaining the point spread function of a point light source of the light source model after the action of the optical system; obtaining error point spread functions under different optical system models; performing weighted averaging on each error point spread function according to a preset weight to obtain a comprehensive point spread function; forming an array diagram by the comprehensive point spread function according to light distribution, and carrying out convolution on the array diagram and a shot image to obtain a degraded image simulating a blurring effect caused by aberration; and applying the distortion model of the optical system to the shot image to obtain a distortion-simulated deformation effect image, and realizing imaging simulation of the optical system based on the deformation effect image and the degraded image. According to the invention, the imaging accuracy and efficiency can be improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing, and more specifically, to an imaging simulation method, device and electronic equipment based on a point spread function. Background Art

[0002] Image quality is a critical indicator in various imaging systems, such as optical microscopes, astronomical telescopes, digital cameras, and medical imaging equipment. Image quality directly affects the accuracy and effectiveness of observation, analysis, and diagnosis using these systems. Traditional methods for evaluating image quality mainly include modulation transfer function (MTF), signal-to-noise ratio (SNR), etc., but these methods often require complex experimental equipment and measurement methods, and in some cases cannot accurately reflect the comprehensive performance of the imaging system.

[0003] At present, some research results have proposed methods that can already evaluate imaging quality, but most of these methods are concentrated on specific types of imaging systems, such as optical microscopes or astronomical telescopes, and mostly focus on theoretical analysis, lacking specific simulation steps and implementation methods; in addition, in complex optical imaging systems, traditional experimental methods require complex experimental equipment and measurement methods, and the existing simulation platforms cannot fully and accurately reflect the performance of the imaging system. Therefore, the accuracy of the simulation results and the simulation efficiency of the current imaging simulation methods still have certain defects.

[0004] Therefore, how to improve the accuracy and efficiency of imaging quality is an urgent problem to be solved. Summary of the invention

[0005] In view of the defects of the prior art, the purpose of the present application is to provide an imaging simulation method, device and electronic equipment based on a point spread function, aiming to solve the current problems of poor accuracy and efficiency of imaging simulation.

[0006] In a first aspect, the present application provides an imaging simulation method based on a point spread function, comprising:

[0007] Establishing a light source model corresponding to the image captured by the optical system, and obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system;

[0008] According to the lens material and processing error of the optical system, the geometric model of the optical system is updated to obtain optical system models with different errors, and the error point spread functions under different optical system models are obtained;

[0009] Perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function;

[0010] The integrated point spread function forms an array diagram according to light distribution, and convolves the array diagram with the captured image to obtain a degraded image that simulates the blur effect caused by the aberration;

[0011] The distortion model of the optical system is applied to the captured image to obtain a deformation effect image simulating the distortion, and the imaging simulation of the optical system is realized based on the deformation effect image and the degraded image.

[0012] The embodiment of the present application can more accurately simulate the imaging effect of the actual optical system by comprehensively considering factors such as the material, processing error, aberration and distortion of the optical system, thereby improving the accuracy and efficiency of the imaging simulation, and by weighted averaging the point spread function under multiple errors, it can reflect the comprehensive impact of different errors, avoiding the deviation that may be caused by considering only a single error. Through the application of convolution operations and distortion models, images that are closer to the imaging effect of the real optical system can be generated.

[0013] Optionally, the updating of the geometric model includes:

[0014] The geometric model is updated according to the curvature radius, thickness, refractive index distribution of each lens of the optical system, and microscopic irregularities and position deviations of the lens surface;

[0015] The parameters of each lens are randomly perturbed within the tolerance range to obtain optical system models with different errors.

[0016] Optionally, the step of obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system includes:

[0017] Dividing the captured image into blocks to obtain a plurality of blocks;

[0018] The center point of each field of view block is used as the sampling point of the current field of view range;

[0019] Performing ray tracing calculation on the optical system using the sampling points to obtain a diffuse spot after the point light source passes through the optical system;

[0020] The point spread function is determined based on the diffuse speckle.

[0021] Optionally, the performing ray tracing calculation on the optical system using the sampling points to obtain the diffuse spot of the point light source after passing through the optical system includes:

[0022] Determine the light path of the sampling point light through the optical system, and determine the coordinates of the intersection point formed on the imaging plane;

[0023] Perform pixel attribution calculation on the intersection coordinates to determine the image plane discrete pixels of each intersection coordinate;

[0024] The number of received intersections of the discrete pixels on the image plane is counted, the number of intersections is mapped into image pixel values, the image pixel values ​​are normalized, and the diffuse spots are obtained according to the processed pixel images.

[0025] Optionally, each error point spread function is weighted averaged according to a preset weight to obtain a comprehensive point spread function, including:

[0026] Assigning different weights to each of the error point spread functions according to a preset weighting factor, weighting different error point spread functions of the same sampling point, and obtaining a comprehensive spread function after comprehensively considering the processing error;

[0027] The weighting factor is determined based on the frequency or severity of the error.

[0028] Optionally, the convolving the array image with the captured image to obtain a degraded image simulating a blur effect caused by aberration includes:

[0029] Convolve each integrated point spread function with the original image to obtain a set of degraded images under a single field of view;

[0030] The degraded image set is fused using an interpolation method to obtain degraded images under different fields of view of the optical system.

[0031] Optionally, applying the distortion model of the optical system to the captured image to obtain a deformation effect image simulating the distortion includes:

[0032] A distortion model of the optical system is established, pixel values ​​of the captured image are obtained by using an interpolation method, and a deformation effect caused by the distortion is simulated according to the pixel values ​​to obtain a deformation effect image.

[0033] In a second aspect, the present application further provides an imaging simulation device based on a point spread function, comprising:

[0034] A function determination module, used to establish a light source model corresponding to the image captured by the optical system, and obtain a point spread function of a point light source of the light source model after being acted upon by the optical system;

[0035] A model updating module, used to update the geometric model of the optical system according to the lens material and processing error of the optical system, obtain optical system models with different errors, and obtain error point spread functions under different optical system models;

[0036] A weighted average module is used to perform weighted average on each error point spread function according to a preset weight to obtain a comprehensive point spread function;

[0037] A convolution module, used to form an array diagram of the integrated point spread function according to the light distribution, and convolve the array diagram with the captured image to obtain a degraded image with a blur effect caused by simulated aberration;

[0038] The distortion simulation module is used to apply the distortion model of the optical system to the captured image to obtain a deformation effect image of simulated distortion, and realize imaging simulation of the optical system based on the deformation effect image and the degraded image.

[0039] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0041] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0042] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0043] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0044] (1) Through in-depth analysis of the light source model and point spread function, this application can more accurately evaluate the imaging performance of the optical system under different conditions, and can improve the accuracy and efficiency of imaging simulation by clarifying the impact of various factors (such as lens design, material properties, processing errors, etc.) on the final image quality. By weighting the updated model to obtain a comprehensive point spread function, the aberrations in the optical system can be more accurately simulated, further improving the accuracy of imaging quality prediction.

[0045] (2) By establishing a comprehensive point spread function and performing convolution operations, the present application can generate more realistic degraded images, predict image blur and distortion phenomena that may occur under actual shooting conditions, comprehensively analyze and improve image quality, and enhance the reliability of image processing and analysis.

[0046] (3) This application can effectively correct the distortion in the image by establishing a distortion model of the optical system and applying the interpolation method to calculate the pixel value, which can improve the accuracy and clarity of the image and make it closer to the actual scene. After the image is distorted, subsequent image analysis and processing can be performed more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the flow of the imaging simulation method based on the point spread function provided in an embodiment of the present application;

[0048] Figure 2 is a schematic diagram of the structure of a double Gaussian lens provided in an embodiment of the present application;

[0049] Figure 3 is a point spread function simulation result diagram provided in an embodiment of the present application;

[0050] Figure 4 is a schematic structural diagram of an imaging simulation device based on a point spread function provided in an embodiment of the present application;

[0051] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0054] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0055] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0056] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0057] Next, the technical solutions provided in the embodiments of the present application are introduced.

[0058] Reference Figure 1 , the present application provides an imaging simulation method based on a point spread function, comprising:

[0059] S101. Establishing a light source model corresponding to the image captured by the optical system, and obtaining a point spread function of the point light source of the light source model after being acted upon by the optical system;

[0060] S102. According to the lens material and processing error of the optical system, the geometric model of the optical system is updated to obtain optical system models with different errors, and the error point spread functions under different optical system models are obtained;

[0061] S103. Perform weighted averaging on each error point spread function according to a preset weight to obtain a comprehensive point spread function;

[0062] S104. Forming an array diagram of the integrated point spread function according to the light distribution, convolving the array diagram with the captured image to obtain a degraded image with a blur effect caused by simulated aberration;

[0063] S105. Apply the distortion model of the optical system to the captured image to obtain a deformation effect image simulating the distortion, and implement imaging simulation of the optical system based on the deformation effect image and the degraded image.

[0064] First, a light source model is established and a point spread function (PSF) is obtained through the above S101.

[0065] A light source model is established for the scene photographed by the optical system. This model is usually based on a collection of object points or point light sources to simulate how the optical system processes light emitted from different positions.

[0066] Specifically, according to the size and resolution requirements of the actual shooting scene, as well as the wavelength distribution, a light source model of three colors of red R, green G, and blue B for the point spread function is established.

[0067] In general, the captured scene is regarded as an image, and the pixel unit of the image is the basic light-emitting unit. For a color image, it contains three light-emitting units, R, G, and B, and the light source model of the corresponding color needs to be defined according to their respective wavelength distributions. When the working distance is long, each pixel can be regarded as a point light source, that is, when defining the starting information of the light, the starting positions of all light rays are consistent, and the direction of the light follows a specific angular distribution, such as the Lambertian distribution. The wavelength of the light is defined by the spectral distribution of the shooting scene, thereby completing the establishment of the corresponding light source model. When the area of ​​the pixel cannot be ignored, a small number of pixel units can be selected by experimental means to measure their angular distribution, and the area of ​​the light source can be defined according to the size of the pixel unit to establish a light source model that is closer to the actual scene.

[0068] Furthermore, the point spread function (PSF) is obtained. The point spread function is the light intensity distribution formed on the image plane when the optical system images a single point light source. Through ray tracing or optical simulation, the point spread function (PSF) of the point light source in the light source model after passing through the optical system is calculated.

[0069] The geometric model of the optical system is updated through S102. The lens material and processing errors are taken into consideration, and the initial optical system geometric model is adjusted according to the material properties (such as refractive index) and processing errors (such as deviations of lens curvature and thickness) of the actual optical system to generate an optical system model with different errors.

[0070] Then, for each updated optical system model, the corresponding point spread function is recalculated to simulate the imaging quality changes caused by manufacturing errors or material inhomogeneity.

[0071] Weighted average error point spread function is performed in step S103. All calculated error point spread functions are weighted averaged according to preset weights to obtain a comprehensive point spread function.

[0072] Furthermore, the blur effect of aberration is simulated in S104. The integrated point spread function is formed into an array diagram according to the light distribution, and the array diagram is convolved with the captured image to simulate the blur effect caused by the aberration of the optical system and generate a degraded image.

[0073] It should be noted that the convolution process in this embodiment is essentially to apply the PSF to the entire image to simulate the blur when the optical system is imaging.

[0074] Finally, according to the distortion characteristics of the optical system, the captured image is deformed to generate a deformation effect image that simulates the distortion. Distortion usually manifests itself as stretching, compression or bending of the image edge.

[0075] By combining the deformation effect image and the degraded image, the imaging simulation of the optical system is completed, simulating the image effects that may be produced by the real optical system.

[0076] The embodiment of the present application can more accurately simulate the imaging effect of the actual optical system by comprehensively considering factors such as the material, processing error, aberration and distortion of the optical system, thereby improving the accuracy and efficiency of the imaging simulation, and by weighted averaging the point spread function under multiple errors, it can reflect the comprehensive impact of different errors, avoiding the deviation that may be caused by considering only a single error. Through the application of convolution operations and distortion models, images that are closer to the imaging effect of the real optical system can be generated.

[0077] Optionally, the updating of the geometric model includes:

[0078] The geometric model is updated according to the curvature radius, thickness, refractive index distribution of each lens of the optical system, and the microscopic irregularity and position deviation of the lens surface;

[0079] The parameters of each lens are randomly perturbed within the tolerance range to obtain optical system models with different errors.

[0080] Optionally, the step of obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system includes:

[0081] Dividing the captured image into blocks to obtain a plurality of blocks;

[0082] The center point of each field of view block is used as the sampling point of the current field of view range;

[0083] Performing ray tracing calculation on the optical system using the sampling points to obtain a diffuse spot after the point light source passes through the optical system;

[0084] The point spread function is determined based on the diffuse speckle.

[0085] Specifically, in this embodiment, the field of view of the captured scene image is divided into blocks, the center of each field of view block is selected as the sampling point of the current field of view range, and the optical system is subjected to ray tracing calculation to obtain the diffuse spot after the point light source passes through the optical system, that is, the PSF;

[0086] For optical imaging systems, when a point light source passes through the optical system, it is no longer focused on one point on the image plane due to various aberrations and diffraction effects, but appears as a diffuse spot with a certain area distribution. PSF provides the intensity distribution of a single point light source on the imaging plane through the optical system, describes the distribution of the spatial response of the system when imaging a point source or a small object, contains all possible aberrations and defocus information in the system, and is crucial for understanding and improving image quality.

[0087] This application provides a PSF calculation method based on ray tracing. This method is based on the traditional geometric ray tracing algorithm. By tracing a large number of light rays through the actual optical system and recording their intersection points on the imaging plane, the PSF of the entire system is constructed. The specific process is as follows:

[0088] Determine the light path of the sampling point light through the optical system, and determine the coordinates of the intersection point formed on the imaging plane;

[0089] Perform pixel attribution calculation on the intersection coordinates to determine the image plane discrete pixels of each intersection coordinate;

[0090] The number of received intersections of the discrete pixels on the image plane is counted, the number of intersections is mapped into image pixel values, the image pixel values ​​are normalized, and the diffuse spots are obtained according to the processed pixel images.

[0091] Specifically, the light emitted from the object point (sampling point) is refracted and reflected by the optical system to form a unique intersection point on the imaging plane. The coordinates of the intersection point can be expressed as (x i ,y i ), where i corresponds to the number of the light rays with different azimuth angles starting from the object point. Since the image plane is composed of discrete pixels, the pixel attribution calculation needs to be performed for the image plane light ray coordinates obtained by tracing.

[0092] After calculating all the light paths starting from the object point, the distribution of their intersection points on the discrete pixels of the image plane is quantitatively counted. By calculating and accumulating the number of intersection points received by each pixel, the number of rays for each pixel is obtained, and these values ​​are mapped to the corresponding pixel values. In order to facilitate intuitive comparison, appropriate normalization is required to ensure that the sum of the values ​​in the entire pixel matrix is ​​1.

[0093] This process can be formalized using the following formula:

[0094]

[0095] In the above formula, psf p,q It represents the normalized pixel value at the pixel number (p,q) in the diffuse speckle distribution map composed of discrete pixels. a and b represent the two-dimensional coordinates corresponding to the center of the pixel. i is the number of the object sampling ray, m is the number of the object sampling rays, n is the number of sampling rays detected on the image plane, and the round function performs rounding operations.

[0096] Optionally, each error point spread function is weighted averaged according to a preset weight to obtain a comprehensive point spread function, including:

[0097] Assigning different weights to each of the error point spread functions according to a preset weighting factor, weighting different error point spread functions of the same sampling point, and obtaining a comprehensive spread function after comprehensively considering the processing error;

[0098] The weighting factor is determined based on the frequency or severity of the error.

[0099] Specifically, this embodiment modifies the model of the optical system according to the lens material and processing tolerance, repeatedly calculates different PSFs of the same sampling point, and weightedly adds the different PSFs according to the user-defined weights to simulate the blurring effect caused by aberrations, and then analyzes the imaging differences caused by processing errors;

[0100] Based on the material properties and processing tolerances of the camera lens, the geometric model of the optical system is adjusted. These adjustments may include parameters such as the radius of curvature, thickness, and refractive index distribution of each lens in the system. In addition, non-ideal factors that may arise during the processing process must also be considered, such as microscopic irregularities and positional deviations on the lens surface. In order to simulate these errors more accurately, the key parameters of each lens are randomly perturbed within its tolerance range to generate a series of lens models with different errors. For each adjusted optical system model, the PSF of the same sampling point is recalculated using the ray tracing algorithm. The specific steps refer to the PSF acquisition process mentioned above.

[0101] In order to simulate the blurring effect caused by aberrations in the actual imaging process, the PSFs obtained by multiple calculations are weighted averaged. The specific steps are as follows: Different weights are assigned to each PSF according to pre-set weighting factors (these factors can be set based on the frequency or severity of the error). Different PSFs at the same sampling point are weighted and added to obtain the PSF after comprehensive consideration of the processing error.

[0102] Optionally, the convolving the array image with the captured image to obtain a degraded image simulating a blur effect caused by aberration includes:

[0103] Convolve each integrated point spread function with the original image to obtain a set of degraded images under a single field of view;

[0104] The degraded image set is fused using an interpolation method to obtain degraded images under different fields of view of the optical system.

[0105] Specifically, in this embodiment, the PSFs of all the sampled field of view points after weighted processing are formed into a PSF array diagram according to the main ray distribution, and the PSF array is convolved with the original image to simulate the blur effect caused by the aberration.

[0106] PSF describes the diffusion effect of a point light source passing through an optical system. It is the response of the optical system to the imaging of a point light source. After obtaining the PSF of an optical system under a specific field of view based on the traditional ray tracing method, it is convolved with the scene image to obtain a degraded image under the aberration of this field of view. Applying distortion processing to the degraded image can add the deformation effect caused by the aberration to the final imaging simulation effect diagram.

[0107] For actual optical systems, their PSF is related to the field of view. An optical system may obtain a very small diffuse spot at its central field of view, while at the edge of the field of view, due to the cumulative effect of various aberrations, the diffuse spot may be larger, resulting in a decrease in image quality. Strictly speaking, in order to obtain accurate image degradation results, it is necessary to calculate the corresponding PSF for each pixel position of the original image on the object side, but this method will lead to a decrease in computational efficiency. Since the PSF distribution of adjacent pixels is similar, sampling and interpolation methods can be used to reduce the amount of calculation. Define the physical size of the original image, uniformly sample X*Y grid points on the original image, that is, divide the entire field of view into X*Y field of view blocks, and each grid point corresponds to the center of a field of view block. The PSF of the corresponding grid point is obtained by the method described in the previous section, and the original image is convolved with the PSF of all grid points to obtain a set of degraded images under a single field of view. The image set is fused by interpolation to obtain the overall degraded image under different fields of view of the optical system. Here, a two-dimensional SINC function is used for interpolation to consider the degraded image I affected by aberrations. A The calculation can be expressed mathematically as:

[0108]

[0109] In the formula, S a,b The PSF is the value of the two-dimensional SINC function with the sampling point indexed as (a, b) as the center and the width of the corresponding field of view block as the unit length. a,b is the PSF corresponding to the sampling point with index (a, b), I s is the original image.

[0110] Optionally, applying the distortion model of the optical system to the captured image to obtain a deformation effect image simulating the distortion includes:

[0111] A distortion model of the optical system is established, pixel values ​​of the captured image are obtained by using an interpolation method, and a deformation effect caused by the distortion is simulated according to the pixel values ​​to obtain a deformation effect image.

[0112] A distortion model is established for the imaging optical system, and the pixel values ​​of the target image are calculated using the interpolation method to simulate the deformation effect caused by the distortion;

[0113] Specifically, since the degraded image obtained by convolving the original image with the PSF only reflects the image blurring phenomenon caused by the aberration of the optical system, in the actual imaging process, in addition to the image blurring caused by the aberration, the image distortion caused by the distortion is also a common factor affecting the image quality. Therefore, in order to more comprehensively simulate the imaging performance of the optical system, this embodiment needs to obtain the distortion model of the optical system and apply this model to the image without considering the influence of the distortion. This step usually involves the image remapping process, that is, according to the distortion model, the position of each pixel in the image is adjusted to simulate the image deformation effect caused by the distortion.

[0114] In the object space, a number of grid points that is consistent with the arrangement of image pixels is set. After the distortion of the optical system, the distribution of the grid points changes, and a coordinate mapping relationship between the two is established, that is, a distortion model is established. When applied to the original image, a simulated distortion result image can be obtained. Since most of the positions of the distorted grid points will not fall exactly on the pixels of the original image, interpolation calculations are required when assigning values ​​to the pixels of the distortion result image. In this embodiment, the nearest neighbor interpolation method is used to assign values ​​to non-integer pixels on the image plane.

[0115] The present application is described in detail below in conjunction with a specific embodiment. This embodiment takes a double Gaussian lens in the Zemax lens database as an example and is described in detail as follows:

[0116] like Figure 2 As shown, the double Gauss lens of the implementation example of the present application includes 2 double convex lenses ①⑥, 2 plano-convex lenses ②⑤ and 2 plano-concave lenses ③④. The lens data and specific structural distances in the example are shown in Table 1. The simulation uses a scene image with a resolution of 640×480, the physical size of the image is 200mm×150mm, and the field of view height of the center of the scene image is (0,0). The distance between the image and the system is set to 300mm, the image plane is 94.071mm away from the rear surface of the last lens of the system, the system wavelength is set to three visible light wavelengths of R (656.3nm), G (587.6nm), and B (486.1nm), and the G light is set to the main wavelength, the entrance pupil diameter is 33.33mm, the distance between the scene image and lens 1 is 300mm, and the aperture stop of the system is set 14.253mm behind lens 3. The simulation results are compared and analyzed with the simulation results of Zemax to verify the accuracy of the imaging quality simulation calculation results.

[0117] Table 1

[0118]

[0119] The first step is to establish a light source model for the point spread function according to the size and resolution requirements of the actual shooting scene. The simulation uses a scene image with a resolution of 640×480, an image physical size of 200mm×150mm, and a working distance of 300mm. The corresponding light source model is established with the pixel unit as the light source. Here, it is established as a point light source model, and the angular distribution of the light is set to fill the entrance pupil surface. The entrance pupil surface of the lens shown in Table 1 is located 58.93976mm behind the front surface of lens ①. (x0, y0, z0) represents the starting position of the light, and (k, l, m) represents the starting direction of the light.

[0120] In the second step, the field of view of the captured scene image is divided into 5×5 blocks, and the center of each field of view block is selected as the sampling point of the current field of view range, that is, the starting position (x0, y0, z0) of the light in the light source model in the previous step. The number of sampling rays is selected as 128×128, and ray tracing calculations are performed on the optical system.

[0121] Light propagates in a straight line in an isotropic homogeneous medium. Any point (x, y, z) on the light can be expressed by known parameters as:

[0122] x=x0+rk,y=y0+rl,z=z0+rm

[0123] When the light propagates to the lens, the intersection direction of the light and the lens surface is solved according to the Newton iteration method, and the propagation direction of the light after passing through the lens surface is solved according to the law of refraction and reflection. Taking the rotationally symmetric surface as an example, its surface equation is:

[0124]

[0125] Where c is the radius of curvature of the rotationally symmetric surface, ρ 2 =x 2 +y 2 is the square of the distance between any point on the surface and the optical axis. When the polynomial sum in the formula is not zero, the surface type is aspherical, a i is the high-order aspheric coefficient; when the sum of all terms is zero, it represents a rotationally symmetric surface, and the surface shape is determined by the coefficient k.

[0126] After obtaining the diffuse spot after the light source passes through the optical system, the resolution of a single PSF is set, which is selected as 64×64 here, and the diffuse spot is discretized and normalized to obtain the corresponding PSF.

[0127] The third step is to adjust the curvature radius, thickness, and refractive index distribution of each lens in the double Gauss lens based on the material properties and processing tolerances of the camera lens. Here, the curvature radius tolerance of the lens is set to ±0.01mm, the thickness tolerance is ±0.05mm, and the refractive index tolerance is ±0.01. The key parameters of each lens shown in Table 1 are randomly perturbed within their tolerance range, and the position deviation of the lens is introduced. It is assumed that the deviation range of the lens along the optical axis is ±0.02mm to simulate the displacement during the assembly process. Finally, 10 groups of optical system models with different errors are generated, including 2 groups of models with curvature radius perturbation, 2 groups of models with thickness perturbation, 2 groups of models with refractive index perturbation, and 4 groups of models with lens position perturbation.

[0128] The PSF calculations are performed multiple times for the 10 different optical system models generated by the above-mentioned processing errors, and the weighting factors used for weighted averaging are set for the PSFs obtained by the multiple calculations, and weighted averaging is performed. These factors are set based on the frequency or severity of the processing errors. Here, the weighting factors of the curvature radius perturbation, thickness perturbation, refractive index perturbation, and position perturbation are set to 0.3, 0.2, 0.2, and 0.3, respectively. Each PSF is multiplied by the corresponding weighting factor and added to obtain the PSF after comprehensive consideration of the processing errors, which is used as the convolution kernel for subsequent image processing.

[0129] The fourth step is to form a PSF array diagram based on the distribution of the main light rays for the comprehensive PSF of the 5×5 sampling field points, as shown in Figure 3 As shown, each PSF is convolved with the original image to obtain a set of degraded images under a single field of view. By means of interpolation, a two-dimensional SINC function is used here to fuse the image set to obtain the overall degraded image I under different fields of view of the optical system. A It is expressed as:

[0130]

[0131] The fifth step is to set a grid of 640×480 points in the object space of the double Gaussian lens. Each grid point corresponds to a pixel value of the degraded image, and the spacing between the grid points is and The distribution of grid points on the image plane is obtained by ray tracing. By means of interpolation, the nearest neighbor interpolation method is sampled here, and the pixel values ​​of the corresponding grid points are assigned to the pixel units of the final imaging effect diagram to simulate the received image effect at the detector end of the final detection system, that is, the imaging effect of the camera, and obtain an imaging quality simulation diagram including the deformation effect.

[0132] Comparing the image degradation effect calculated by this application with the result of Zemax image simulation, the image degradation effect calculated by this application is basically consistent with Zemax in terms of overall imaging and local magnification. In order to quantitatively evaluate the reliability of the calculation results of the degraded image, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are also used to evaluate the similarity between the degraded image and the reference image:

[0133] Table 2 Quantitative comparison of Zemax image simulation results and image degradation results of this application

[0134] Solving Object Double Gaussian lens PSNR / SSIM Zemax 15.7862 / 0.6930 This application 15.9057 / 0.6913 Relative value 26.8894 / 0.9131

[0135] As can be seen from Table 2, the PSNR of the results obtained based on the calculation method described in the present application and the Zemax image simulation results are greater than 26, and the SSIM is greater than 0.91. It can be considered that the simulation results of the two are basically consistent, indicating that the calculation results of the imaging quality simulation method of the rotationally symmetric optical system based on ray tracing described in the present application are reliable.

[0136] Using the above-mentioned imaging quality simulation method, the obtained imaging effect simulation diagram is very close to the accuracy of the existing simulation software. Compared with the existing simulation software, it not only has comparable simulation accuracy and efficiency, but also has higher flexibility in the description of the light source model. It has good universality and can be widely used in the performance evaluation and optimization design of various imaging systems. It can also be combined with experimental measurement methods to apply the simulation structure to the analysis of imaging differences caused by lens materials, processing tolerances, etc.

[0137] Reference Figure 4 The present application also provides an imaging simulation device based on a point spread function, comprising:

[0138] A function determination module 410 is used to establish a light source model corresponding to the image captured by the optical system, and obtain a point spread function of a point light source of the light source model after being acted upon by the optical system;

[0139] A model updating module 420 is used to update the geometric model of the optical system according to the lens material and processing errors of the optical system, obtain optical system models with different errors, and obtain error point spread functions under different optical system models;

[0140] The weighted average module 430 is used to perform weighted average on each error point spread function according to a preset weight to obtain a comprehensive point spread function;

[0141] A convolution module 440 is used to form an array diagram of the integrated point spread function according to the light distribution, and convolve the array diagram with the captured image to obtain a degraded image with a blur effect caused by simulated aberration;

[0142] The distortion simulation module 450 is used to apply the distortion model of the optical system to the captured image to obtain a deformation effect image of simulated distortion, and implement imaging simulation of the optical system based on the deformation effect image and the degraded image.

[0143] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0144] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0145] Reference Figure 5 Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, which may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the method in the above embodiment.

[0146] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0147] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0148] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0149] It is understandable that the processor in the embodiment of the present application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0150] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0151] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0152] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0153] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An imaging simulation method based on a point spread function, characterized in that: include: Establishing a light source model corresponding to the image captured by the optical system, and obtaining a point spread function of a point light source of the light source model after being acted upon by the optical system; According to the lens material and processing error of the optical system, the geometric model of the optical system is updated to obtain optical system models with different errors, and the error point spread functions under different optical system models are obtained; Perform weighted averaging on each error point spread function according to preset weights to obtain a comprehensive point spread function; The integrated point spread function forms an array diagram according to light distribution, and convolves the array diagram with the captured image to obtain a degraded image that simulates the blur effect caused by the aberration; The distortion model of the optical system is applied to the captured image to obtain a deformation effect image simulating the distortion, and the imaging simulation of the optical system is realized based on the deformation effect image and the degraded image.

2. The imaging simulation method based on point spread function according to claim 1, characterized in that: The updating of the geometric model includes: The geometric model is updated according to the curvature radius, thickness, refractive index distribution of each lens of the optical system, and microscopic irregularities and position deviations of the lens surface; The parameters of each lens are randomly perturbed within the tolerance range to obtain optical system models with different errors.

3. The imaging simulation method based on point spread function according to claim 1, characterized in that: The point spread function of the point light source of the light source model obtained after being acted upon by the optical system includes: Dividing the captured image into blocks to obtain a plurality of blocks; The center point of each field of view block is used as the sampling point of the current field of view range; Performing ray tracing calculation on the optical system using the sampling points to obtain a diffuse spot after the point light source passes through the optical system; The point spread function is determined based on the diffuse speckle.

4. The imaging simulation method based on point spread function according to claim 3, characterized in that: The method of performing ray tracing calculation on the optical system using the sampling points to obtain the diffuse spot of the point light source after passing through the optical system includes: Determine the light path of the sampling point light through the optical system, and determine the coordinates of the intersection point formed on the imaging plane; Perform pixel attribution calculation on the intersection coordinates to determine the image plane discrete pixels of each intersection coordinate; The number of received intersections of the discrete pixels on the image plane is counted, the number of intersections is mapped into image pixel values, the image pixel values ​​are normalized, and the diffuse spots are obtained according to the processed pixel images.

5. The imaging simulation method based on point spread function according to claim 1, characterized in that: Each error point spread function is weighted averaged according to the preset weights to obtain a comprehensive point spread function, including: Assigning different weights to each of the error point spread functions according to a preset weighting factor, weighting different error point spread functions of the same sampling point, and obtaining a comprehensive spread function after comprehensively considering the processing error; The weighting factor is determined based on the frequency or severity of the error.

6. The imaging simulation method based on point spread function according to claim 1, characterized in that: The step of convolving the array graph with the captured image to obtain a degraded image with a blur effect caused by simulated aberrations comprises: Convolve each integrated point spread function with the original image to obtain a set of degraded images under a single field of view; The degraded image set is fused using an interpolation method to obtain degraded images under different fields of view of the optical system.

7. The imaging simulation method based on point spread function according to claim 1, characterized in that: Applying the distortion model of the optical system to the captured image to obtain a deformation effect image simulating the distortion includes: A distortion model of the optical system is established, pixel values ​​of the captured image are obtained by using an interpolation method, and a deformation effect caused by the distortion is simulated according to the pixel values ​​to obtain a deformation effect image.

8. An imaging simulation device based on a point spread function, characterized in that: include: A function determination module, used to establish a light source model corresponding to the image captured by the optical system, and obtain a point spread function of a point light source of the light source model after being acted upon by the optical system; A model updating module, used to update the geometric model of the optical system according to the lens material and processing error of the optical system, obtain optical system models with different errors, and obtain error point spread functions under different optical system models; A weighted average module is used to perform weighted average on each error point spread function according to a preset weight to obtain a comprehensive point spread function; A convolution module, used to form an array diagram of the integrated point spread function according to the light distribution, and convolve the array diagram with the captured image to obtain a degraded image with a blur effect caused by simulated aberration; The distortion simulation module is used to apply the distortion model of the optical system to the captured image to obtain a deformation effect image of simulated distortion, and realize imaging simulation of the optical system based on the deformation effect image and the degraded image.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images